Halo 3 [Beta] Mod Discussion

[Prey]

dakisback17.. lawl.

It's cool that you guys are making all these tools, but these are just the beta maps and you won't even be able to test mods since the beta expired.

Not that we could anyway.. (excluding you CLuis )

These are not mods. They are making programs for Rendering

No.

Not entirely. H3Core will do more than just that, much more.

Oh, and i just finished writing up a function that gets correct meta sizes (wooo). Seeing as reflexives are all in their 'pools' now instead of like in Halo2 where they could (normally) be found after the tags meta header; you can treat these meta sizes as header sizes, thus making the latter term obsolete.

http://www.homebrewgames.co.uk/xbox/hal ... s_list.txt
These have also all been hardcoded into the app =x

[xzodia]

so its the meta size without chunks?

[Prey]

so its the meta size without chunks?

..not that the meta size before contained the relevant chunks anyway. (Apart from in the off chance)

Edit* Whoops, slight bug in the code made the output contain some wrong sizes, all fixed now though. Link^^ updated.

[Tural]

As many of you know, in the recent weeks the various teams have been working on a complicated matter, model extraction. This has been disputed by many, and promised by others. However, we have to look at the real situation here, and not let our personal feelings interfere with the responsible action that must be taken.

While the people with the know-how to do so have been carefully working on the tools for asset extraction, the public has been waiting somewhat-patiently for these to be released. I regretfully inform you that you're going to have to continue waiting.

Is this a personal attack on anyone? No. This is the decision that has been made to satisfy both the modding community, as well as the folks at Bungie. This isn't all about us. The Bungie Studios employees have worked very hard for many years on these assets, and we are attempting to come in and take those assets from a gift that was given to us, representing an incomplete state. Bungie did not have to supply a beta; they could have done without it. They chose to release one in the rightful expectation that the players would be responsible about it. How do we offer thanks to them for giving it out? By taking their work from them without their permission? Those should not be the actions occurring in a responsible community, such as we expect this to be.

What will be allowed, you ask? Editing, structure, injecting, pretty much anything excluding the extraction of usable models, bitmaps, sounds, and other assets. We will not allow the publishing of tools to extract usable data such as those listed, we will not allow programming code to be posted concerning the extraction or conversion processes, and we will certainly not allow any content being posted. We greatly encourage you to continue work on programs for editing the maps, as well as research on the methods for extracting the aforementioned data.

Primary reasons for this decision vary, so I'll try to lay out the basic ones for you. We don't want people leaking models, pictures, details, and other information about unreleased content in the maps. We also do not want people ripping from the beta maps and using the models elsewhere. You can not deny that this will happen, because it will. Yes, people will still do such stupid things when the final game comes out, but that is after people have had the chance to experience the full game, and, as I've stated, we do not wish to give a big "Fuck you" to Bungie for giving us the beta. There's no reason to really warrant necessity of content extraction. Yes, there are people, including myself, who would like the models and textures for various reasons, like rendering. However, the main reason for even having the beta modding discussion is not to take content from it, it is to help assist modders in learning how Halo 3's technical stuff works, to give them a head-start on modding when the final game comes out. We're not stopping anyone from modding, we're not stopping anyone from researching, we just don't have a reason to need extraction of this content right now, and the negative effect of doing so greatly outweighs the positive.

I'll reference the effect that Halo 2 cheating had on the modding community. Before Halo 2, there was no major issue with modding games such as Halo 1. With Halo 2 modding came a wealth of cheaters. HaloMods had to be shut down while the staff worked to clean up the forums from the trash these people brought. The events of Halo 2 cheating really hurt the modding community's standing with Bungie. We're already on thin ice, so-to-speak, and we're not going to allow this extraction from the beta, because we really don't need it, and we really don't want to have the integrity of the community threatened once again. We've already had a problem with Halo 3 modding on this site, and we sure do not need another situation like that.

This post does not require discussion. The decision has been made.

[antszy101]

Tural, your post has left me a little confused. It seems to me that you are saying that things like bitm extraction and model extraction won't be allowed when the final game comes out. I agree with your point that getting model and bitmap extraction released before the final game comes out may lower the community's reputation, but I don;t understand why it would not be allowed for the final game.

[GametagAeonFlux]

Tural, your post has left me a little confused. It seems to me that you are saying that things like bitm extraction and model extraction won't be allowed when the final game comes out. I agree with your point that getting model and bitmap extraction released before the final game comes out may lower the community's reputation, but I don;t understand why it would not be allowed for the final game.

It will be allowed for the final game. We're not allowing it with the beta.

[Prey]

No extraction! Gosh, what will we do? ..

Oh, and i just finished writing up a function that gets correct meta sizes (wooo). Seeing as reflexives are all in their 'pools' now instead of like in Halo2 where they could (normally) be found after the tags meta header; you can treat these meta sizes as header sizes, thus making the latter term obsolete.

http://www.homebrewgames.co.uk/xbox/hal ... s_list.txt

I think i'll explain now how i actually got these sizes, so to not keep anyone in the dark:

I first have a list of all the meta offsets from all the tags within a map, i also add to this list the offsets of the tagInfoIndexHeader, tagInfoIndex, classIndex and the Index. Once these are added i have the offsets for just about every object that is to be found in the virtual segment of the map, the only offsets i do not have are those of the reflexives.

I loop through my offset list, getting a basic size for each by subtracting from the offset of the object (another meta, or one of the four above) that comes after the current, with the offset of the current.

I now remove the four things i added earlier, they are no longer needed.

Now to loop through all the tagClasses in the map. Note that i also have another list (the 'classList'); this ones stores for us tagClasses with their sizes. I now loop through this list within the loop i just mentioned and try to find the index of the class i am currently on. If i dont find anything, i add the class to the list.

I then loop through the maps tags of the current class, and if a tags size is smaller than what we've currently got in the classList; i update the list with it.

I do this for all the maps and end up with a list of tagClasses that each contain the smallest size found for that particular class. Stage 1 complete.

Now we need to loop through the maps again. Each time, we add those four objects mentioned at the start to a new list of ours, loop through every tag and add its offset to the list too, and scan its meta for reflexives and add the translations taken from those.

Now scanning for reflexives can be quite a risky business, what if picks up something that isn't a reflexive? Well we do know of some limitations to hopefully stop this from happening:

Chunk Count - Checks if this is zero or higher.
Pointer - Subtracts the magic from it and makes sure the result is within the virtual segment of the map. The fact that we have to subtract the magic from the pointer in the first place, as well as the fact that the magic is always a negative number; means that the pointer must be a negative in the first place, so that by subtracting the magic we get a positive. This a good thing as it narrows down even further our entry level integers; a chunk count that is zero or higher followed by a negative, thanks bungie

Lucky for us that it is such a unique structure, i don't think we'll be picking up any strays (hope not anyhow)

So we've lots of offsets now all jumbled up in our list, so obviously its time to sort them. Once done, we can find the sizes of each object quite simply; subtract from the object in front the offset of the current object. For the last object, we subtract from the offset that marks the eof of the virtual segment.

..and finally loop through our classList again and update any classes if a tag is found to have a smaller meta size than what is currently listed.

Once done for all maps, finish off with the code that'll output it all to a nice .txt file.

The Bad News
There are 2 possible things that could mess up this function:

1. Nested reflexives (reflexives inside reflexives) - This function never incorporates them into the equation. I may think of doing it later when i have time, but in all honesty; it will be quite hard - Mostly because the limitation of size becomess quite variable, and thus quite wrong

2. When the function actually scans for reflexives, in the beginning stages it is a fact that the meta sizes will be wrong (too big), and so reflexives that are found may not be part of that tags meta. Not to worry too much though, as at least two of the three outcomes are harmless:

• The reflexive is part of another tags meta - no worries then, we'll just get duplicate items in the list which will make no difference to the overall outcome.
• The reflexive is nested - Even better. having these will make the overall results much more accurate.
• It isnt a reflexive - Uh oh. Well nothing we can do about that. I mentioned earlier how bullet proof the reflex structure is, so we'll just have to hope it holds out

The Good News
The function maps out the position of every reflexive pointed to from the meta. So with a few extra lines of code, i can all this outputted into plugins for the release of H3Core

Christ, i didnt realize i wrote so much, its only like 200 lines of code >_>

[xbox7887]

The bad news
Nested reflexives are pretty much everywhere

[Shalted]

Nested reflexives are pretty tricky to deal with, I have a TON of code that maps out the tags based on info from each map, but it needs debugging, and I'm not really 100% up to finishing it. Although I guess I could pull a preliminary batch of IFPs/Ents/whatever the hell you want to call them, from all three maps, it wouldn't have the nested structs, but it would be better than nothing.

By the way, Prey

You may want to check to make sure all "reflexives" you scan for are 4-BYTE aligned. To further reduce the probability of false hits.

I'm also having a little trouble calculating sizes, there are some weird memory pointers in the map, but you can have fun figuring it out when you get there

[VSG1990]

Is this post still alive?, I was just wondering about the development of the tools.

[Tural]

If there was important news, it would be posted.

[{SMT}iguana]

what happened to that halo 3 mc bitmap (if u have send it to me.)

[warlord]

what happened to that halo 3 mc bitmap (if u have send it to me.)

it was removed and sending you it would go against international copyright laws.

[Shadow LAG]

I can now confirm that all past halo 3 beta moding programs will not work in the final and that the format is somewhat different.

[-DeToX-]

The mapheader is shifted now, and some parts different. As are other things.

[DEEhunter]

mmm i cant open shared.map with H3B edit it just says "shared map" in a little window.

[Tural]

You can't open any retail maps in any Beta utilities. Shared never opened in the Beta anyways, as it was just raw data. In the retail, it appears to have a header. Again, it comes down the the header being different from the Beta to the retail. You won't be able to open any retail maps until someone releases a tool to do so.

[Pyroman]

Yes I do hope to have a tool soon...
*hint**hint*

[shade45]

Heres what I got so far

From what I can tell the maps are structured like this:

Code: Select all

Header //12284 bytes
String Id Index
String Id Table
Unused 2048 bytes
File Names Table
Unused 2048 bytes
File Names Index

And this is what I've gotten mapped in the header so far

Code: Select all

string head; //0
int version; //4
int map_size; //8
int memory_modifier; //16
string time_stamp; //284
int string_id_count; //344
string map_name; //396
string scenario_name; //432
int file_table_count; //692
string foot; //12284

Anyone got any findings to compare? or cracked the formula to get the index offset yet?

[Darco]

Hopefully textures/models will extract in the same way once the header problem gets figured out.