Site hosted by Angelfire.com: Build your free website today!

Fighting doubling bugs

Updated 20.01.03

Easiest way: Don't update your mods while using a savegame.

Okay, that way may be quite dumb.. so here are some solutions.

The Morrowind File Data splits into three different kinds of files:

- esm-data (mainfiles, containing objects and references)
- esp-data (plugins, containing objects and references)
- ess-data (savegames, containing references only)
Each is loaded and compiled separately, the lower overriding the upper.

From the point of a developer this makes perfectly sense: ess-data is newer as esp-data, which is newer than esm-data.
Data is processed as the newer data overriding and substituting the older one.
Period. That was done with Morrowind.

The processing sequence for Morrowind Data is:

- loading esm
- no checking at all, just simply altering objects and adding(!) references
- loading esp
- checking wether an esp has all loaded esm referenced correctly
- giving an errormessage if they are not
- loading ess
- checking wether an ess has all loaded esm and esp referenced correctly
- giving an errormessage if they are not

This works nicely.. you load the esms, overwrite identical entries with esps, finally overwriting them again with ess.

If some files of the same priority modify the same object, only the most recent change will take effect. So newer esms overwrite older esms, newer esps overwrite older esps and older esms and newer ess overwrite all older files. Heed the warning, that esms do not overwrite older esms in respect of references. Objects only.

If an esp has not referenced all esms, there will be an error, which can be fixed by loading the esp into the TESCS along with all needed esms and resave it. Conflicts will be pointed out, and therefore can be fixed, so the esp is running fully compatible with all the esms.

If an ess has not referenced all esms and esps, there will be an error, too, but that is not fixed that easy. But mostly the references stored in the ess will not conflict with the game, as it only stores changes done to specific things (npcs, items, journal,...), not adding objects to the world or modifying objects in the world.

With one exception.

It stores the position of any moved item as a reference (safe statics, for they shouldn't be moved ingame). They can be moved by the player (items, non-teleporting doors) or move themselves around (NPCs, creatures).

If you update an esp or esm (any esp or esm) or add an esp or esm (any esp or esm) and use your old ess all esp-data will be counted "newer" as the ess-data. But ess-data is processed after loading the esm and esp-data. So the ess/esp-references cannot be overridden by the ess-references for the esm/esp-objects are "newer" than the ess-references to them. The game will add a double and there will be a much discussed bug called "doubling":




Doubling may occur to any object (safe statics) in the game, as mentioned above, but creatures are the most disturbing problem, as can be seen.

You may delete any doubled reference by bringing up the console via [~] (or [^] on german keyboards), clicking on one of the doubled reference, so its name is displayed in the console-window and typing:
disable [Enter]
This will remove the reference out of the cell-data.
As a warning from Puma: Be careful not to do this with creatures that are constantly making sounds "looping sounds" in GIANTS like Trolls, Nymphs, Liches,.. because the sounds will not be removed from the spot they were removed.
That's a pretty lame way,too, especially if there is lot of doubling (and lots of mods)

So there should be a better one.

If you look inside an ess file, there will be couple of "MAST" fields in the header - one for each esm and one for each esp. Each "MAST" field has "DATA" field which contains plugin file size.

As ghostwheels tests showed, esm/esp-file size record (MAST record) inside ess-file serves only for checking whether plugin has changed and displaying appropriate warning. It's completely irrelevant for the question if doubling occurs or not.
So Morrowind can ONLY check esp-updates, if
either
the esp file date is newer than the ess file date
or
if the ess references different esps or esms, than those checked in the Data Files
So there are only 2 ways doubling can actually occur at all:


- You update an existing esp
All you have to do after is to set the timestamp of the ess-file to be newer than the date of your last modified esp.

You may do this by using the explorer, touch utility or pplusmain explorer enhancement

There can't be any conflicts, for the ess only references data you encountered. If new content is added, fine, it is not saved in your ess, so it won't be overridden by it. If data is deleted out of the game, fine, your ess will have no objects for its references and cut it out as well. (Displaying an errormessage)

The only possible conflicts may occur if references (NOT objects) encountered are altered, but most clean mods avoid altering references without assigning a new object to their alteration. Only Dialogue and Journal Entries are somewhat of a problem, but known to modders and checked intensively during betatests. Some may require you to start a new game, but this is mentioned explicitly in the responding readme-files.


- You add or remove/rename an esp
Sorry, this is inconvenient for there is (up to date) no way to include or exclude an esm or esp-file from the ess header safe hexediting manually or loading and resaving ingame (which will cause doubling). Beside this wouldn't be safe to hexedit, for plugins, compared to savegames come with their own object data and not just stored references of other files objects. Even if your ess is of a newer date, for the savegames header does not reference the new esps it will cause problems. There is no valid or clean possibility of avoiding doubling while adding/removing/renaming esps, while the mod whose content is doubled is an esp, too.


But there seems to be some good news. For the way MW handles data it -seems- to be quite safe to add mods. Added content is not referenced in the ess, so it cannot double. Modified or better moved references in visited cells -WILL- double and if a mod is deleting content (wich do very,very few) the delete may be ignored. If a mod adds references or objects or modifies objects only (as the most) MW internal data processing will not screw up and cause doubling.
Much worse is the matter with renaming or removing mods. The ess does reference data, that is no longer there, causing lots of errormessages, a complete rechecking of all objects of stored references and very,very likely doubling of all objects, that are NOT individually named NPCs and NOT at the same position in the esp and in the ess (eg, creatures, guards, items, doors,...)

This most visibly effects creature mods like

* Giants
* Giants Dragons
* Morrowind Additions
* Morrowind Advanced (Passive Healthy Wildlife)
* Wilderness Creatures Mod

So what to do?

The solution is quite simple. Not as simple as the one above, but quite. Make them esm. The How-To will be covered in the other tutorial, so here is only the reason, why.

If we make them esm, they will be loaded and compiled prior to the esps.

If you do not like errormessages new esms still have to be referenced in the respective esp and ess files. So you'll have to load each esp in the TESCS and save it with all esms loaded and have to load and resave your ess once.

After resaving your ess with the custom made esms checked, you can modify, add, drop or rename your esps to your hearts content. Creatures (as referenced in the esms) won't double, as they are compiled before the esp-data is loaded and compiled and checked separately for changes against the ess, when it will be loaded.

For this reason, updating any esp doesn't effect them any more
- besides the esp containing data, which will override the custom esm-data like using fix leveled lists instead of the "add to leveled list" commands, that came with tribunal
But that's what Horatios Leveled List Merger is for.

Cleanig up corrupted ess

This comes straight from Pumas Page. This is done by going into a interior cell (preferably a small shack with no creatures in it at all) and saving your game. Exit MW. Before you start your game again uncheck the esp from data files. Then load the ess you just made. There will be many error messages so just ignore them and keep clicking yes to continue. Once done clicking save the game again. Exit MW. Check the esp now in data files. Now load up the save and everything should be okay.

TESAME:
Those methods, however, will clean up doubling but will not repair any corruption (damage) that may have happened to your save. Using The Elder Scrolls Advanced Mod Editor (TESAME) to clean and fix your save is the best way to do both in my experience.

You simply get the save from the Save folder. Next, change the extension from .ess to .esp Now open TESAME and edit the save like you would a plugin. I recomend experimenting with a back up save and try deleting any refrences to the mods that are giving you problems. I also recomend deleting all or most "exterior" cells especially the ones giving you trouble with TESAME. Interior cells have a less chance of not working right but if you encounter wierdness in an interior cell do the same to it. The drawback to this is that cells will "reset" back to their original state so characters you may mave killed will show up again. Same goes for the Seyda Neen start characters and boat. If you encounter this just delete/disable NPCs and objects that should not be there anymore with the console. Your local map in those areas will also be reset (not the world map). Reseting cells is the only way to fix your save game cells, though.


One petition left: PLEASE READ

Don't run around excited, trying to make your new house mod or custom made island an esm-file. This is only necessary for Mods, that modify lots of moving Npcs or creatures located in cells, that are not included in your mod. If the cells are included in your mod, they will only be affected, when you update your mod. So here go with the guidelines for updating mods without having doubling occur, explained above. Best example to validate this is, that doubling will not occur with Firemoth, which adds lots of creatures to it's own cells.


Thanks and greetings go to:
- PumaMan - for creating such awesome mods, that I couldn't stand seeing them flawed and got involved with doubling
- Daerk of MeMod - for his input on esm-building
- ghostwheel - for pointing out that esp-filsize doesn't matter for doubling. A thing that I was simply to lazy to check.
- Horatio - for listening to my barely understandable ramblings and creating awesome tools
- Tenaka - for I promised testing his mod, but was very distracted by this thing here. I'll start right now, pal. Don't want Morgiah assuming I skipped our date ^^