Jump to content


Photo

How-to: Converting existing mods to the EE engine


2 replies to this topic

#1 CamDawg

CamDawg

    Seven billion ton robot monster

  • Gibberling Poobah
  • 9188 posts
  • Gender:Not Telling

Posted 26 August 2013 - 06:58 AM

While I've run down some engine differences over in the IESDP forum, I'd thought I'd put together something more focused on the nuts-and-bolts on getting your mod to work on the new EE engines. The good news is that the major issues will (eventually) be handled by WeiDU itself (multi-language support, different save game paths, etc.) so everyone should head over to the WeiDU forum to give Wisp a big thank you for his work.

That being said, there are still some issues modders will need to address to get their mods to work on BGEE and BG2EE. The good news is that changes for BG2 mods should be fairly minor; BG mods will have some extra work to do.

The broad overview

The EE engine's closest equivalent would be the BG2 ToB engine with TobEx, BG2 Fixpack, and 1PP installed. There are also new 2das, opcodes, and scripting and several formerly broken/non-working aspects of the engine now work. I want to focus this how-to more on conversion, so I'm not going to address the new possibilities too much here.

Save games and baldur.ini are now located in the user's documents folders. Multiple languages are supported by having different tlk files in the game's lang folder (i.e. English talk is in lang/en_us). These changes will be handled by WeiDU itself.

The file formats are the same as BG2. In a few cases a few previously unused blocks are being utilized (i.e. like TobEx, bit 25 at 0x18 in the item header is used to flag critical hit protection). We'll do our best to continue to document changes in the IESDP.

File names are the same as the underlying games--Beregost is ar3300.are in BGEE, just like it was in BG. sw1h03.itm is Kondar. sppr101.spl is Bless. Early on, the EE developers were using the original Bio naming conventions (i.e. blun37.itm is a new BGEE club) but they've since started using their own modding prefixes, OH or BD for mainline content, and BP for Black Pits junk. You'll also spot an occasional modder prefix, too (my own CD is there in a few places, for example).
 
BGIIEE and IWDEE strings are pretty much exactly where they are in BG2 and IWD. That is not, unfortunately, the case with BGEE and BG.

Player education

Even after mods are nicely converted, there are still some critical issues on which we, as modders, need to educate players. First and foremost, the EEs are being actively developed and every time a patch is applied, it will wipe the player's current mod setup. A player in the middle of a modded game may want to delay applying a patch until their game is finished, since even a mod re-install may not allow them to continue their game as it was.

The other important point is how WeiDU will handle multi-language support. The first time a player installs a mod on EE, they will be prompted to select a language to use on their EE game (this is different than what language the mod will use to install). This allows WeiDU to select and use the correct tlk file in its modifications. The language preference will be saved in a configuration file--if the player later changes their mind about the language, we all need to learn how to walk them through that. Since the nuts and bolts are still being decided about language selection, I'll defer this to the beta thread in the WeiDU forums.

Get to the conversion already

So, the basic of basics. WeiDU, as of v232 (current version as of this update is v236), already support EE detection with both the GAME_IS and ENGINE_IS triggers, so it's nice and easy:
 

ACTION_IF GAME_IS BGEE THEN BEGIN // bgee stuff, use BG2EE for bg2ee
  // do bgee stuff
END

Item names and descriptions

The EE engine automatically generates the usable/unusable block of an item description, so you don't actually need it in the text. The other convention is that unique items now use their name--so sw1h03,itm was Bastard Sword +1, +3 vs. Shapeshifters in BG/BG2, but is known as Kondar +1 in BGEE/BG2EE. hamm03 is Ashideena +2, ring06 is the Ring of the Princes +1, and so on. I've been working around this with a special EE tra file (see Item Upgrade v40 for an example). The mod loads the usual setup.tra file in the language selection, but then I override the item description strings with a custom tra file. So, for example, this would be your normal setup.tra:
 


@122 = ~Club +5: Pitchwife~
@123 = ~Club +5: Pitchwife
Forged for <CHARNAME> by amalgamating the oaken, acidic club Blackblood and the splintered club Gnasher, Pitchwife pulses with destructive power. Covered in a tar-like pitchy substance that burns flesh on contact, Pitchwife also drives aggravating spines and splinters into the wound.

Equipped Abilities: 25% acid resistance
THAC0: +5 bonus
Damage: 1D6 + 5, +5 acid damage, slivers add 3 points of extra damage each round for four rounds
Damage type: crushing
Weight: 3
Speed Factor: 1
Proficiency Type: Club
Type: 1-handed
Requires: 5 Strength
Not Usable By:
Mage~

@124 = ~Mace +5: Skullcrusher~
@125 = ~Mace +5: Skullcrusher
This mace is permanently bloodstained; a telltale trace of its many years spent bashing heads. The weapon would have been the bane of humanoids throughout its history, but those that wielded it seem to have been purposely left out of historic record. Recently Skullcrusher was reforged into an even more awesome weapon by <CHARNAME>.

STATISTICS:

THAC0: +5 bonus
Damage: 1D6 + 6, +6 extra damage to humanoid creatures, lowers target's maximum hit points by 2 for 10 rounds
Damage type: crushing
Weight: 6
Speed Factor: 2
Proficiency Type: Mace
Type: 1-handed
Requires: 10 Strength
Not Usable By:
Druid
Mage
Thief~

This is setup_ee.tra:
 

@123 = ~Club +5: Pitchwife
Forged for <CHARNAME> by amalgamating the oaken, acidic club Blackblood and the splintered club Gnasher, Pitchwife pulses with destructive power. Covered in a tar-like pitchy substance that burns flesh on contact, Pitchwife also drives aggravating spines and splinters into the wound.

Equipped Abilities: 25% acid resistance
THAC0: +5 bonus
Damage: 1D6 + 5, +5 acid damage, slivers add 3 points of extra damage each round for four rounds
Damage type: crushing
Weight: 3
Speed Factor: 1
Proficiency Type: Club
Type: 1-handed
Requires: 5 Strength
~
@125 = ~Mace +5: Skullcrusher
This mace is permanently bloodstained; a telltale trace of its many years spent bashing heads. The weapon would have been the bane of humanoids throughout its history, but those that wielded it seem to have been purposely left out of historic record. Recently Skullcrusher was reforged into an even more awesome weapon by <CHARNAME>.

STATISTICS:

THAC0: +5 bonus
Damage: 1D6 + 6, +6 extra damage to humanoid creatures, lowers target's maximum hit points by 2 for 10 rounds
Damage type: crushing
Weight: 6
Speed Factor: 2
Proficiency Type: Mace
Type: 1-handed
Requires: 10 Strength
~

Note how it only has descriptions strings (@123 and @125) and not the names (@122 and @124). The EE tra is not a full tra file; it overrides only the strings that need to be changed for an EE install. You should probably leave a comment in your EE tra file to let your translators know they can also copy their work over and save them some time. In the tp2, you'd just detect EE, load the custom tra, and then proceed as normal:
 

// load up ee-specific tra files on BG2EE
ACTION_IF GAME_IS BG2EE THEN BEGIN
  LOAD_TRA ~c2/%LANGUAGE%/setup_ee.tra~
END

COPY ~c2/itm/c2blun01.itm~ ~override~
SAY NAME2 @122
SAY DESC @123

COPY ~c2/itm/c2blun02.itm~ ~override~
SAY NAME2 @124
SAY DESC @125

EE games, since the extra tra was loaded, will pull @123 and @125 from setup_ee.tra whereas non-EE games will pull those strings from the regular setup.tra file.

Journal entries

This section of the tutorial was thoughtfully provided by Wisp.

The journal has been revamped for EE; the most noticeable improvement is that entries cluster under a given heading and can be collapsed. The bad news for modders is that the old methods of manipulating the journal won't work unless the entries are already registered in the bgee.sql file. In other words, code up your dialogues and scripts as usual but then we need to do an extra step. Enter WeiDU's new ADD_JOURNAL command:
 

ADD_JOURNAL [ EXISTING ] [ MANAGED ] [ TITLE ( text ) ] reference list [ USING traFile list ]

If the game is not BGEE, this action does nothing. If the game is BGEE, this action patches BGEE.SQL with the provided quests and journal entries, so they will work with BGEE’s journal system.


To understand what you are doing with ADD_JOURNAL, it is necessary to first understand a little about how the BGEE journal system works. In BGEE's system, each quest forms a group, consisting of a title and one or more quest entries. A quest entry corresponds exactly to the journal entry of BG2. What's new in this system is the groups and their titles.

To illustrate this with an example, consider the following quest, which is Bodhi's kidnapping of the romanceable NPC E. G. McExample:
 

@1 = ~The Final Battle with Bodhi

We were ambushed by Bodhi and she kidnapped E. G.!~

@2 = ~The Final Battle with Bodhi

My beloved E. G. has been turned into a vampire!~

@3 = ~The Final Battle with Bodhi

E. G. has been restored to life! Joy!~

In BG2, these three journal entries are not related beyond what the modder has scripted. Under the BGEE system, however, they would all go into a single group, with the title "The Final Battle with Bodhi". To make this quest compatible with the BGEE system, we use ADD_JOURNAL:

ADD_JOURNAL @1 @2 @3 USING ~mymod/tra/%LANGUAGE%/bodhi.tra~

ADD_JOURNAL allows you to specify any number of journal entries at the same time, and they need not all share a title. WeiDU will automatically extract the title of the journal entry ("The Final Battle with Bodhi", in this case) and group all journal entries that share a title into a single BGEE-group. You can also specify multiple tra files, with the usual restriction that you can't have and use e.g., @1 from two different tra files.

For example, suppose bodhi.tra also contained this quest:
 

@100 = ~Cookies!

I must get Bodhi's recipe for cookies.~

@101 = ~Cookies!

I was successful in convincing Bodhi to share her cookie recipe with me. Now for the nom-noms!~

@102 = ~Cookies!

Defeat! I had barely opened my mouth before Bodhi shot me down. Sad, sad day.~

We can add both quests in a single ADD_JOURNAL, like so:

ADD_JOURNAL @1 @2 @3 @100 @101 @102 USING ~mymod/tra/%LANGUAGE%/bodhi.tra~

Two quest groups will be created, one with the title "The Final Battle with Bodhi", containing the journal entries @1, @2 and @3, and another with the title "Cookies!", containing the journal entries @100, @101 and @102.

If you want to add journal entries with different titles to the same group, you can use the TITLE keyword. Example:
 

ADD_JOURNAL TITLE (~Interactions with Bodhi~) @1 @2 @3 @100 @101 @102 USING ~mymod/tra/%LANGUAGE%/bodhi.tra~

In this case, a single quest group will be created, with the title "Interactions with Bodhi" and containing all 6 journal entries.

Be default, ADD_JOURNAL will add entries as a new quest group, even if another quest group with the same title already exists. If you want to add entries to an existing quest group, you use the EXISTING keyword. If EXISTING is specified, ADD_JOURNAL looks for existing quest groups with matching titles and adds the specified journal entries to those groups. If no groups with matching titles exist, the journal entries will be added as a new group. You can combine TITLE and EXISTING (but it is not necessary).

ADD_JOURNAL's last keyword, MANAGED, is not yet useful and will not be explained here.

It should also be emphasized that ADD_JOURNAL does not replace the usual UNSOLVED_JOURNAL, SOLVED_JOURNAL, AddJournalentry() etc. ADD_JOURNAL augments the existing methods for adding journal entries and is used as an addition.

Finally, one piece of advice on using ADD_JOURNAL:
When you use EXISTING, it's best if you also use TITLE. BGEE stores the quest-group titles as separate STRREFs, so you can e.g., use ADD_JOURNAL EXISTING TITLE (#31270) to be sure that your journal entries really are added to the right group. Consider "The Final Battle with Bodhi" and "The Final Battle with Bodhi."; they are subtly different and BG2 and many mods are inconsistent on whether to use a full stop or not. WeiDU 232 will automatically strip any terminal full stop on titles it extracts from journal entries (but not from titles you provide through TITLE) to eliminate this source of error (the BGEE style is to not have any terminal full stop on its quest-group titles).

Existing resource file names

As mentioned briefly above, BGEE and BG2EE will use the same file names as the original game--this is great news for BG2 mods (since no changes required) and potentially the biggest obstacle for BG/BGT/Tutu mods.

The good news is that most of this is already alleviated for people who were using the Tutu/BGT variable libraries originally developed for Tutu/BGT compatibility. Probably the best place to get the libraries are from BG1 NPC's Github repo. If you look inside a mod like BG UB, you'll notice that it detects the game and then loads a library full of variable declarations for game resources, and then the tp2 is full of those resources. I.e. the library will declare
 

OUTER_SPRINT "NEBaldursGate_SplurgingSturgeon_L2" "AR0104" // for vanilla BG
OUTER_SPRINT "NEBaldursGate_SplurgingSturgeon_L2" "AR7204" // for BGT
OUTER_SPRINT "NEBaldursGate_SplurgingSturgeon_L2" "FW0104" // for Tutu

since this area has a different file name under each of these games. In the tp2/dialogues/scripting you'll never see the area file directly referenced, just the variable:
 

COPY_EXISTING ~%NEBaldursGate_SplurgingSturgeon_L2%.ARE~ ~override~
  // patch stuff

The attached library will set up all of these variables for BGEE, so if you've already taken this step to make your mod Tutu/BGT compatible you should be more or less good to go with some slightly altered detection code and a library load. If you haven't done so already, well, you're in for a miserable slog.

Audio and area tilesets

EE allows for ogg files to be played directly, negating the need for the usual ogg > wav conversion. Audio is now best handled by WeiDU's new HANDLE_AUDIO function, which will automatically convert (if needed) and copy audio files for you--refer to the WeiDU documentation for more information. HANDLE_AUDIO should really be the standard for all mods, not just EE.

EE uses a new PVRZ compression for tilesets which, at present, no modding tool can really handle properly. I believe DLTCEP can read them, but no tool can compress into PVRZ. The good news is that the old TIS files still work just fine. Just like HANDLE_AUDIO, HANDLE_TILESETS should be your new standard for unpacking tilesets--again, refer to the WeiDU documentation for Wisp's notes.

Portraits

Given that EE is written for a larger screen by default, the small and medium size portraits have been bumped up. The epilogue portrait remains 210x330, but the medium portrait is now 169x266 (from 110x170 in BG/BG2) and the small 54x84 (from 38x60 in BG/BG2).

Kits

A lot of the previous hardcoded aspects of kits are now softcoded--for example, monks and sorcerers can now be kitted; you can use a custom HP table for your kit; backstab multipliers can be applied for non-thieves; and all sorts of other goodies. (Yes, I need to write a new kit-making tutorial to explore all of the new possibilities.)

For the purposes of conversion though, WeiDU's ADD_KIT no longer quite adds to every 2DA needed. Fortunately Wisp has already solved this problem with a library that's included with WeiDU. In the WeiDU download package, look in the lib folder for fl#add_kit_ee folder, and specifically for the readme.html file within. You essentially include the tpa and call the macro after you run ADD_KIT--if you're not taking advantage of the new EE softcoding, that's more or less it. If you do want to take advantage, you can specify a custom HP table, default colors and biography, etc. through this macro. The macro will not do anything on a non-EE game, so you do not need to wrap it in a conditional statement; you can just call it directly.

Character Sets

To get a lot of fancy characters in for the translators, the EE versions use a different character encoding. This causes problems for non-English tra files in particular, but can also cause problems for English tra files. Non-converted tra files can introduce garbled text or even crash the game for your players. Again, Wisp to the rescue with HANDLE_CHARSETs.
 
As of v236, this is a standalone library; starting with v237 it will be included in WeiDU itself. The basic idea is that, when a mod install detects an EE game, HANDLE_CHARSETs will get invoked and convert your translation files to the proper UTF-8 encoding. For Windows packages you'll need to include a copy of the iconv utility; OS X and Linux users will either already have this utility or will need to install it themselves. Rather than re-hash what Wisp has already documented, I'll simply refer you to the linked thread above for help and questions.

General baseline content changes

Both BGEE and BG2EE will include One Pixel Productions and the BG2 Fixpack by default. (Some aspects of BG2FP could be fixed directly in the engine and some fixes didn't apply to BGEE, so it's not exactly the same as what's in the current BG2FP release.) Most aspects of TobEx are already included or being added in future patches.
 
Area MOS files are larger than vBG2, to accommodate the larger EE mini-maps.

BGEE adds three new NPCs and associated quests; BG2EE will add four. For the most part you can think of these as NPC mods (albeit with large side quests).

Black Pits is a new, separate campaign (i.e. not related to the Bhaalspawn), accessible from the game's main menu.

There are also a few new items scattered around the existing games. BGEE adds some Eastern weapons and some other more mundane items throughout the game, as does BG2EE. For the most part these additions will have minimal impact on mods except in very specific circumstances--i.e. both BG Unfinished Business and BGEE decided to give a new club to the great druid in Cloakwood.

There are some tentative plans to include some other material as a baseline, but nothing I can comment on at present.


Edited by CamDawg, 08 October 2014 - 09:09 AM.
more HANDLE_CHARSETs stuff

I came here with a simple dream: a dream of killing all humans. And this is how it must end? Who's the real seven billion ton robot monster here? Not I. Not... I.


#2 CamDawg

CamDawg

    Seven billion ton robot monster

  • Gibberling Poobah
  • 9188 posts
  • Gender:Not Telling

Posted 24 September 2014 - 10:12 AM

Alright, made some updates.

  • Discussion has been split off over yonder.
  • Kit updates now reference Wisp's better macro.
  • Added HANDLE_CHARSETs section for mods with translations.
  • Rather than my own attached library with errors, referred folks to BG1 NPC's Github for the cpmvar libraries, since those will probably be the most thorough and up-to-date ones.
  • Other miscellaneous notes and updates

I came here with a simple dream: a dream of killing all humans. And this is how it must end? Who's the real seven billion ton robot monster here? Not I. Not... I.


#3 Fiann of the Silver Hand

Fiann of the Silver Hand
  • Modders
  • 387 posts
  • Gender:Male
  • Location:MURICA

Posted 02 January 2016 - 02:51 PM

There are also new 2das, opcodes, and scripting and several formerly broken/non-working aspects of the engine now work. I want to focus this how-to more on conversion, so I'm not going to address the new possibilities too much here.

 

Is there a thread for this aspect that is as nicely organized as this one?





Reply to this topic



  


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users