Rebuilding Blucher's alternate BG2 NPCs

[Nythrun]

Latest build:

Nythrun's Level 1 NPCs, Beta 6

 

"Version Six: Ultimate Beta"

 

Temporarily available for download here.ETA 10:00pm CST/USA Thursday, October 17, 2007

 

Nythrun deserves cookies, flowers, chocolate, and about $10,000 of development money, if anyone is rich enough!

 

-cmorgan, October 13, 2007

 

==== Original Thread Follows ====

 

EDIT by cmorgan, October 11, 2006

 

This topic was split from Azazzello's original post here, as enough conversation and interest was taking place to explore this as a community project.

 

For code contributions, visit the Wiki here.

 

For discussions, ideas, and coding examples, browse this thread and add your contributions!

 

Synopsis: Azazello is proposing and acting on an idea to build Blucher's original non-WeiDU mod into a concept mod that allows players to change existing NPC classes and kits with a great deal of freedom.

 

 

 

 

Nythrun's original reply starts here:

 

Well, maybe not so boring, since you're basically going to be rewriting the spell memorization tables from scratch anyway to get level one NPCs with 89,000 (or whatever) experience. It's pretty domineering as patching goes.

 

Though if you actually want every extant kit as an option you should probably get comfy with LAUNCH_PATCH_MACRO in SUBCOMPONENTs or the huge amount of redundant code will glaze your eyes for sure.

[cmorgan]

This is what comes of spending lots of comfy time in Tutu-land. I just got what you folks really are talking about. BG2 expects players to start at a relatively high level...

 

so if I finally understand what Nythrun and the bigg are saying, in order to do this via patching

 

Macready has a very simple, elegant solution to assignment. He does indeed use new base .cres, just repatches them to use the existing strrefs and soundset:

 

/* Ajantis: Cavalier */
BEGIN @101
SUBCOMPONENT @100

/* Read strrefs from existing creature */
COPY_EXISTING	~_ajanti.cre~	~override/macdummy.cre~

 /* Read names */
 READ_LONG 0x0008 "creatureN1"
 READ_LONG 0x000C "creatureN2"

 /* Read creature soundset array in one chunk */
 READ_ASCII 0x00A4 "creatureSoundset" (400)

 INNER_ACTION BEGIN
/* Copy in new version, update strrefs */
COPY ~NPCKits/cre/_ajanti_cavalier.cre~		~override/_ajanti.cre~
	 ~NPCKits/cre/_ajanti4_cavalier.cre~		~override/_ajanti4.cre~
	 ~NPCKits/cre/_ajanti6_cavalier.cre~		~override/_ajanti6.cre~

  WRITE_LONG 0x0008 "%creatureN1%"
  WRITE_LONG 0x000C "%creatureN2%"

  /* Write creature soundset array in one chunk */
  WRITE_EVALUATED_ASCII 0x00A4 "%creatureSoundset%" #400
 END
 BUT_ONLY_IF_IT_CHANGES

 

The problem with the class reassignment for the above mod is you need to either

 

a. build a .cre file for each of the individual NPCs for each class and kit option, assign the spells, etc, and then borrow the above idea to swap the .cre file into the installation, or

 

b. Build a massive patch which

1. reads in the information from the offsets as above

2. overwrites the class and kit offsets with the offset value for the new class/kit combo

3. adds appropriate weapons/armor into the cre, removing/destroying the old ones

4. adds in both memorized and learned spells

 

The part I am still not sure about is the discussion about spells. Either way, the BG2 spell tables need to be extended backwards to L1, right? Or does this already happen somewhere, like BGT, and is already out there?

 

Because with the bigg's kind work towards ADD_LOTS_OF_LONG_FUNCTIONS_FOR_US_NONPROGRAMMERS, adding kits, memorized spells, and spellbook entries are built into WeiDU now such that a COPY_EXISTING could add all but the class.

 

I must be missing something important here...

[CamDawg]

I suspect much of this work is done, just scattered about. tweaks' ToB-Style NPCs is probably a good start for preserving massive chunks of info from a creature file about to be overwritten (soundsets, inventories, spells, etc.), and both Fixpack and DR muck about with spell-replacement algorithms. Fixpack destroys Nalia and Edwin's spellbooks for a complete re-write while the DR algorithms are much nicer but simpler, only replacing spells.

[cmorgan]

I am going to say up front that this is a cool idea that I would never use, as I stop changing NPCs at Imoen (she is always a Swashbuckler dualed to Mage for me), but definitely something that others might like. As lots of experienced folks have done all the hard work, and most of this is like Cam indicated (basically pulling together elements of other mod's work), it sounds like it is possible for those of us less-experienced folks to assist with legwork.

 

I like option b alot better. It sounds like it gives folks much more flexibility. To be safest, in option b you would blank all offsets dealing with spells at the top of the install anyways, right? The fixpack blasting it all away in this case is a better option, as you may not want a spellbook at all. So actually you build a section that scans the install for the various kits and use R_F to pop up the availability of the .tph subsections of the tp2 code.

BEGIN ~Change Valygar's Class~
 SUBCOMPONENT ~Hexblade~ (FILE_EXISTS_IN_GAME ~swordandfistflagfile~) 
 GROUP ~Change BG2 NPCs~
COPY_EXISTING ~valyg9.cre~ ~override~
PATCH_IF (SOURCE_SIZE>0x02d4) BEGIN
  LAUNCH_PATCH_MACRO ~BlankNPCSpellsItemsProficienciesBtUsingAShorterNamefortheTPH~
  LAUNCH_PATCH_MACRO ~hexblade~
  LAUNCH_PATCH_MACRO ~hexproficiencies~
  LAUNCH_PATCH_MACRO ~hexitems~
END ELSE BEGIN
PATCH_PRINT
 ~Valygar appears to be defective. Let us leave him to brood on his cursed ancestry and not patch.~
END
BUT_ONLY_IF_IT_CHANGES

SUBCOMPONENT ~WizardSlayer~
GROUP ~Change BG2 NPCs~
 COPY_EXISTING ~valyg9.cre~ ~override~
PATCH_IF (SOURCE_SIZE>0x02d4) BEGIN
  LAUNCH_PATCH_MACRO ~BlankNPCSpellsItemsProficienciesBtUsingAShorterNamefortheTPH~
  LAUNCH_PATCH_MACRO ~wizardslayer~
  LAUNCH_PATCH_MACRO ~wizardslayerproficiencies~
  LAUNCH_PATCH_MACRO ~wizardslayeritems~
END ELSE BEGIN
 PATCH_PRINT
~Valygar appears to be defective. Let us leave him to brood on his cursed ancestry and not patch.~
 END
 BUT_ONLY_IF_IT_CHANGES

 

Probably the safest way is to actually blank all spells, items, etc, and use the tph to add the appropriate stuff back into the .cre, using

ADD_KIT, ADD_KNOWN_SPELL, REMOVE_KNOWN_SPELL, ADD_SPELL, ADD_CRE_ITEM etc?

[Nythrun]

No matter how it's done there'll be some risks. I'm going to talk through a few of them step by step - please don't think I'm being condescending, there's bound to be someone reading who could benefit from this approach, even if you know every bit of it already (and you probably do).

 

Since Valygar has been such a good stoic about the teasing, we'll pick on Minsc for awhile (who needs our help, or a doctor, or something). If each Minsc creature is listed individually

COPY_EXISTING ~minsc7.cre~   ~override~
		  ~minsc8.cre~   ~override~
		  ~minsc9.cre~   ~override~
		  ~minsc10.cre~  ~override~
		  ~minsc12.cre~  ~override~
		  ~minsc14.cre~  ~override~

then we'll run into problems when FlimFandang0 releases his "Start BG2 at level three and cap out at level nine because high levels make no sense" mod, because his minsc3.cre isn't on the list until we add it there (along with an ALLOW_MISSING at the start of the .tp2). And just doing a COPY_EXISTING_REGEXP ~minsc[0-9]\.cre~ sort of thing causes problems when we accidently destroy the spellbooks of Marcie Ingles's New Super Clerics from her highly acclaimed "Mighty priests rough you up and take your lunch money" mod. So instead, let's steal borrow some code from CamDawg, where we copy every creature and only patch them if the name matches "Minsc" - and then incorporate an additional check to make sure the death variable matches. Checking IESDP, we find that a creature's full name is stored at 0x8 and is four bytes long, and that its tooltip name is stored at 0xc and is also four bytes long, and that its death variable is stored at 0x280 and is thirty-two bytes long. Now we can be maximally finicky about patching all Minsc all the time, but only Minsc.

COPY_EXISTING_REGEXP GLOB ~^.+\.cre$~   ~override~
 READ_LONG  0x0008 "ln"		ELSE 0  // we're reading the long name here
 READ_ASCII 0x0280 "dv"   (0x20) ELSE 0  // and reading the death variable here
 PATCH_IF   (("%ln%"=9501) OR ("%ln%"=9482))   BEGIN // these are the two "Minsc" strrefs in Dialog.tlk 
PATCH_IF ("%dv%" STRING_COMPARE_CASE "MINSC"=0) BEGIN // and here's the death variable check
END // PATCH_IF death variable check
 END // PATCH_IF long name match
BUT_ONLY_IF_IT_CHANGES // do not litter the backup directory please

Unfortunately, this code isn't doing anything useful yet, so let's start small and decide how to patch the known spells area of the creature for now.

 

REMOVE_KNOWN_SPELL is definately an option, and it's easy to write

REMOVE_KNOWN_SPELL ~sppr101~
			   ~sppr103~
			   ~sppr104~

where we list by hand everything in the ranger spellbook. But what if some other mod made Minsc a Blade, and we have to get rid of arcane spells and some innates? What if The Darkest Day is installed and a dozen new ranger spells have been added? There are a lot of possible variations on what spells Minsc could know, and any variation that isn't accounted for that a user has installed before a rekit mod is a potential game crasher. So the surgical removal method could work, but it's unwieldy. Perhaps the tabula rasa is a better choice, especially as there's no need to worry about mod conflicts at the moment and we can handle everything with READing, WRITEing, and basic arithmetic...

 

Referring back to the IEDSP, we find 0x02a0 is listed as "Known Spells Offset". Because the list of known spells in a .cre doesn't always start in the same place, there has to be a marker somewhere in the file that lets the game know where to seek this information. Now that we can find the marker, we can tell WeiDU to go find the list for us with a

READ_LONG 0x2a0 "known_spells_offset"

. Of course, this only states where the list begins, and doesn't say where the list ends - and there's no "Known Spells End Here" offset. This matters because we only want to erase the spells known and not a bunch of the rest of the file too. Which means we'll have to figure out a way to make WeiDU find the file size for us. Fortunately that's easy

The length of the Spells Known array can either be always the same, or vary based on something. Checking IEDSP again, there's no listing for a fixed table size. What there is, however, is helpful description about the Spells Known array - specifically, that each spell that's known has eight bytes devoted to its resource reference, two devoted to its spell level, and two devoted to its type. Because that's all there is to a Spells Known array, we know that the array has to be (twelve bytes multiplied by number of spells known) in size. The number of spells know isn't something we've found yet - but fortunately IEDSP lists 0x02a4 as "known spells count", exactly what we need.

READ_LONG	0x02a0  "known_spells_offset"
READ_LONG	0x02a4  "known_spells_count"
DELETE_BYTES "%known_spells_offset%"   (12*"%known_spells_count%")

And we're done, right?

 

Not quite; not only is there a Known Spells Offset, there are offsets to several other parts of the file too. Because we've just removed a chunk of the landscape, these maps may not be accurate anymore. Whoops. This, too, however is easy to fix We'll just have to READ all of these other offsets (there are five) and adjust them if they need adjusting. If these offsets are less than the value of Known Spells array offset, it means they refer to an earlier part of the .cre than the part we just excised, and if these offsets are greater than the value of the Known Spells array offset, then something in between has changed and we need to patch them. Hmm, if...patch - sounds like PATCH_IF to me

READ_LONG	0x02a0	kso
READ_LONG	0x02a4	ksc
READ_LONG	0x02a8	smo
READ_LONG	0x02b0	mso
READ_LONG	0x02b8	iso
READ_LONG	0x02bc	ilo
READ_LONG	0x02c4	elo
DELETE_BYTES kso	   (0x0c*ksc)
PATCH_IF	 (smo>kso) BEGIN
 WRITE_LONG 0x02a8	(smo-(0x0c*ksc))
END
PATCH_IF	 (mso>kso) BEGIN
 WRITE_LONG 0x02b0	(mso-(0x0c*ksc))
END
PATCH_IF	 (iso>kso) BEGIN
 WRITE_LONG 0x02b8	(iso-(0x0c*ksc))
END
PATCH_IF	 (ilo>kso) BEGIN
 WRITE_LONG 0x02bc	(ilo-(0x0c*ksc))
END
PATCH_IF	 (elo>kso) BEGIN
 WRITE_LONG 0x02c4	(elo-(0x0c*ksc))
END

I like short variable names (ksc rather than known_spells_offset) because the long ones confuse me - you don't have to use them. I like having "12" written as "0x0c" because changing number base in the same line looks ugly to me - you don't have to do that either. And I like having everything in neat columns because I have obsessive compulsive disorder because I make fewer mistakes that way - you definately don't have to do that.

 

Naturally, there are trade-offs involved in this method too - Minsc has lost his punch-drunk rage (which we'll probably want to add back with an ADD_KNOWN_SPELL later). And if you have Nythrun's mod that grants Minsc the "Summon Cricetinae Horde" innate (which allows Minsc to summon a legion of hampsters to eat Goodberrys and stand around looking fuzzy) then that's gone too, and you can do a file check to restore it if you're feeling silly. Much like the REMOVE_KNOWN_SPELL option we mentioned way back, there's going to be some manual touching up needed to ensure total compatablity. The difference here is, we don't have to add any specifics and the ones we do include are there only because we want them.

 

And now we're done

[cmorgan]

Now *THAT* is what I call a real, true, honest-to-the-gods tutorial!

 

 

Anytime anyone wants to lay things out like that, I will happily study and learn

[Azazello]

Although patching would be more elegant, it is much more problematic. Maybe we should go back to simple roots… Blucher’s method was perhaps more enlightened than even he imagined: create Level 1 templates for each vanilla NPC, for each vanilla kit/class/whatever. The mod’s install options are a simple 2-branch choice tree: Which NPC ya’ want? Which kit/class/whatever ya’ want?

 

What to do about minsc7, minsc9, minsc14, etc.? Simple: minsc7 is minsc1 with minimum experience to make Level 7; minsc14 is minsc1 with minimum experience to make Level 14. Etc. These files will overwrite equivalent files in the override folder, so this new mod should be installed after any mod that makes changes to these NPCs.

 

Consequence: when the player acquires the NPC, s/he has to sit there for possibly a half-hour choosing the NPC’s proficiencies, spells, whatever.

 

You know what? That’s exactly the benefit of this mod – the player gets to choose the growth path of the NPC.

 

OK, what about new kits, new classes, and new NPCs? Well, either the creators of those mods must make Level 1 equivalents that can then be added to this expanded Blucher mod; or they can wait until an elegant method is developed using patching.

 

So, what am I missing?

 

If this block-headed approach is doable, I am more than willing to make the templates, Blucher’s permission inclusive, of course. Just gimme the links on how to do this, and I’m on it.

[berelinde]

That's really cool!

[cmorgan]

OK, folks, so why don't we put our heads together and come up with a combination.

 

Why don't we

1. create the base .cres for each NPC at level 1 in each of the general classes, without kit, using the original .cre files as a base.

L1MinscFighter.cre

L1MinscDruid.cre

L1MinscPaladin.cre

 

etc.

 

with the only restrictions being the stats on the original .cre (I mean, really, anyone who makes Minsc a Sorceror should go play another game ). And with an empty spellbook, if appropriate?

 

2. Use SUBCOMPONENT Copy to override to build the expected base creatures for the version, reapplying the appropriate DV, script assignment, etc for Tutu, BG2, or BGT, patching the _cre with the correct amount of experience, but leaving the level at 0 (I assume this is how TOB style NCs work; we could also bring DavidW in, because he uses a form of this for SCS) and flag the override for the following Forced Subcomponent set...

A_I F_E_I_G ~FW0100.are~ THEN BEGIN

COPY ~L1NPC\Minsc\L1MinscPaladin.cre~ ~override\_MINSC.CRE~

END ELSE ect., etc.

 

3. Use FORCED_SUBCOMPONENT COPY_EXISTING to give a patching choice of kit, and the appropriate changes

 

4. Still stuck with spell choice - how does TOB-style and SCS deal with this?

 

 

I think if people are interested, we could Development-Wikki this one into existance (I am still assumng that NiGHTMARE is around and reading this, becuase I thought he had a mod that changes NPCs, and I would really not be thrilled with building competition - I would rather community ghost-write additions and help with expanding existing mods, if possible).

[cmorgan]

If you guys can give me a day or so, I am still absorbing Nythrun's cool tutorial post, and I may have a simple way of combining this with the patching stuff reducing the .cre load to just one new .cre per NPC per class, as per Macready/Nythrun -- just gotta think and poke through IESDP more.

 

Quick question:

 PATCH_IF   (("%ln%"=9501) OR ("%ln%"=9482))   BEGIN // these are the two "Minsc" strrefs in Dialog.tlk

 

OK, this gets more tricky -- are those strrefs stable over different versions, say Tutu/EasyTutu_SOA, EasyTutu_TOB, BGT, BG2, etc? Would it be strong enough a ckeck to only use DV (which means special treatment for BGT, as they have some versions of Imoen, for example, which use imoen2, etc).

 

Given that, how about we create a base set of "incomplete" .cres for each NPC, copied to override based on version, patch them with the existing offsets a-la Macready, then use the tph patching to assign a kit fleshing out the cre (using TRUECLASS as the default FORCED_SUBCOMPONENT)

 

Trying to figure out how to avoid creating all of the .cres Nythrun mentioned, plus the duplicate set for Tutu, plus the duplicate set for BGT all at once.

[Guest erik]

Use READ_STRREF on the name strref you've already read out of the .cre and compare the string to "Minsc". That's foolproof, once you get it working. Can't remember where I saw the working example, though. Easy enough to sketch a method, though:

 

(insert copy-all-cre-files-regexp here)
 READ_STRREF name "your_variable_here"
 PATCH_IF "%your_variable_here%" STRING_EQUAL "Minsc"

 

et voila! (untested, but you get the idea)

 

-erik

[5th_horseman]

I've posted about this over at spellhold, my coding skills are very weak, but the ideas that I had were along the lines, creating generic templates for the various classes/kits and useing code from "Tob style NPC's or "Devine Remix" that transfered the details (ie.stats, xp, gender, ect) from the orginal npc to the new version generated from the template.

 

The idea was to create a mod that compatable all the kits available and change the kits selectable when generateing a new charactor, the mod would be installed last so all kits and class mods installed.

 

The mod would check what mods had been installed, this would allow or disallow which npc's, kits, class mods there were to be modified.

[CamDawg]

Let's simplify the conversation:

1. Information that needs to be saved from the NPC being changed: Name, XP, colors, portraits, resistances, soundset, sex, gender, scripts, dialogue file, DV, general, race, specific, alignment.
   
2. What should be determined by their class/kit: HP, thieving skills, THAC0, ApR, saves, racial enemy.
   
3. Stuff that's based on a little of both: Animation, AC, level (single v dual/multi), attributes, spells, equipment, profs.
   
4. Stuff that should be the same for everyone: Rep, lore, gold, flags, state, intoxication, luck, BG profs, tracking, morale (et al), EA, functional specs.
   

Stuff on point #1 can be read from the character file about to be replaced, a la ToB-style NPCs. To cover #2 & #4, create a template creature in every kit/class combo, to which you'd simply write info from #1 and #3. Point #3 will be the meat of the mod and can be handled by lookup tables. For example, a proficiency distribution map for Minsc's stats as a cleric, ranger, wizard slayer, etc.

 

I'd suggest limiting the changes within reason--either to what their attributes would allow, or with slight rebalancing. I don't think anything that requires an alignment change (i.e. Korgan the ranger) makes sense, or which requires significant backstory changes. I.e. sorry, Edwin as a cleric negates his personality, but Edwin the sorcerer...

[cmorgan]

PATCH_IF (("%SOURCE_RES%" STRING_COMPARE_CASE "imoen15" = 0)

 

was another way --

 

and for anyone wanting to both see what is possible and fry their brains at the same time, the BG2 Fixpack is both a goldmine of information and enough to send a sane, non-programmer person running back to cute little tp2 tricks!

 

CamDawg, that sounds like a plan (except... someone will have to explain to me what/how you set up a lookup table for minsc's stats -- using small words and lots of examples, as if to a very precocious child ).

[CamDawg]

CamDawg, that sounds like a plan (except... someone will have to explain to me what/how you set up a lookup table for minsc's stats -- using small words and lots of examples, as if to a very precocious child ).

Let's say we're going to allow Minsc to be a ranger, paladin, fighter, barbarian, or cleric. His proficiencies will vary slightly from each incarnation--he gets an extra two pips in dual-wielding for being a ranger, gets fewer pips and choices as a cleric, etc.. Rather than trying to do this with a bunch of PATCH_IFs, I'd create tables and go from there:

 

WEAPON					RANGER  FIGHTER  BARBARIAN  CLERIC  PALADIN
BASTARDSWORD			  0	   0		0		  0	   0
LONGSWORD				 0	   0		0		  0	   0
SHORTSWORD				0	   0		0		  0	   0
AXE					   0	   0		0		  0	   0
TWOHANDEDSWORD			2	   2		2		  0	   2
KATANA					0	   0		0		  0	   0
SCIMITARWAKISASHININJATO  0	   0		0		  0	   0
DAGGER					0	   0		0		  0	   0
WARHAMMER				 0	   0		0		  1	   0
CLUB					  0	   0		0		  0	   0
SPEAR					 0	   0		0		  0	   0
HALBERD				   0	   0		0		  0	   0
FLAILMORNINGSTAR		  0	   0		0		  0	   0
MACE					  0	   0		0		  1	   0
QUARTERSTAFF			  0	   0		0		  0	   0
CROSSBOW				  0	   0		0		  0	   0
LONGBOW				   2	   2		2		  0	   2
SHORTBOW				  0	   0		0		  0	   0
DART					  0	   0		0		  0	   0
SLING					 0	   0		0		  0	   0
2HANDED				   0	   0		0		  0	   0
SWORDANDSHIELD			0	   0		0		  0	   0
SINGLEWEAPON			  0	   0		0		  0	   0
2WEAPON				   2	   0		0		  0	   0

 

Toss this in as MyMod/Minsc/profs.2da and then just READ_2DA_Entry in the correct column. Same could be done for attributes, THAC0, etc., etc. You could do this with multiple tables (i.e. one for profs, one for attributes, etc.) or just a single monster table from which you read all of this data for their class/kit.

 

edit: I should point out that this is just an example. Realistically I think Minsc's choices should only be ranger (any kit), fighter (any kit), and barbarian. You would need a column for every kit in addition to the trueclass.