Changing Class/Race/Kit Restrictions with WeiDU

[CamDawg]

Well, since it's now part of the WeiDU documentation, I guess it would be funny if this wasn't at least mirrored here.

This was a topic that I struggled with while trying to convert some kits with unusual item restrictions. Thanks to WeiDU's bitwise operators, this is a process that can be done dynamically and non-destructively and affect all items of a particular type, even if added or altered by another mod (assuming yours is installed after others of course). The basic idea is to construct a function in WeiDU that:

1. Searches through all item files in the game
2. Reads selected data from an item (type, usability, etc.)
3. Alters specific data on an item without changing anything else

This tutorial is basically an expansion on Japheth's excellent READ/BYTE/LONG/SHORT tutorial and I suggest you take a look at that before going further.

My initial problem came about when trying to make specific items usable or unusable by a particular class/race/kit. If you look at the item file structure at IESDP, you see that all of the unusability flags of an item are controlled by the individual bits of bytes 0x1e, 0x1f, 0x20 and 0x21 for class/race alignment restrictions and 0x29, 0x2b, 0x2d and 0x2f for the individual kit restrictions. (Near Infinity combines and displays the four bytes of class/race restrictions as a single chunk of data.)

A quick aside about notation before I proceed. When describing a binary number in WeiDU, it should be preceded with "0b".  When describing a hexadecimal number, it should be preceded with "0x". There are 8 bits in a byte and bits are labelled from right to left. For example, a byte that has its second bit set to "1" and all other bits set to "0" would be written as 0b00000010.

If you want to alter the usability for a single class or race, you shouldn't overwrite any of these particular bytes without first checking their existing values, or else you will change the item's usability for all 8 classes or races that are stored in that byte.  Bitwise operators provide an easy way to perform changes to a single bit within a byte, so you can alter the usability for one class or race without affecting the others.

First we need to look at what the new operators BAND and BOR do with bits. They are both ways of combining two bytes, based on different rules. Both BAND and BOR compare the individual bits (bit 0 vs bit 0, bit 1 vs bit 1, etc. all the way through bit 7 vs bit 7) of two bytes. For each individual bit, the following tables are used to determine the value:

0 BAND 0 = 0
0 BAND 1 = 0
1 BAND 0 = 0
1 BAND 1 = 1

0 BOR 0 = 0
0 BOR 1 = 1
1 BOR 0 = 1
1 BOR 1 = 1

So if byte 0x23 is 0b00110101 and byte 0x24 is 0b10010001, then (BYTE_AT 0x23) BAND (BYTE_AT 0x24) is 0b00010001 whereas (BYTE_AT 0x23) BOR (BYTE_AT 0x24) is 0b10110101.  This may be easier to see when the numbers are lined up:

     0b00110101 
BAND 0b10010001
---------------
   = 0b00010001

     0b00110101 
 BOR 0b10010001
 --------------
   = 0b10110101

Note that the symbols & and | can be used as shortcuts for BAND and BOR. These should not be confused with the symbols && and ||, which are shortcuts for AND and OR.

Back to relating this to unusability in items. For an item to be flagged as unusable, the corresponding bit must be set to 1. A 0 means a particular item is usable by the class/race. Let's look at the example of changing mage robes usable by bards. This would go into your TP2 file.

COPY_EXISTING_REGEXP GLOB ~^.+\.itm$~ ~override~ // copy all item files
  PATCH_IF (SOURCE_SIZE > 0x71) BEGIN
    READ_SHORT 0x1c type // read the bytes containing item type
    READ_BYTE  0x1e usability1 // read the byte containing bard usability flag
    READ_BYTE  0x20 usability3 // read the byte containing the mage usability flag
    PATCH_IF (type == 67 || type == 2) BEGIN // if it is a robe or armor
      PATCH_IF ((usability3 BAND 0b00000100) == 0b00000000) BEGIN // if it is usable by mages
        WRITE_BYTE 0x1e (usability1 BAND 0b10111111)
      END
    END
  END
  BUT_ONLY_IF_IT_CHANGES

This is the same basic idea in Japheth's BYTE tutorial, except now we're utilizing the bitwise operators in the writing and evaluation commands. The mage usability flag is bit 2 in byte 0x20 (IESDP). By performing (usability3 BAND 0b00000100), we mask out 7 of the bits, ignoring what values they may have, leaving a number where only bit 2 retains its original value.  When we compare that result to 0b00000000, what we've done is isolate bit 2 and check whether or not it is set to "0" or not.  If bit 2 is set to "0", that means the item is usable by mages. When the item is is a robe or armor that's usable by mages, we write (usability1 BAND 0b10111111) into the bard usability byte, 0x1e. Using a "1" in bits 0-5 and bit 7 ensures that the original values of those flags are preserved, and using "0" in bit 6 ensures that bit 6 is set to "0" regardless of its previous value, making the item usable by bards whether it was before or not.

As a side note, rather than reading individual bytes, you could read the entire usability block by performing a READ_LONG at 0x1e. It's not recommended because bitwise operations become more unwieldy when you have to write out 32 individual bits instead of just 8.

Let's try another example. Let's try making large weapons such as two-handed swords and halberds unusable by short folks: gnomes, halflings, and dwarves.

COPY_EXISTING_REGEXP GLOB ~^.+\.itm$~ ~override~ // copy all item files
  PATCH_IF (SOURCE_SIZE > 0x71) BEGIN
    READ_BYTE 0x21 usability4 // read the byte containing race usability flags we're interested in
    READ_BYTE 0x31 proficiency // read the byte containing weapon proficiency type
    PATCH_IF (proficiency == 93 || proficiency == 99) BEGIN // if it is a two-handed sword or halberd
      WRITE_BYTE 0x21 (usability4 BOR 0b00010101) // make unusable by dwarves, halflings, and gnomes
    END
  END
  BUT_ONLY_IF_IT_CHANGES

In this case, rather than reading the 'type' byte of the item, I've opted for the 'proficiency' byte. Many two-handed swords in BG2 are classed as long swords for some reason, so in this case, proficiency is a better indicator if it is a two-handed sword IMHO.

The unusability flags for dwarves, halflings and gnomes are all in byte 0x2f at bits 0, 2, and 4 respectively (thanks again IESDP). By writing (usability4 BOR 0b00010101) to byte 0x2f, we're preserving the values of bits 1, 3, and 5-7 while setting the 0, 2, and 4 bits to "1" (unusable) regardless of their previous values.

With the addition of the WeiDU variable THIS, many bitwise patches can be written with less code. Whenever a WRITE_BYTE, WRITE_SHORT, or WRITE_LONG command is performed, WeiDU will first execute a READ_BYTE, READ_SHORT, or READ_LONG and store the previous value in a variable named THIS. Using THIS allows us to avoid issuing our own READ_BYTE commands ahead of time.

Here are the same patches from above, with one READ_BYTE command in each replaced by using THIS.

COPY_EXISTING_REGEXP GLOB ~^.+\.itm$~ ~override~ // copy all item files
  PATCH_IF (SOURCE_SIZE > 0x71) BEGIN
    READ_SHORT 0x1c type // read the bytes containing item type
    READ_BYTE  0x20 usability3 // read the byte containing the mage usability flag
    PATCH_IF (type == 67 || type == 2) BEGIN // if it is a robe or armor
      PATCH_IF ((usability3 BAND 0b00000100) == 0b00000000) BEGIN // if it is usable by mages
        WRITE_BYTE 0x1e (THIS BAND 0b10111111)
      END
    END
  END
  BUT_ONLY_IF_IT_CHANGES

COPY_EXISTING_REGEXP GLOB ~^.+\.itm$~ ~override~ // copy all item files
  PATCH_IF (SOURCE_SIZE > 0x71) BEGIN
    READ_BYTE 0x31 proficiency // read the byte containing weapon proficiency type
    PATCH_IF (proficiency == 93 || proficiency == 99) BEGIN // if it is a two-handed sword or halberd
      WRITE_BYTE 0x21 (THIS BOR 0b00010101) // make unusable by dwarves, halflings, and gnomes
    END
  END
  BUT_ONLY_IF_IT_CHANGES

Thanks especially to Smoketest for helping me with this. Thanks also to Wes Weimer for his excellent tool, Japheth for the tutorial that inspired this and the help given, and the IESDP team.