Jump to content

Photo

Where is stuff?


25 replies to this topic

#1 subtledoctor

subtledoctor
  • Modders
  • 2855 posts
  • Gender:Male

Posted 23 August 2018 - 05:49 AM

Trying to navigate this mod is maddening.  I'm playing around with ways to make the install process shorter.  I've got an idea for the General AI component, since that takes 6+ hours for me with a bunch of NPC/spell/kit mods. 

 

But the Initialize AI component takes about 2 hours as well, with only a single large kitpack installed before SCS.  So I'm trying to see if I can streamline things there as well.  From the mod's .tp2 I can find my way to /stratagems/caster_shared/initialize.tpa... but there I only see:

      LAF collect_the_immune END
      LAF build_instant_spells END
      LAF build_prebuff_spells END

 

Where are those functions defined? I've tried looking through files in /lib and /sfo but I can't find them.


Faiths & Powers: Spell spheres and kit pack for priests and paladins
Might & Guile: Tweaks and kits for warriors and rogues
Scales of Balance: Game tweaks and rule overhauls
NPC_EE: More options for NPCs in BGEE, SoD, & BG2EE


#2 lynx

lynx
  • Modders
  • 3147 posts
  • Gender:Male
  • Location:Ljubljana, Slovenija

Posted 23 August 2018 - 06:29 AM

If you don't have a clone to git grep through, github is a good basic searching alternative:

https://github.com/G...shared.tph#L170


GemRB - IE anywhere.
Mages needed! Looking for Planescape: Torment testers
Play android version IS NOT SUPPORTED ANYMORE: reported bugs will be ignored! Still looking for builders ...


#3 Mad Mate

Mad Mate
  • Members
  • 209 posts
  • Gender:Male

Posted 23 August 2018 - 06:30 AM

All 3 functions are defined in caster_shared\caster_shared.tph



#4 subtledoctor

subtledoctor
  • Modders
  • 2855 posts
  • Gender:Male

Posted 23 August 2018 - 06:40 AM

Got it.  I wonder which part causes that component to become so much slower when there are other mods installed... maybe the "read_in_spellcodes" macro (which is in lib_macro, which I already had open in my text editor :p )... worth a look.


Faiths & Powers: Spell spheres and kit pack for priests and paladins
Might & Guile: Tweaks and kits for warriors and rogues
Scales of Balance: Game tweaks and rule overhauls
NPC_EE: More options for NPCs in BGEE, SoD, & BG2EE


#5 DavidW

DavidW
  • Gibberlings
  • 4672 posts
  • Gender:Male

Posted 23 August 2018 - 07:06 AM

Just to flag that while I'm delighted when people fix bugs during periods where I'm short of time myself, I'm really reluctant to have people mess with the code otherwise, unless I've been able to check it personally. SCS is extremely complicated and coded very differently from other WEIDU mods.

 

(Which isn't to say I'm not interested in optimization suggestions, as long as they don't cut against other design goals - although those particular functions get implemented in a completely different way on my current beta.)



#6 Bartimaeus

Bartimaeus
  • Members
  • 438 posts
  • Gender:Not Telling

Posted 23 August 2018 - 09:36 AM

If you don't have a clone to git grep through, github is a good basic searching alternative:

https://github.com/G...shared.tph#L170

 

You can also use notepad++'s "search in files" tool.



#7 subtledoctor

subtledoctor
  • Modders
  • 2855 posts
  • Gender:Male

Posted 23 August 2018 - 09:45 AM

To clarify, I'm not doing anything in one of the Github repos. I'm working on a small addition to one of the macros in the downloaded copy of the mod on my computer. Theoretically it could speed up installation of the AI component by an order of magnitude for players with lots of mods. The basic idea is sketched out here:
http://gibberlings3....153#entry258644

If it works well, only then will I present the code to you as a proposed change, either by posting it here, or by PM, or by a pull request on Git. In either case the goal is simply to create something for you to evaluate - not to distribute any changes on my own.

Faiths & Powers: Spell spheres and kit pack for priests and paladins
Might & Guile: Tweaks and kits for warriors and rogues
Scales of Balance: Game tweaks and rule overhauls
NPC_EE: More options for NPCs in BGEE, SoD, & BG2EE


#8 DavidW

DavidW
  • Gibberlings
  • 4672 posts
  • Gender:Male

Posted 23 August 2018 - 02:45 PM

That's fine... but bear in mind that I'm really unlikely to take anyone else's code directly into a release of SCS that I do myself. I'm much more interested in suggestions and identified problems than I am with actual code. And I wouldn't want you or anyone to feel you've done lots of work and then be disappointed that it doesn't get used.

 

(Of course, the current RC release of SCS isn't one I'm doing myself, so taking something into that pro tem is another matter. But my next version is likely to start from the v30 code, and incorporate my versions of the various bugfixes and patches people have done, rather than just start from those patches.)



#9 subtledoctor

subtledoctor
  • Modders
  • 2855 posts
  • Gender:Male

Posted 23 August 2018 - 08:13 PM

Well, maybe I misunderstood one of your other posts.  But anyway I don't want to propose something until I have an idea it can be successfully implemented.

 

The nut of it is that the General AI component is extremely broad in how it applies class abilities to AI .CRE files.  Consider that, due to weird and unfortunate engine limitations, multiclass kits do not use their own class ability tables, but instead use the trueclass tables of their constituent classes - CLABFI01.2da, CLABTH01.2da, etc.  Aquadrizzt wrote up a function that allows any modder who makes new kits to create multiclass kits, automagically converting your kit's custom CLAB table into a bunch of 177 effects targeting your kit, in the trueclass CLAB table.  It works great and it unleashes a whole new area of creative mods that were not fully realizable before.  But it can lead to an AP_ spell being applied at every level of the base CLAB table, and each such spell can have dozens of 177 effects.

 

Similarly, a number of divine-class mods that contain sphere systems use the CLAB tables to grant spells to priests via opcode 171.  This results in a dozen-plus spells being applied to priests, each of which has numerous 171 effects.

 

These kinds of things absolutely choke the SCS AI macros.  I just did a test in BGEE.  I installed a single mod before SCS: Faiths & Powers, which has a few dozen cleric/druid/paladin/ranger kits, a handful of multiclass kits, and a sphere system.  The SCS General AI component went from about an hour to install, to 7 hours. 

 

Here's another example: in attempting to update Refinements for the EE games, I wrote up an alternate version of the Use Scrolls ability, which does not require installing several hundred extra "shadow kits" but does apply a single spell to thieves which contains about 100 opcode 180 effects.  It worked great, but I had to abandon it because it caused the installation of SCS AI to jump to about 20 (!) hours.

 

The point is not to complain; SCS AI is great and I don't mind running it overnight when it results in a better game.  But, none of the things I mentioned above have any effect on enemy creatures.  The 2,000% increase in installation time made zero difference in the game.  So, notwithstanding that you profess not to care about players use other mods in conjunction with SCS (setting aside that describes 99% of users), I still think it bears asking whether SCS can avoid burning so much time and heat and electricity when it doesn't actually do anything.  I'm pretty sure there are no enemy creatures with multiclass kits, and SCS does not assign any such kits to any creatures.  I'm pretty sure that, while SCS will make use of many Refinements HLAs, "Use Scrolls" is not one of them.

 

So I'm poking through the SCS code to see if I can spot a way that it can be tweaked to disregard such spells, getting on quickly with its business of improving the AI while ignoring the stuff that slows it down and doesn't affect the AI anyway.  SCS doesn't have to do anything complicated to achieve this; all it has to do is look at a blacklist of spells to disregard before it processes the game's CLAB tables and applies them to enemies.  It would be the responsibility of other modders to specify which spells should be disregarded.  Those other modders will likely hear reports from players like "your mod makes SCS slow to a crawl" and they will have an incentive to address that concern.  They can do something relatively simple like:

ACTION_IF NOT FILE_EXISTS_IN_GAME ~noscsai.2da~ BEGIN

<<<<<<<< %MOD_FOLDER%/inline/noscsai.2da
2DA V1.0
****
QDMulti     QD_MC>>>>>>>>

 COPY ~%MOD_FOLDER%/inline/noscsai.2da~ ~override/noscsai.2da~

END

APPEND ~override/noscsai.2da~ ~FNP         D5SM
FNP         D5SP
FNP         D5SF ~

And SCS can do something like, look for any spells beginning with the letters in column 2 of "noscsai.2da" and exempt them from consideration in the general AI component.  Theoretically, that little bit of inter-mod communication should result in install times an order of magnitude faster.


Edited by subtledoctor, 23 August 2018 - 08:33 PM.

Faiths & Powers: Spell spheres and kit pack for priests and paladins
Might & Guile: Tweaks and kits for warriors and rogues
Scales of Balance: Game tweaks and rule overhauls
NPC_EE: More options for NPCs in BGEE, SoD, & BG2EE


#10 DavidW

DavidW
  • Gibberlings
  • 4672 posts
  • Gender:Male

Posted 23 August 2018 - 10:03 PM

So: I absolutely agree with you, this is a problem in the SCS enforce_class function that needs fixing. I'm grateful for you catching it; I might well not have noticed it myself.

 

BUT I don't think your solution is optimal, and my own version does it a bit differently.

 

Also:


So, notwithstanding that you profess not to care about players use other mods in conjunction with SCS

 

I don't think I've professed any such thing. If SCS didn't care about other mods, I'd just distribute its end-product and save 95% of install time (and loads of my own time). SCS works really hard to work with whatever's already installed. What I'm less bothered by is supporting megamods where people have hundreds of not-that-carefully-chosen things installed.


Edited by DavidW, 23 August 2018 - 10:03 PM.


#11 ALIENQuake

ALIENQuake
  • Modders
  • 630 posts
  • Gender:Male
  • Location:Poland

Posted 24 August 2018 - 01:07 AM

@DavidW I'm sure that any improvements which you will make for "small/medium BG2 installation" will have positive impact for larger installation.



#12 subtledoctor

subtledoctor
  • Modders
  • 2855 posts
  • Gender:Male

Posted 24 August 2018 - 05:36 AM

Sorry if I misstated. Frankly I have no interest in megamods and I think that anyone who just points BWS to "tactical" has made their bed and they can sleep in it.

But I've noticed that mods that are not huge or overly resource-intensive, like one with a handful of multiclass kits, unfortunately cause problems for SCS by necessity. I don't think this is a fault of design; just a sub-optimal result of interacting with the same parts of the game. It doesn't appear intractable - there seems to be a LOT of room for relatively painless optimization. So, I've been tinkering a bit.

(Unfortunately, my current idea didn't provide nuch of a speed boost, either because I didn't target the best part of SCS' macros or because my regexp didn't work.)

Faiths & Powers: Spell spheres and kit pack for priests and paladins
Might & Guile: Tweaks and kits for warriors and rogues
Scales of Balance: Game tweaks and rule overhauls
NPC_EE: More options for NPCs in BGEE, SoD, & BG2EE


#13 K4thos

K4thos
  • Modders
  • 1208 posts
  • Gender:Male

Posted 24 August 2018 - 06:06 AM

or because my regexp didn't work

 

regexp? Weidu is not suited for that. Just for perspective - I've managed to reduce EET install time from 1 hour to about 10 minutes (from which most of it is unbiffing and biffing). Regexp that on clean game took weidu 40 minutes (or close to 2 hours with mods like BG1NPC Project and some NPCs installed on BG:EE) now takes about 10 seconds (exact same code, just in pure lua, which btw. is infamous for slow regexp processing). Of course I'm not expecting that DavidW will switch to lua, just informing that there are indeed ways to optimize mod installation time drastically (if it's really bcs regexp what slows down SCS)


Edited by K4thos, 24 August 2018 - 07:04 AM.


#14 Jarno Mikkola

Jarno Mikkola

    The Imp

  • Modders
  • 6830 posts
  • Gender:Male
  • Location:The town where the dead haven't keeled over, yet. In Finland.

Posted 24 August 2018 - 06:40 AM

or because my regexp didn't work

regexp? Weidu is not suited for that. Just for perspective - I've managed to reduce EET install time from 1 hour to about 10 minutes (from which most of it is unbiffing and biffing). Regexp that on clean game took weidu 40 minutes (or close to 2 hours with mods like BG1NPC Project and some NPCs installed on BG:EE) now takes about 10 seconds (exact same code, just in pure lua, which btw. is infamous for slow regexp processing). Of course I'm not expecting that DavidW will switch to lua, just informing that there are indeed ways to optimize mod installation time drastically (if it's really regexp what slows down SCS)

This is intresting ... in a minor way to me... cause I can't understand any of that, but... I might be intrested in seeing what the original looked like ? So I could compare and contrast... so I might be able to learn. I have done that with other projects and I can sey it helped lots.
Cause I am pretty sure you aren't exactly doing the "same" thing. Example would be, find Sarevok's .cre file and copy it... you can use COPY, or you can use regexp to find all .cre's and then read a unique fact and then if the file is correct size and valid use a copy... the later being a lot more file reading and slower, but it can deliver better results... while the best results might not be what are wanted.

I can attest that I can appreciate what with subtledoctor's intent here is, and would like to see it realized.

Edited by Jarno Mikkola, 24 August 2018 - 06:42 AM.

Welcome to the sanity, you are free to search for the limit, it's out there, we drew it in the sand.
Here's how to install all the ... mods you ever really could want to Infinity Engine games. I removed the stable word from there as Roxanne began to add BS mods that are likely to break compatibility from the BWS.

#15 K4thos

K4thos
  • Modders
  • 1208 posts
  • Gender:Male

Posted 24 August 2018 - 07:16 AM

I might be intrested in seeing what the original looked like ? So I could compare and contrast...

 

 

the linked file mentions original weidu macro names and file that still has them.





Reply to this topic



  


0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users