subtledoctor Posted August 23, 2018 Share Posted August 23, 2018 Trying to navigate this mod is maddening. I'm playing around with ways to make the install process shorter. I've got an idea for the General AI component, since that takes 6+ hours for me with a bunch of NPC/spell/kit mods. But the Initialize AI component takes about 2 hours as well, with only a single large kitpack installed before SCS. So I'm trying to see if I can streamline things there as well. From the mod's .tp2 I can find my way to /stratagems/caster_shared/initialize.tpa... but there I only see: LAF collect_the_immune END LAF build_instant_spells END LAF build_prebuff_spells END Where are those functions defined? I've tried looking through files in /lib and /sfo but I can't find them. Quote Link to comment
lynx Posted August 23, 2018 Share Posted August 23, 2018 If you don't have a clone to git grep through, github is a good basic searching alternative: https://github.com/Gibberlings3/SwordCoastStratagems/blob/d25ff1b6568690c1bc4dc3b8c8788e4c7d205780/stratagems/caster_shared/caster_shared.tph#L170 Quote Link to comment
Mad Mate Posted August 23, 2018 Share Posted August 23, 2018 All 3 functions are defined in caster_shared\caster_shared.tph Quote Link to comment
subtledoctor Posted August 23, 2018 Author Share Posted August 23, 2018 Got it. I wonder which part causes that component to become so much slower when there are other mods installed... maybe the "read_in_spellcodes" macro (which is in lib_macro, which I already had open in my text editor )... worth a look. Quote Link to comment
DavidW Posted August 23, 2018 Share Posted August 23, 2018 Just to flag that while I'm delighted when people fix bugs during periods where I'm short of time myself, I'm really reluctant to have people mess with the code otherwise, unless I've been able to check it personally. SCS is extremely complicated and coded very differently from other WEIDU mods. (Which isn't to say I'm not interested in optimization suggestions, as long as they don't cut against other design goals - although those particular functions get implemented in a completely different way on my current beta.) Quote Link to comment
Bartimaeus Posted August 23, 2018 Share Posted August 23, 2018 If you don't have a clone to git grep through, github is a good basic searching alternative: https://github.com/Gibberlings3/SwordCoastStratagems/blob/d25ff1b6568690c1bc4dc3b8c8788e4c7d205780/stratagems/caster_shared/caster_shared.tph#L170 You can also use notepad++'s "search in files" tool. Quote Link to comment
subtledoctor Posted August 23, 2018 Author Share Posted August 23, 2018 To clarify, I'm not doing anything in one of the Github repos. I'm working on a small addition to one of the macros in the downloaded copy of the mod on my computer. Theoretically it could speed up installation of the AI component by an order of magnitude for players with lots of mods. The basic idea is sketched out here: http://gibberlings3.net/forums/index.php?showtopic=29153&do=findComment&comment=258644 If it works well, only then will I present the code to you as a proposed change, either by posting it here, or by PM, or by a pull request on Git. In either case the goal is simply to create something for you to evaluate - not to distribute any changes on my own. Quote Link to comment
DavidW Posted August 23, 2018 Share Posted August 23, 2018 That's fine... but bear in mind that I'm really unlikely to take anyone else's code directly into a release of SCS that I do myself. I'm much more interested in suggestions and identified problems than I am with actual code. And I wouldn't want you or anyone to feel you've done lots of work and then be disappointed that it doesn't get used. (Of course, the current RC release of SCS isn't one I'm doing myself, so taking something into that pro tem is another matter. But my next version is likely to start from the v30 code, and incorporate my versions of the various bugfixes and patches people have done, rather than just start from those patches.) Quote Link to comment
subtledoctor Posted August 24, 2018 Author Share Posted August 24, 2018 (edited) Well, maybe I misunderstood one of your other posts. But anyway I don't want to propose something until I have an idea it can be successfully implemented. The nut of it is that the General AI component is extremely broad in how it applies class abilities to AI .CRE files. Consider that, due to weird and unfortunate engine limitations, multiclass kits do not use their own class ability tables, but instead use the trueclass tables of their constituent classes - CLABFI01.2da, CLABTH01.2da, etc. Aquadrizzt wrote up a function that allows any modder who makes new kits to create multiclass kits, automagically converting your kit's custom CLAB table into a bunch of 177 effects targeting your kit, in the trueclass CLAB table. It works great and it unleashes a whole new area of creative mods that were not fully realizable before. But it can lead to an AP_ spell being applied at every level of the base CLAB table, and each such spell can have dozens of 177 effects. Similarly, a number of divine-class mods that contain sphere systems use the CLAB tables to grant spells to priests via opcode 171. This results in a dozen-plus spells being applied to priests, each of which has numerous 171 effects. These kinds of things absolutely choke the SCS AI macros. I just did a test in BGEE. I installed a single mod before SCS: Faiths & Powers, which has a few dozen cleric/druid/paladin/ranger kits, a handful of multiclass kits, and a sphere system. The SCS General AI component went from about an hour to install, to 7 hours. Here's another example: in attempting to update Refinements for the EE games, I wrote up an alternate version of the Use Scrolls ability, which does not require installing several hundred extra "shadow kits" but does apply a single spell to thieves which contains about 100 opcode 180 effects. It worked great, but I had to abandon it because it caused the installation of SCS AI to jump to about 20 (!) hours. The point is not to complain; SCS AI is great and I don't mind running it overnight when it results in a better game. But, none of the things I mentioned above have any effect on enemy creatures. The 2,000% increase in installation time made zero difference in the game. So, notwithstanding that you profess not to care about players use other mods in conjunction with SCS (setting aside that describes 99% of users), I still think it bears asking whether SCS can avoid burning so much time and heat and electricity when it doesn't actually do anything. I'm pretty sure there are no enemy creatures with multiclass kits, and SCS does not assign any such kits to any creatures. I'm pretty sure that, while SCS will make use of many Refinements HLAs, "Use Scrolls" is not one of them. So I'm poking through the SCS code to see if I can spot a way that it can be tweaked to disregard such spells, getting on quickly with its business of improving the AI while ignoring the stuff that slows it down and doesn't affect the AI anyway. SCS doesn't have to do anything complicated to achieve this; all it has to do is look at a blacklist of spells to disregard before it processes the game's CLAB tables and applies them to enemies. It would be the responsibility of other modders to specify which spells should be disregarded. Those other modders will likely hear reports from players like "your mod makes SCS slow to a crawl" and they will have an incentive to address that concern. They can do something relatively simple like: ACTION_IF NOT FILE_EXISTS_IN_GAME ~noscsai.2da~ BEGIN <<<<<<<< %MOD_FOLDER%/inline/noscsai.2da 2DA V1.0 **** QDMulti QD_MC>>>>>>>> COPY ~%MOD_FOLDER%/inline/noscsai.2da~ ~override/noscsai.2da~ END APPEND ~override/noscsai.2da~ ~FNP D5SM FNP D5SP FNP D5SF ~ And SCS can do something like, look for any spells beginning with the letters in column 2 of "noscsai.2da" and exempt them from consideration in the general AI component. Theoretically, that little bit of inter-mod communication should result in install times an order of magnitude faster. Edited August 24, 2018 by subtledoctor Quote Link to comment
DavidW Posted August 24, 2018 Share Posted August 24, 2018 (edited) So: I absolutely agree with you, this is a problem in the SCS enforce_class function that needs fixing. I'm grateful for you catching it; I might well not have noticed it myself. BUT I don't think your solution is optimal, and my own version does it a bit differently. Also: So, notwithstanding that you profess not to care about players use other mods in conjunction with SCS I don't think I've professed any such thing. If SCS didn't care about other mods, I'd just distribute its end-product and save 95% of install time (and loads of my own time). SCS works really hard to work with whatever's already installed. What I'm less bothered by is supporting megamods where people have hundreds of not-that-carefully-chosen things installed. Edited August 24, 2018 by DavidW Quote Link to comment
AL|EN Posted August 24, 2018 Share Posted August 24, 2018 @DavidW I'm sure that any improvements which you will make for "small/medium BG2 installation" will have positive impact for larger installation. Quote Link to comment
subtledoctor Posted August 24, 2018 Author Share Posted August 24, 2018 Sorry if I misstated. Frankly I have no interest in megamods and I think that anyone who just points BWS to "tactical" has made their bed and they can sleep in it. But I've noticed that mods that are not huge or overly resource-intensive, like one with a handful of multiclass kits, unfortunately cause problems for SCS by necessity. I don't think this is a fault of design; just a sub-optimal result of interacting with the same parts of the game. It doesn't appear intractable - there seems to be a LOT of room for relatively painless optimization. So, I've been tinkering a bit. (Unfortunately, my current idea didn't provide nuch of a speed boost, either because I didn't target the best part of SCS' macros or because my regexp didn't work.) Quote Link to comment
K4thos Posted August 24, 2018 Share Posted August 24, 2018 (edited) or because my regexp didn't work regexp? Weidu is not suited for that. Just for perspective - I've managed to reduce EET install time from 1 hour to about 10 minutes (from which most of it is unbiffing and biffing). Regexp that on clean game took weidu 40 minutes (or close to 2 hours with mods like BG1NPC Project and some NPCs installed on BG:EE) now takes about 10 seconds (exact same code, just in pure lua, which btw. is infamous for slow regexp processing). Of course I'm not expecting that DavidW will switch to lua, just informing that there are indeed ways to optimize mod installation time drastically (if it's really bcs regexp what slows down SCS) Edited August 24, 2018 by K4thos Quote Link to comment
Jarno Mikkola Posted August 24, 2018 Share Posted August 24, 2018 (edited) or because my regexp didn't work regexp? Weidu is not suited for that. Just for perspective - I've managed to reduce EET install time from 1 hour to about 10 minutes (from which most of it is unbiffing and biffing). Regexp that on clean game took weidu 40 minutes (or close to 2 hours with mods like BG1NPC Project and some NPCs installed on BG:EE) now takes about 10 seconds (exact same code, just in pure lua, which btw. is infamous for slow regexp processing). Of course I'm not expecting that DavidW will switch to lua, just informing that there are indeed ways to optimize mod installation time drastically (if it's really regexp what slows down SCS) This is intresting ... in a minor way to me... cause I can't understand any of that, but... I might be intrested in seeing what the original looked like ? So I could compare and contrast... so I might be able to learn. I have done that with other projects and I can sey it helped lots. Cause I am pretty sure you aren't exactly doing the "same" thing. Example would be, find Sarevok's .cre file and copy it... you can use COPY, or you can use regexp to find all .cre's and then read a unique fact and then if the file is correct size and valid use a copy... the later being a lot more file reading and slower, but it can deliver better results... while the best results might not be what are wanted. I can attest that I can appreciate what with subtledoctor's intent here is, and would like to see it realized. Edited August 24, 2018 by Jarno Mikkola Quote Link to comment
K4thos Posted August 24, 2018 Share Posted August 24, 2018 I might be intrested in seeing what the original looked like ? So I could compare and contrast... the linked file mentions original weidu macro names and file that still has them. Quote Link to comment
Recommended Posts
Join the conversation
You are posting as a guest. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.