Immutability and encapsulation in mod design

[c4_angel]

thanks

Edited September 24, 2018 by c4_angel

[subtledoctor]

I have questions.

 

1) Is immutability really all that huge a concern? Concrete example: I have a couple mods that need an external space to build e.g. .D files before compiling. Something like:

COPY ~mymod/component/dialog.d~ ~mymod/temp_workspace~
APPEND_OUTER ~mymod/temp_workspace/dialog.d~ ~more stuff~
APPEND_OUTER ~mymod/temp_workspace/dialog.d~ ~even more stuff~
APPEND_OUTER ~mymod/temp_workspace/dialog.d~ ~you get the idea~
COMPILE ~mymod/temp_workspace~

Especially given that successive installs will overwrite the initial file at the outset, I don't see much harm in using a designated, and contained, subdirectory of the mod folder. (I even named the folder "break_glass" in one of the mods, because if anyone gets tripped up on reinstalls they can just purge that folder.) Because TBH I find it extremely annoying how the game folder becomes a riotous dumping ground for mod files. A zillion copies of the Weidu executable, every .debug file, now external workspace directories sitting right in front of the user even though they are very clearly not meant to be touched by the user? I get the principle of immutability here, but is there no way to achieve its benefits without adding even more cruft to a game folder already overfilled with cruft?

 

2) My (admittedly less-than-perfect) understanding of Weidu functions is that they are really designed and meant to be used for allowing input options and variable-determined output. Not as mere wrappers for a bunch of normal Weidu code. For the latter, I have been using macros, and then I "LAM" the component in my .tp2 file.

 

- Do macros similarly clear variables and arrays etc.?

 

- Can we really just drop a bunch of commands into a function and call it a day? I don't have a specific instance at hand, but I definitely remember trying to use DEFINE_ACTION_FUNCTION in that way about a year ago, and Weidu gave me parse errors until I gave up and just called it a macro.

Edited September 24, 2018 by subtledoctor

[Ardanis]

 

For Encapsulation: Why not put CLEAR_EVERYTHING as the first line in ALWAYS? Any disdavantages?

Yes. It means you can't read in data in your ALWAYS that's to be used in any component.

 

Example: SCS's ALWAYS block goes through every spell in the game and sets a variable with name equal to its ids reference, value equal to its spell file. It also goes through and finds a scroll for each wizad spell, and records that as the value of a variable. It takes 2-5 seconds and I'd rather not have those 2-5 seconds every component.

 

At a more conceptual level, I'd like my encapsulation to be built in to the way the code is structured, not forced explicitly. One of the main points of FUNCTIONs is to deliver this; why not use it? Put another way: why not avoid defining variables with global scope in the first place unless you want them to have global scope, rather than just clearing out the global scope on a regular basis?

 

I thought ALWAYS is supposed to run always, i.e. you'll spend 2-5 seconds re-reading data regardless of how you clean up the environment.

 

And while I agree that wrapping the component code is a good habit to have, if you absolutely want to prevent rogue variables slipping into the next component, then always clearing everything *is* how you structure the code to eliminate the human error factor - with a single switch you ensure nothing will slip through even if you forgot to properly encapsulate a component or two - just like you used to forget cleaning up the backup folder.

 

1) Is immutability really all that huge a concern?

It's more like something to be aware of, than a real issue. Unless you do a great lot of stuff in multiple places inside your folder, or forget things easily.

 

2) My (admittedly less-than-perfect) understanding of Weidu functions is that they are really designed and meant to be used for allowing input options and variable-determined output. Not as mere wrappers for a bunch of normal Weidu code. For the latter, I have been using macros, and then I "LAM" the component in my .tp2 file.

The idea was to control the input and output, using only what you want and removing anything else. It's perfectly normal to have no input/output whatsoever, especially if the code can be used across multiple mods/components (look at DS for example).

 

Note that unless it was changed in recent years, functions still accept external input that wasn't forcefully set in function definition.

DEFINE_ACTION_FUNCTION print
  INT_VAR a=1 b=2 BEGIN
    PRINT ~a=%a%, b=%b%, c=%c%~
END
OUTER_SET a=10
OUTER_SET c=30 // you don't really want this to slip inside, but unless I'm mistaken it still does
LAF print INT_VAR b=200 END // will print ~a=1, b=200, c=30~

- Do macros similarly clear variables and arrays etc.?

No.

 

- Can we really just drop a bunch of commands into a function and call it a day? I don't have a specific instance at hand, but I definitely remember trying to use DEFINE_ACTION_FUNCTION in that way about a year ago, and Weidu gave me parse errors until I gave up and just called it a macro.

Yes, you need to define it the same way you define macros. The only difference are the optional INT_VAR, STR_VAR etc. Edited September 24, 2018 by Ardanis

[DavidW]

 

 

For Encapsulation: Why not put CLEAR_EVERYTHING as the first line in ALWAYS? Any disdavantages?

Yes. It means you can't read in data in your ALWAYS that's to be used in any component.

 

Example: SCS's ALWAYS block goes through every spell in the game and sets a variable with name equal to its ids reference, value equal to its spell file. It also goes through and finds a scroll for each wizad spell, and records that as the value of a variable. It takes 2-5 seconds and I'd rather not have those 2-5 seconds every component.

 

At a more conceptual level, I'd like my encapsulation to be built in to the way the code is structured, not forced explicitly. One of the main points of FUNCTIONs is to deliver this; why not use it? Put another way: why not avoid defining variables with global scope in the first place unless you want them to have global scope, rather than just clearing out the global scope on a regular basis?

 

I thought ALWAYS is supposed to run always, i.e. you'll spend 2-5 seconds re-reading data regardless of how you clean up the environment.

 

Most of my ALWAYS block is wrapped in

ACTION_IF !VARIABLE_IS_SET do_this_once BEGIN
OUTER_SET do_this_once=1

[do some stuff]


END

And while I agree that wrapping the component code is a good habit to have, if you absolutely want to prevent rogue variables slipping into the next component, then always clearing everything *is* how you structure the code to eliminate the human error factor - with a single switch you ensure nothing will slip through even if you forgot to properly encapsulate a component or two - just like you used to forget cleaning up the backup folder.

Well, of course I agree that this is as much aesthetics and taste as anything else.

 

That said, if *everything* you do is call functions, there can't be anything left to slip into the next component. (It's not quite true that everything I do is call functions, because my ALWAYS block defines a bunch of variables, but it's relatively easy to keep track of those, and in any case they don't get changed from component to component.)

Edited September 25, 2018 by DavidW

[DavidW]

Because TBH I find it extremely annoying how the game folder becomes a riotous dumping ground for mod files. A zillion copies of the Weidu executable, every .debug file, now external workspace directories sitting right in front of the user even though they are very clearly not meant to be touched by the user? I get the principle of immutability here, but is there no way to achieve its benefits without adding even more cruft to a game folder already overfilled with cruft?

I think this concern is overrated. I mean, tidy up after yourself as much as you can, of course, but all I'm really proposing is adding one folder to what is usually several dozen folders, and I really don't think end users care.

 

Put it this way. I moved SCS over to immutability around v22, which I think was about five years ago. I thought about discussing it publicly then; the reason I didn't was partly that I expected lots of people to complain about stratagems_external. Fast forward 5 years, and so far as I'm aware no-one has even noticed; certainly I've never seen a comment about it, and a google search doesn't show one up.

[Jarno Mikkola]

Put it this way. I moved SCS over to immutability around v22, which I think was about five years ago. I thought about discussing it publicly then; the reason I didn't was partly that I expected lots of people to complain about stratagems_external. Fast forward 5 years, and so far as I'm aware no-one has even noticed; certainly I've never seen a comment about it, and a google search doesn't show one up.

Right. So apparently you never saw this. Too bad you responded to it with:

Yeah, and the SCS takes two folders.

I'm not sure I understand the point you're making.

If you need google to interprite the world for you, you live under that corporate foots repression. Or the mindcontrol of the Google Inc.

Drone.

That's your "no-one has even noticed" gone.

 

I also wanted to, but never did add to the thread in PPG that anyone could throw in a baldur.baf file that would never compile into the stratagems_external/workspace -folder and your mods would never then compile. The reason for this is that I lacked the time to actually search the exact details I need to do this. As I assumed you don't just compile files strait from the folder.

[Ardanis]

 

 

Most of my ALWAYS block is wrapped in

Ah, I see. Point taken.

Well, of course I agree that this is as much aesthetics and taste as anything else.

That said, if *everything* you do is call functions, there can't be anything left to slip into the next component. (It's not quite true that everything I do is call functions, because my ALWAYS block defines a bunch of variables, but it's relatively easy to keep track of those, and in any case they don't get changed from component to component.)

 

I meant it more like - if system's stability depends on plugging ten same holes in different places, then you should assume you'll miss some, or may forget to plug new ones when adding new stuff. Learned it hard way during SoD

[DavidW]

 

Put it this way. I moved SCS over to immutability around v22, which I think was about five years ago. I thought about discussing it publicly then; the reason I didn't was partly that I expected lots of people to complain about stratagems_external. Fast forward 5 years, and so far as I'm aware no-one has even noticed; certainly I've never seen a comment about it, and a google search doesn't show one up.

Right. So apparently you never saw this. Too bad you responded to it with:

 

 

Yeah, and the SCS takes two folders.

I'm not sure I understand the point you're making.

 

 

I'm distinguishing "meaningful complaints" from "incomprehensible gnomic utterances". (But, sure, I don't mind replacing "no-one cared" with "only Jarno cared" if you like. And if our evil corporate overlords are hiding other objectors to me, feel free to point them out.)

 

I also wanted to, but never did add to the thread in PPG that anyone could throw in a baldur.baf file that would never compile into the stratagems_external/workspace -folder and your mods would never then compile. The reason for this is that I lacked the time to actually search the exact details I need to do this. As I assumed you don't just compile files strait from the folder.

 

Bizarrely enough, I don't issue blanket COMPILE commands to files I didn't build myself.

Edited September 25, 2018 by DavidW
marginally toned down the sarcasm

[subtledoctor]

TBH I do find the presence of the strategems_external folder to be slightly annoying. I don't mind because the mod is great. But I might start to get a lot more annoyed if every mod added one.

 

It just really seems like it would be better housekeeping to put all the .debug files into a /debug subdirectory, and all the mod_external folders into a /workspace subdirectory.

 

Obviously this is a very minor gripe.

[AL|EN]

TBH I do find the presence of the strategems_external folder to be slightly annoying. I don't mind because the mod is great. But I might start to get a lot more annoyed if every mod added one.

 

It just really seems like it would be better housekeeping to put all the .debug files into a /debug subdirectory, and all the mod_external folders into a /workspace subdirectory.

 

Obviously this is a very minor gripe.

If you create 'debugs' directory, weidu will put all .debug files there. Anyway, strategems_external is not a problem compared to something like this: https://image.prntscr.com/image/4y645suhRnuMeNSHPLQaww.png

[DavidW]

It just really seems like it would be better housekeeping to put all the .debug files into a /debug subdirectory, and all the mod_external folders into a /workspace subdirectory.

I agree, and effectively that's what I do. "stratagems_external" is shared by SCS and Wheels of Prophecy (both their backup and workspace files live in it), and in fact by the IWD-EE converter, not that I released that publicly. Other people are welcome to use it; I might rename it to "weidu_external" or something next time around, to make that more obvious. (That's what the template on this thread does.)

[Jarno Mikkola]

 

It just really seems like it would be better housekeeping to put all the .debug files into a /debug subdirectory, and all the mod_external folders into a /workspace subdirectory.

I agree, and effectively that's what I do. "stratagems_external" is shared by SCS and Wheels of Prophecy (both their backup and workspace files live in it), and in fact by the IWD-EE converter, not that I released that publicly. Other people are welcome to use it; I might rename it to "weidu_external" or something next time around, to make that more obvious. (That's what the template on this thread does.)

I would recommend that you would go to PPG and ask from Wisp... on which directory he would recommend... and make tutorials about while doing so. The above is good one, I conseed that, if this were legit. Or Wisp could do it here, if here were here ... narf.

PS, I read the .debug file note from subtledoctor as, kinda intended towards Wisp, rather than you alone. Cause the BiG Would tools actually do things similar already, but if all mods would do that, it would help that too... but then the weidu.exe users could be worse off, as requiring to open the folder all the time can be a hassle.

Edited September 25, 2018 by Jarno Mikkola

[AL|EN]

On 9/19/2018 at 3:51 AM, DavidW said:

• Immutability: Nothing in your mod folder should change at all when the mod is installed.

Does creating files inside mod folder is also consider as 'change' in this context? If yes, does user-generated configs should also be outside of the mod folder, in order to achieve 100% mod immutability? Not that it is any kind of the problem with them left inside mod folder, just wanted to clarify the subject.

Edited January 17, 2019 by AL|EN

[DavidW]

10 hours ago, AL|EN said:

Does creating files inside mod folder is also consider as 'change' in this context? 

That depends what does the creating. If something other than the mod itself creates it, it's probably fine. (I suppose if you're being purist then it would be better for all config files, user and otherwise, to be outside the folder, so that you can have different configs on different installs, but at this point it doesn't really matter.) 

[subtledoctor]

On 9/18/2018 at 10:18 PM, DavidW said:

How to make your mod folder immutable

1. Put your backup folder somewhere else. SCS backs up to stratagems_external/backup/stratagems. Wheels of Prophecy backs up to stratagems_external/backup/wheels. My (hypothetical) secret project mod backs up to stratagems_external/backup/secret_project. And if you want to adopt this approach, you're welcome to back up to stratagems_external/backup/mymod. (And if someone wants to try this but balks at having their backup folder SCS-branded, I'm quite happy to shift to a new name - weidu_external, say.

2. Do all your assembly in an external directory. SCS has

OUTER_SPRINT workspace "stratagems_external/workspace"

in its ALWAYS block, and then just copies any file that's being worked on to %workspace%. (SCS and Wheels of Prophecy share a workspace directory, since the working assumption is that all its contents can be discarded at any time.)

So, I'm trying to get my mods to comply with these principles, and slowly making progress.  But some of us need some REAL basic guidance.  In the current one I'm working on, I need to build a .D file dynamically, so I want to do that in a subfolder of "/weidu_external/mymod/workspace" or someplace similar.  But how to create the folder structure?  If I do something like...

COPY ~mymod/files/file1.d~ ~weidu_external/workspace~
	APPEND_OUTER ~weidu_external/workspace/file1.d~ ~stuff~

Will that not overwrite anything?  Create the directory if it doesn't exist, and leave existing files there alone if it does exist?  Do we need to do anything else to conditionally create the folder?

Second: from the examples here and the current SCS behavior, it seems you anticipate that all mods will use /weidu_external/workspace for working files, all mixed together.  You don't think it's worth creating subfolders for each mod?  (From a pure design perspective, that is - set aside namespace issues, which should be avoided by other means.)

Third: for the sake of reducing clutter, is there any reason the .debug logs shouldn't be shunted away into the weidu_external folder?  My MacOS micro mod manager already sequesters them into /logs, but I'm thinking about changing that to /weidu_external/debug_logs...