A central IE mod repository and mod manager?

[corvias]

I need a project. I would like to start a discussion about building a central, mirrorable, repository for infinity engine mods. I myself am not a modder, but I've been playing IE games and mods since the 90s. I DO have some database and web dev skills, though (hobbyist, not pro).

 

What I have in mind should be familiar to any JavaScript web devs and Linux enthusiasts who might be reading. It would be similar to npm, apt, pacman, etc. When a mod creator released a mod, she would submit something to the mod repository with info on the mod and download links.

 

The repo DB would only be the first part of the project. The second would be a cross-platrorm GUI mod manager that used the repo as it's backbone. Players could fire up the mm, it would sync with the online DB, allow them to install mods, inform them of updates, downgrade to older versions, and even export/import mod manifests to easily reproduce different loadouts. Similar to mod packs in other gaming communities.

 

Before I start devoting time to this, I want to guage how much interest the community would have in these tools?

[Roxanne]

Did you check out this BWS https://forums.beamdog.com/discussion/44476/tool-big-world-setup-bws-mod-manager-for-baldurs-gate-enhanced-edition-trilogy-for-windows

 

The team that maintains it can be best reached here http://www.shsforums.net/topic/56670-big-world-setup-an-attempt-to-update-the-program/

[lynx]

I'd start with porting or interfacing BWS' backend to be usable on other systems. It's the closest thing to what you describe.

 

I don't think there'd be that much interest though.

[Bill Bisco]

If you could make your project work across multiple systems Windows, Mac, Linux, Android, Ios. (BWP is windows only). Yes there is a large interest for this and this has been desired for years.

[lynx]

It has been almost 20 years and nobody thought it enough important, even though it's less work than it seems. Even if none of the classic nexus type sites are open source (no idea), an existing package manager could have easily been reused. Yet nobody missed it enough.

[corvias]

I might have gotten ahead of myself. The online DB is the main piece. The mod manager would be longer term. I have looked at BWS. It's not quite what I'm going for. What I'm proposing is to have a single online DB, with an IE mod-specific schema. It would have a minimal web frontend for easy searching, but also a REST interface so people could write software (like mod managers) that queried it for updates or mod metadata. It would NOT store the mod files or host forums. Just links, name, author, version, description, etc.

 

But Lynx makes a valid point: it has been 20 years and the community hasn't seen this as a priority.

[lynx]

BWS is hosted on github and its mod description files have a defined format, so the db part is solved. Nobody wrote a nicer web frontend yet, afaik.

 

As far as extra services, I know only of my tiny rewriter that it uses to lessen the maintenance burden.

https://github.com/Gibberlings3/modhub

[Jarno Mikkola]

I have looked at BWS. It's not quite what I'm going for.

What's the actual difference then ?

 

your project work across multiple systems Windows, Mac, Linux, Android, Ios. (BWP is windows only).

Yeah, and the BWS is what's actually talked about, not the .pdf project that's based on .bat files.

[Mike1072]

I'm a fan of the idea.

 

I think BWS is a very handy tool for automating mod installation, but we could do with a system that isn't tied to AutoIt and is less associated with megamods.

 

Hosting mod metadata and dependency/conflict information in an online database would provide a lot of flexibility. If authors could directly submit and modify their mod information through a web interface, that would give them greater control and encourage them to keep the details up to date.

[corvias]

I'm a fan of the idea.

 

I think BWS is a very handy tool for automating mod installation, but we could do with a system that isn't tied to AutoIt and is less associated with megamods.

 

Hosting mod metadata and dependency/conflict information in an online database would provide a lot of flexibility. If authors could directly submit and modify their mod information through a web interface, that would give them greater control and encourage them to keep the details up to date.

Heck, with a RESTful interface and github, they wouldn't even need to go to the site. Just have a script that sends a .json file containing the metadata it to the mod db when they commit a release to git.

[corvias]

 

I have looked at BWS. It's not quite what I'm going for.

What's the actual difference then ?

 

your project work across multiple systems Windows, Mac, Linux, Android, Ios. (BWP is windows only).

Yeah, and the BWS is what's actually talked about, not the .pdf project that's based on .bat files.

 

BWS is a logical application of the BWP guide. It's meant for automating megainstalls. The initial purpose of my proposal is for cataloging and searching, as well as a datastore to build new tools off of. No offense to the creators of BWS, but their format for their description files isn't suited for this. It makes sense for a desktop app, but not an online DB. What I was thinking was using couchDB, which stores data in json. Once the data is in there, you can query it and ask it to return the data in whatever standard or homebrew form you wanted: .json, html, xml, csv, ini, etc. It's relatively easy to write new output functions.

[lynx]

Well yeah, ini files aren't that flexible, but they'd be a fine base, especially since you don't need any tools to edit them (and json doesn't support comments unless you have a parser extension). The more you have to reinvent and maintain, the lesser chances of success. A github hook looks like a natural candidate to get custom imports without needing to duplicate base data maintenance.

[AL|EN]

 

BWS is hosted on github and its mod description files have a defined format, so the db part is solved. Nobody wrote a nicer web frontend yet, afaik.

Are you thinking about parser for online BWS ini files which will show mod info and conflicts? And putting it as a webpage? Nice idea

 

Hosting mod metadata and dependency/conflict information in an online database would provide a lot of flexibility.

It's already hosted as online database:

 

- mod list https://bitbucket.org/BigWorldSetup/bigworldsetup/src/4cf6f2bd421a1bf5eae29658e8461bd01531e294/BiG%20World%20Setup/Config/Global/?at=master

- conflicts for BG2/BG2EE/BGT/EET https://bitbucket.org/BigWorldSetup/bigworldsetup/src/4cf6f2bd421a1bf5eae29658e8461bd01531e294/BiG%20World%20Setup/Config/BWP/Game.ini?at=master

 

So you can get all the data which you need by reading BWS Docs and parsing online ini BWS files.

 

 

If authors could directly submit and modify their mod information through a web interface, that would give them greater control and encourage them to keep the details up to date.

 

They already can "directly submit and modify their mod information through a web interface" - by making a forum post "there is new version of my mod, do all BWS related work for me"

Lucky for BWS mantainers, some modders provide new template with new data and I'm grateful for those additional effort. They also can send request / Pull Request directly into BWS if they don't agree about BWS conflicts. There was never a situation when modders was not able to contribute into BWS by simply telling us what they want.

 

How the online database and web interface could solve fundamental issue: most of the modders doesn't care about BWS/mod databases because it involves more work for them.

 

Heck, with a RESTful interface and github, they wouldn't even need to go to the site. Just have a script that sends a .json file containing the metadata it to the mod db when they commit a release to git.

 

That's is something which I and you could do bun not people who post their mods at beamdog forums as an attachment with files for override folder. The only situation when such 'script' would work is when executing it, will dynamically generate everything which is required for BWS/database information: mod name

mod version

component list

launguage list

requirements for components (supported games, supported game version, require of internal mod components, require of external mod components) - Currently is impossible due to weidu limitation

possible answer list for components - Currently is impossible due to weidu limitation

 

( and finally maybe components installation order for eg, Scales of Balance, component 180 after SCS and atweaks )

 

I'm happy that new ideas are coming out, but when I look at the overall picture of the games/mods/modders/sites/hosintg/tools/BWS, i see main problems/blockers to make things better for community. Nothing is related to the problem "how the data is stored", it would be then very easy to improve.

[Wisp]

Out of curiosity, what's hidden behind the concept of "mod manager"? I mean, WeiDU keeps track of current state and allows you to atomically add and remove mods, but obviously there's something else to which it refers.

[subtledoctor]

I feel like it generally implies something like BWS or FOMM: a program with a GUI, where you can see and "manage" mods, and then install everything at once by hitting "go."

 

In my mind Weidu is a mod installer program, with some very nice management features.