Tran G. (Revernomad) Khoa 85bc406236 fix: smooth GUI during pipeline downloads and harden wizard connection test
GUI log batching (_poll_log now drains queue into a single CTkTextbox.insert
call per 80 ms tick instead of N calls, each with see("end") scroll).

_QueueWriter strips ANSI/CSI escape codes and bare \r before enqueuing so
tqdm progress output is legible in the log textbox. OSC sequences terminated
by both BEL (\x07) and ST (\x1b\) are handled.

Wizard "Test Connection" moved off the main thread: requests.get runs in a
daemon thread; result posted back via after(0, ...). Widget refs captured
before thread launch to prevent stale updates if user navigates away. Bare
except narrowed to TclError (destroyed-widget guard only).

Code quality: import os moved to module level in app.py; _read_raw_config()
helper extracted to deduplicate dual raw config.json reads; return type
annotations added to _get_view_class, _get_dashboard, and cfg property.

Tests: 11 new unit tests for _QueueWriter (RED -> GREEN on OSC-ST fix).
2026-04-08 17:27:25 +07:00
2026-04-08 15:33:58 +07:00
2026-04-08 15:33:58 +07:00
2026-04-08 15:33:58 +07:00
2026-04-08 15:33:58 +07:00
2026-04-08 15:33:58 +07:00

arma-modlist-tools

Python toolchain for managing Arma 3 mod presets: parse launcher exports, compare presets, download mods from a Caddy file server, and create junction/symlink links to an Arma 3 Server installation.


Table of Contents

  1. Prerequisites
  2. Installation
  3. Configuration
  4. Quick Start — Full Pipeline
  5. Individual Scripts
  6. Migrating Existing Mods
  7. Folder Structure
  8. Moving to a New Device
  9. Running Tests

Prerequisites

Requirement Version Notes
Python >= 3.9 python --version
requests any pip install requests
tqdm any pip install tqdm
customtkinter any pip install customtkinter (GUI only)
Windows or Linux Windows uses junctions, Linux uses symlinks

Run the dep checker to confirm everything is ready:

python check_deps.py

Installation

# Clone the repo
git clone https://git.revoluxiant.io.vn/revernomad17/arma-modlist-tools.git
cd arma-modlist-tools

# Install dependencies
pip install -r requirements.txt

# Copy config template and fill in your credentials/paths
cp config.template.json config.json

Edit config.json — see Configuration below.


Configuration

All scripts read a single config.json in the project root. Copy the template and fill in your values:

{
  "server": {
    "base_url": "https://your-caddy-server/arma3mods/",
    "username": "your_username",
    "password": "your_password"
  },
  "paths": {
    "arma_dir":    "C:\\Path\\To\\Arma 3 Server",
    "downloads":   "downloads",
    "modlist_html": "modlist_html",
    "modlist_json": "modlist_json"
  }
}
Key Description
server.base_url Root URL of your Caddy file server. Must have a trailing slash.
server.username HTTP Basic Auth username.
server.password HTTP Basic Auth password.
paths.arma_dir Absolute path to the Arma 3 Server directory where links will be created.
paths.downloads Where mod files are downloaded locally. Relative to project root.
paths.modlist_html Folder containing exported Arma 3 Launcher preset .html files.
paths.modlist_json Folder where parsed/compared JSON files are saved.

Note: config.json is in .gitignore because it contains credentials. Never commit it.


Quick Start — Full Pipeline

Place your Arma 3 Launcher preset exports (.html) into the modlist_html/ folder, then run:

python run.py

This runs all four steps in sequence:

Step 1/4: Parse presets       — modlist_html/*.html  ->  modlist_json/*.json
Step 2/4: Compare presets     — produces modlist_json/comparison.json
Step 3/4: Fetch mods          — downloads from server  ->  downloads/
Step 4/4: Link mods           — creates junctions/symlinks in Arma 3 Server dir

Skip flags

python run.py --skip-fetch --skip-link          # parse + compare only
python run.py --skip-parse --skip-compare --skip-fetch   # link only
python run.py --skip-parse --skip-compare --skip-fetch --group shared
Flag Skips
--skip-parse Step 1 (HTML parsing)
--skip-compare Step 2 (preset comparison)
--skip-fetch Step 3 (downloading)
--skip-link Step 4 (linking)
--group GROUP Link step: only link this one group (e.g. shared)

Safe to re-run. Every step is idempotent — existing files are skipped, already-linked mods are skipped.


Individual Scripts

check_deps.py

Verify Python version and required packages before running anything else.

python check_deps.py
  Python         3.9.2         OK
  OS             Windows Server

  requests       2.33.0        OK
  tqdm           4.67.3        OK

  All checks passed. Ready to run.

parse_modlist.py

Parse all .html preset exports in modlist_html/ and save each as JSON in modlist_json/.

How to get the HTML files: In Arma 3 Launcher → Mods → Preset → right-click → Export to HTML. Place the exported files in modlist_html/.

python parse_modlist.py
150th_MW_2026_v1.0.html  ->  modlist_json/150th_MW_2026_v1.0.json  (52 mods)
150th_WW2_2026_V1.0.html ->  modlist_json/150th_WW2_2026_V1.0.json (48 mods)

Output JSON per preset:

{
  "preset_name": "150th_MW_2026_v1.0",
  "source_file": "150th_MW_2026_v1.0.html",
  "mod_count": 52,
  "mods": [
    {
      "name": "CBA_A3",
      "source": "steam",
      "url": "https://steamcommunity.com/sharedfiles/filedetails/?id=450814997",
      "steam_id": "450814997"
    }
  ]
}

compare_modlists.py

Compare all presets in modlist_html/ and produce modlist_json/comparison.json with a shared/unique breakdown.

python compare_modlists.py
Compared: 150th_MW_2026_v1.0, 150th_WW2_2026_V1.0
  Shared mods           : 28
  Unique to 150th_MW... : 24
  Unique to 150th_WW2...: 20
  -> modlist_json/comparison.json

Output comparison.json structure:

{
  "compared_presets": ["Preset_A", "Preset_B"],
  "shared": {
    "mod_count": 28,
    "mods": [...]
  },
  "unique": {
    "Preset_A": { "mod_count": 24, "mods": [...] },
    "Preset_B": { "mod_count": 20, "mods": [...] }
  }
}

Requires: at least 2 .html files in modlist_html/.


fetch_mods.py

Download all mods from the Caddy file server into downloads/, organized by group.

python fetch_mods.py
Loaded comparison: 150th_MW_2026_v1.0, 150th_WW2_2026_V1.0
Total mods to consider: 72

Building server index...  87 mods indexed

[1/70] @ace  ->  downloads/shared/@ace/  (group: shared)
  addons/ace_common/...    5.2 MB  [################] 100%
  ...
  Done  42 downloaded   8.2 MB

Overwrite existing? [s]kip / [o]verwrite: s
  • Mods shared across all presets go into downloads/shared/
  • Preset-unique mods go into downloads/<preset_name>/
  • Saves modlist_json/missing_report.json listing any mods not found on server
  • Prompts once if destination folders already exist

Requires: modlist_json/comparison.json (run compare_modlists.py first).


Manage junction/symlink links between downloads/ and the Arma 3 Server directory.

Check status

python link_mods.py status --group shared
  Group  : shared
  Path   : downloads/shared
  Arma   : C:\...\Arma 3 Server

  Mod                                                  Status
  ----------------------------------------------------------
  @ace                                                 [LINKED]
  @cba_a3                                              [------]

  1 / 2 linked
python link_mods.py link --group shared
python link_mods.py unlink --group shared

Prompts for confirmation before removing links. Removing a link does not delete the mod files in downloads/.

List available groups

Omit --group to see what groups exist:

python link_mods.py status

Windows: Creates NTFS directory junctions (mklink /J). No administrator rights required. Linux: Creates standard symlinks (os.symlink).


report_missing.py

Check which mods from comparison.json are missing from the file server. Saves modlist_json/missing_report.json.

python report_missing.py
Checking server index...  87 mods indexed
Cross-referencing 72 required mods...

  Missing from server (2):

  steam_id        Group                         Name
  --------------- ----------------------------- ----------------------------------------
  2648308937      150th_WW2_2026_V1.0           IFA3 AIO
  463939057       shared                        ACE3

  70 / 72 found on server
  Report saved: modlist_json/missing_report.json

Requires: modlist_json/comparison.json.


sync_missing.py

Re-check the server for mods that were previously missing and download any that have since been added to the server.

python sync_missing.py
Loading missing report: 2 mods previously missing
Re-checking server index...  89 mods indexed

  Newly available: 1 mods
  Still missing  : 1 mods

  [+] @ace  ->  downloads/shared/
     Done  8.2 MB

  Missing report updated: 1 still missing

  Linking newly added mods...
    shared    1 new linked, 27 already linked

Flow:

  1. Loads modlist_json/missing_report.json
  2. Re-checks server index for newly available mods
  3. Downloads newly available mods to the correct group folder
  4. Updates missing_report.json (removes mods now downloaded)
  5. Runs linker for affected groups — existing links are safely skipped

Requires: modlist_json/missing_report.json (run report_missing.py or fetch_mods.py first).


update_mods.py

Re-download mod files that have changed on the server without changing the modlist structure (same mods, updated file versions).

Detection uses file size comparison: a file is re-downloaded if it is missing locally or its local size differs from the server-reported size. Use --force to re-download everything unconditionally.

python update_mods.py                       # check all groups and mods
python update_mods.py --group shared        # one group only
python update_mods.py --mod @ace            # one specific mod
python update_mods.py --force               # re-download all files
python update_mods.py --force --group shared
Building server index...  87 mods indexed

  Mode: size-check
  Checking 28 mod folder(s)...

  [=] @cba_a3     shared     4 files   up-to-date
  [+] @ace        shared     42 files  3 updated  (8.2 MB)
  [=] @rhsusaf    150th_MW   88 files  up-to-date

  Total: 134 files checked, 3 updated, 8.2 MB downloaded

No re-linking needed. Junctions already point at the downloads/ folders, so updated files are immediately visible to the Arma 3 Server.


check_names.py

Diagnostic tool that compares mod folder names in downloads/ against the server's canonical names, and optionally fixes mismatches.

Status codes:

Status Meaning
OK Disk folder name matches server name exactly
MISMATCH Disk name differs from server canonical name (rename needed)
ID_COLLISION Local meta.cpp has a wrong publishedid that belongs to a different mod
NOT_ON_SERVER No matching folder found on server

Report only

python check_names.py
python check_names.py --group shared    # limit to one group
  Disk name                                     Group                     Status / Server name
  --------------------------------------------  ------------------------  ---------------------------------------------------
  @ace                                          shared                    OK  (@ace)
  @CBA_A3                                       shared                    MISMATCH  ->  @cba_a3
  @NIArms All in One- ACE Compatibility         150th_MW_2026_v1.0        ID_COLLISION  @Realistic Ragdoll Physics (local id: 1234567)
  @150th Languard Zeus Tools                    150th_WW2_2026_V1.0       NOT_ON_SERVER

  80 OK,  1 mismatch,  1 id_collision,  1 not on server

  Run with --fix to rename mismatched folders and --fix-ids to correct wrong steam IDs in meta.cpp.

Fix mismatched folder names

Renames MISMATCH folders on disk to the server's canonical name and updates the arma_dir junction to match.

python check_names.py --fix
  [+] @CBA_A3  ->  @cba_a3

Fix wrong steam IDs

Corrects the publishedid in local meta.cpp files for ID_COLLISION entries. Uses comparison.json as the authoritative source of steam IDs (which came from the Steam Workshop URLs in your HTML presets).

python check_names.py --fix-ids
  [+] @NIArms All in One- ACE Compatibility   meta.cpp: 1234567 -> 9876543

Both fixes at once:

python check_names.py --fix --fix-ids

Requires: modlist_json/comparison.json for --fix-ids (run run.py --skip-fetch --skip-link first).


run.py

Orchestrator that chains all four pipeline steps. Described in Quick Start above.


gui.py

Launch the graphical interface for the toolchain.

python gui.py

Opens a CustomTkinter desktop window with a sidebar navigation and the following views:

View Purpose
Dashboard Overview: status, quick stats, recent activity
Mods Browse and manage downloaded mods by group
Tools Link/unlink, rename, sync missing, check server
Logs Real-time log output from pipeline operations
Settings Edit config.json (server URL, paths, credentials)

On first launch (no config.json), a setup wizard walks you through creating one.

Requires: customtkinter (pip install customtkinter).


Migrating Existing Mods

If the Arma 3 Server already has mods installed and you want to bring them under this toolchain without re-downloading:

Step 1 — Generate comparison.json:

python run.py --skip-fetch --skip-link

Step 2 — Check for name mismatches before linking:

python check_names.py

Step 3 — Fix any issues:

python check_names.py --fix          # rename mismatched folders
python check_names.py --fix-ids      # fix wrong steam IDs in meta.cpp

Step 4 — Create links:

python run.py --skip-parse --skip-compare --skip-fetch

Instead of downloading, you can create junctions from downloads/{group}/@ModName pointing to wherever the mods already live on disk:

mklink /J downloads\shared\@ace "C:\existing\path\@ace"

Then run the link step normally.


Folder Structure

arma-modlist-tools/
|
|- arma_modlist_tools/        # Python package (library code)
|   |- __init__.py            # Public exports
|   |- parser.py              # HTML preset parser
|   |- compare.py             # Preset comparison
|   |- fetcher.py             # Caddy server downloader
|   |- linker.py              # Junction/symlink manager
|   |- reporter.py            # Missing-mod report builder
|   |- config.py              # config.json loader
|   |- compat.py              # OS detection + encoding fix
|
|- gui/                       # GUI package (CustomTkinter desktop app)
|   |- __init__.py            # Theme setup + run_app() entry point
|   |- app.py                 # Main window, view management, pipeline runner
|   |- wizard.py              # First-run config setup wizard
|   |- _constants.py          # Window size, color, path constants
|   |- _io.py                 # stdout/stderr → thread-safe queue for live logs
|   |- views/
|       |- dashboard.py       # Overview view
|       |- mods.py            # Mod browser view
|       |- tools.py           # Tool actions view
|       |- logs.py            # Real-time log view
|       |- settings.py        # Config editor view
|
|- modlist_html/              # INPUT: put your .html preset exports here
|   |- MyPreset_A.html
|   |- MyPreset_B.html
|
|- modlist_json/              # OUTPUT: generated JSON files (gitignored)
|   |- MyPreset_A.json
|   |- MyPreset_B.json
|   |- comparison.json
|   |- missing_report.json
|
|- downloads/                 # OUTPUT: downloaded mod files (gitignored)
|   |- shared/
|   |   |- @ace/
|   |   |- @cba_a3/
|   |- MyPreset_A/
|       |- @rhsusaf/
|
|- config.json                # YOUR config (gitignored — contains credentials)
|- config.template.json       # Template to copy from
|- requirements.txt
|- selection.json             # GUI selection state (persisted between sessions)
|
|- gui.py                     # GUI entry point
|- run.py                     # Orchestrator (parse + compare + fetch + link)
|- parse_modlist.py           # Step 1 standalone
|- compare_modlists.py        # Step 2 standalone
|- fetch_mods.py              # Step 3 standalone
|- link_mods.py               # Link management (status/link/unlink)
|- report_missing.py          # Missing mod report
|- sync_missing.py            # Sync newly available missing mods
|- update_mods.py             # Re-download updated mod files
|- check_names.py             # Diagnose and fix folder name / steam_id issues
|- check_deps.py              # Dependency checker
|- test_suite.py              # Test suite

Moving to a New Device

  1. Clone the repo:

    git clone https://git.revoluxiant.io.vn/revernomad17/arma-modlist-tools.git
    cd arma-modlist-tools
    
  2. Install dependencies:

    pip install -r requirements.txt
    
  3. Create your config:

    cp config.template.json config.json
    # Edit config.json with correct arma_dir and server credentials
    
  4. Add your preset exports: Copy your .html files from the Arma 3 Launcher into modlist_html/.

  5. Verify and run:

    python check_deps.py
    python run.py
    

The downloads/ folder can be several GB. On a second device you can either let run.py re-download everything, or copy the downloads/ folder manually and run python run.py --skip-fetch to skip downloading and just create links.


Running Tests

The test suite covers all modules with 96 tests. No network connection required.

python test_suite.py
------------------------------------------------------------
  compat            6 tests
  config            5 tests
  parser            9 tests
  compare           8 tests
  fetcher          19 tests   (pure functions, no network)
  reporter          8 tests
  linker           12 tests   (uses temp dirs)
  __init__          2 tests
  check_names      16 tests
  integration       2 tests
  gui._io          11 tests   (QueueWriter, no GUI required)
------------------------------------------------------------
  Results: 95 passed, 1 failed, 0 skipped  (96 total)

The 1 failing test is a pre-existing comparison snapshot mismatch unrelated to the GUI changes.

Description
No description provided
Readme 540 KiB
Languages
Python 86.6%
HTML 13.4%