Compare commits

...

13 Commits

Author SHA1 Message Date
Tran G. (Revernomad) Khoa
b24828ac68 docs: update CLAUDE.md, README, and Vietnamese guide for migration step
- CLAUDE.md: document migrator.py algorithm and junction-removal rationale
- README.md: pipeline now 5 steps, add --skip-migrate flag, add migrator.py
  to folder structure, update test count 142 -> 158
- docs/huong-dan-su-dung.md: 5-step pipeline table, new glossary entry,
  updated footer version note
2026-04-14 15:11:39 +07:00
Tran G. (Revernomad) Khoa
48637ffe90 feat: add migrate step to move mod folders between groups on preset change
Before step_fetch, scan all downloads/ subdirs and move any mod that
comparison.json now assigns to a different group. Matching uses steam_id
(via meta.cpp publishedid) first, normalized name as fallback.

Stale junctions in arma_dir are removed before the folder move so
step_link can re-create them pointing to the new location.

- New arma_modlist_tools/migrator.py: migrate_mod_groups()
- run.py: step_migrate(), --skip-migrate flag, wired into dispatch loop
- gui/app.py: step_migrate inserted as Step 3/5 between compare and fetch
- gui/locales.py: add step3/4/5 names (en + vi), renumber old 3->4, 4->5
- test_suite.py: 7 new migrator tests (158 total, 0 failed)
2026-04-14 15:08:40 +07:00
Tran G. (Revernomad) Khoa
45cb023513 fix: align mods view column headers with data rows
- Use padx=(8,4) consistently for the name column in both header and rows,
  removing the leading-space text hack in row labels
- Add a 16px spacer at the right end of the header to compensate for
  CTkScrollableFrame's internal scrollbar width
- Centre-align Downloaded and Linked columns (header + tick/cross labels)
- Unify update button width to 80px to match header col width
2026-04-14 14:42:29 +07:00
Tran G. (Revernomad) Khoa
ecfa5fa636 fix: match mod folder by steam_id when folder name diverges from modlist name
_find_folder in mods.py now has a fourth fallback: reads publishedid from
meta.cpp inside each candidate folder and matches against mod["steam_id"].
Fixes mods appearing as "not downloaded" when the folder name on disk differs
from the name in the modlist but the mod content (meta.cpp) is correct.

Also adds 8 tests covering all four match strategies and edge cases.
2026-04-14 14:37:32 +07:00
Tran G. (Revernomad) Khoa
fd513b3688 docs: document run_tool UTF-8 encoding fix and update_mods orphan removal 2026-04-11 09:28:39 +07:00
Tran G. (Revernomad) Khoa
50990cca4e feat: remove orphan local files that no longer exist on server in update_mods 2026-04-11 09:21:45 +07:00
Tran G. (Revernomad) Khoa
4fde566cf4 Merge branch 'main' of https://git.revoluxiant.io.vn/revernomad17/arma-modlist-tools 2026-04-10 22:38:20 +07:00
Tran G. (Revernomad) Khoa
68fcaaf6d9 fix: decode run_tool subprocess output as UTF-8 to handle tqdm Unicode chars on Windows 2026-04-10 22:37:35 +07:00
revernomad17
06f0c6eb92 fix: guard against None stdout in fix_console_encoding for pythonw.exe
When launched via pythonw.exe (no console), sys.stdout/stderr are None.
Accessing .encoding on None raised AttributeError, caught by the GUI's
pipeline import guard and shown as 'Failed to load pipeline'. Added
None check before the encoding check in fix_console_encoding(), added
a test, and documented the pitfall in CLAUDE.md.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-09 10:10:13 +07:00
Tran G. (Revernomad) Khoa
3276f4b63f fix: silent pipeline log and server indexing progress
Three issues caused the Logs view to appear blank during a real pipeline run:

1. `from run import step_fetch, step_link` was outside the worker's
   try/except/finally. An import failure silently killed the thread,
   leaving _pipeline_done uncalled and the Run button stuck disabled
   forever. Now wrapped in its own try/except that posts the error to
   the log and resets the UI.

2. `build_server_index` makes N sequential HTTP requests (one per mod
   folder's meta.cpp) with no output during the scan. Added an optional
   `progress_fn(current, total, name)` callback; step_fetch wires it to
   print progress every 25 folders so the log never goes silent.

3. No immediate feedback after clicking Start — the log was blank until
   the worker thread started printing. Now posts a "Pipeline started"
   banner from the main thread before the worker launches.
2026-04-08 23:35:26 +07:00
Tran G. (Revernomad) Khoa
e0c2dfb32a docs: update README and VI guide for orphan cleanup and test count
- Add quick-reference card at top of README (short-form cheat sheet)
- Document clean_orphans.py in ToC, Individual Scripts, folder structure
- Update GUI Tools view table: note Clean Orphans tab
- Fix Running Tests: 96 → 142, remove stale '1 failing test' note, list all 15 groups
- Add clean_orphans.py to folder structure tree
- Add cleaner.py to arma_modlist_tools/ listing
- VI guide: add summary table for Tools tabs, document Clean Orphans tab in full
- VI guide: add 'Mod thừa / Orphan' to glossary
2026-04-08 20:11:41 +07:00
Tran G. (Revernomad) Khoa
5c824280c6 chore: ignore coverage artifacts (.coverage, *.py,cover, htmlcov/) 2026-04-08 20:04:02 +07:00
Tran G. (Revernomad) Khoa
90cc6c00ff feat: add orphan mod cleanup tool with GUI integration and live-server tests
- Add arma_modlist_tools/cleaner.py: find_orphan_folders() detects @ModName
  folders no longer referenced in comparison.json; uses _normalize_name from
  fetcher for consistent three-level matching
- Add clean_orphans.py: CLI with --dry-run and --yes/-y flags; junction-safe
  deletion via _is_junction() guard before shutil.rmtree
- Add Clean Orphans tab to gui/views/tools.py: scrollable checkbox list,
  background scan/delete threads, pending-done-msg pattern for post-scan
  status, EN/VI localization strings in gui/locales.py
- Add 23 unit tests (section 12), 6 E2E subprocess tests (section 13),
  23 coverage-gap tests (section 14), 9 live-server fetcher tests (section 15)
- Fix leaked builtins.open mock in _test_read_os_release_parses_file
- Overall coverage: 84% → 93%; fetcher.py: 36% → 72%
2026-04-08 20:02:42 +07:00
17 changed files with 2012 additions and 97 deletions

7
.gitignore vendored
View File

@@ -16,6 +16,13 @@ dist/
build/
.eggs/
# Coverage
.coverage
*.cover
*.py,cover
htmlcov/
coverage.xml
# Virtual environments
venv/
.venv/

View File

@@ -101,17 +101,40 @@ Pass 2 builds `ok_disk_names` — the set of disk names that already match the s
- `logs.py` — real-time log viewer fed from the stdout/stderr queue
- `settings.py` — in-app editor for `config.json` (server URL, paths, credentials)
**`_find_folder` (mods.py) — three-level name matching:** The mods view resolves a mod's local folder by mod name from `comparison.json`, which may differ from the server-canonical folder name used by the fetcher. Lookup order:
**`_find_folder` (mods.py) — four-level name matching:** The mods view resolves a mod's local folder by mod name from `comparison.json`, which may differ from the server-canonical folder name used by the fetcher. Lookup order:
1. Exact: `@{mod_name}`
2. Case-insensitive: `@CBA_A3` matches `CBA_A3`
3. Normalized (`_normalize_name`): strips all non-alphanumeric — handles punctuation/spacing differences, e.g. `@US GEAr- Units (IFA3)` matches `US GEAr: Units (IFA3)` (both → `usgearunitsifa3`)
4. Steam ID via `meta.cpp`: reads `publishedid` from each folder's `meta.cpp` and matches against `mod["steam_id"]` — handles the case where the folder name bears no resemblance to the modlist name but the mod content is correct
**`selection.json`** — GUI selection state file, tracked in git. Persists which mods/groups are selected between GUI sessions. Written by the GUI; safe to delete (GUI recreates it on next save).
**`run_tool` subprocess streaming:** Tool scripts are launched via `subprocess.Popen` (not `subprocess.run`) with `stdout=PIPE, stderr=STDOUT`, read line-by-line via `iter(proc.stdout.readline, "")`, and posted to the log queue immediately. Python's own output buffering is disabled with the `-u` flag and `PYTHONUNBUFFERED=1` in the environment — without these, output would batch inside the pipe and only appear when the script exits.
**`run_tool` subprocess streaming:** Tool scripts are launched via `subprocess.Popen` (not `subprocess.run`) with `stdout=PIPE, stderr=STDOUT`, read line-by-line via `iter(proc.stdout.readline, "")`, and posted to the log queue immediately. Python's own output buffering is disabled with the `-u` flag and `PYTHONUNBUFFERED=1` in the environment — without these, output would batch inside the pipe and only appear when the script exits. The `Popen` call uses `encoding="utf-8", errors="replace"` and sets `PYTHONUTF8=1` in the child environment so that tqdm's Unicode block characters (e.g. `▉`) don't crash the pipe reader on Windows, where the default `charmap` codec cannot decode them.
**GUI threading model:** Every network or long-running operation runs in a `threading.Thread(daemon=True)` so the Tkinter event loop is never blocked. The only safe way to update widgets from a background thread is `self.after(0, callback)` — never touch widgets directly from a worker thread. `_poll_log` drains the entire log queue in one `after(80, ...)` tick and does a single batched `CTkTextbox.insert()` call rather than one per log entry, keeping the UI smooth even when `tqdm` emits many rapid updates during downloads. The wizard's "Test Connection" button follows the same pattern: `requests.get` runs in a daemon thread; the result is posted back via `self.after(0, ...)` with widget references captured *before* the thread starts, so stale references cannot update the wrong widgets if the user navigates away mid-request.
**`run_pipeline` worker — import guard:** `from run import step_fetch, step_link` is performed inside its own `try/except` *before* stdout is redirected. If this import fails for any reason the exception is posted to the log via `self.after(0, ...)` and `_pipeline_done` is called so the UI resets cleanly. Previously an import failure would silently kill the worker thread and leave the pipeline button disabled forever.
**`build_server_index` progress callback:** Accepts an optional `progress_fn(current, total, name)` callback. `step_fetch` in `run.py` uses this to print `Indexing N/M: @FolderName` every 25 folders so the log never goes silent during the server scan phase. The library itself never calls `print` — the caller owns the I/O.
### `migrator.py` — mod group migration
Before `step_fetch` runs, `step_migrate` moves locally-downloaded mod folders to match the group assignments in the new `comparison.json`. This avoids re-downloading mods that already exist on disk under a different group when presets are switched (e.g. `A``A_v1`).
**Algorithm:**
1. `_build_local_index(downloads)` — scans every `downloads/{group}/@Folder`, reads `meta.cpp` to extract `publishedid`, builds `{steam_id → (group, path)}` and `{norm_name → (group, path)}` maps.
2. `_build_target_list(comparison)` — flattens `comparison.json` into `[(new_group, steam_id, mod_name)]`.
3. For each target: locate mod on disk (steam_id first, normalised name fallback); skip if already in correct group or destination exists; remove stale junction from `arma_dir` if present; move folder with `shutil.move`.
**Junction removal is critical:** a stale junction (target moved away) still has the reparse point attribute, so `_is_junction()` returns `True` and `link_group` would skip it as `already_linked` without recreating it at the new path. Removing the junction before the move lets `step_link` recreate it correctly.
**CLI:** `python run.py --skip-migrate` bypasses the step if needed.
### `update_mods.py` — orphan file removal
After downloading updated files, `update_mods.py` compares every file in the local mod folder against the server's file list and **deletes any local files that no longer exist on the server**. This prevents stale `.pbo` or `.bisign` files from accumulating when a mod's content changes upstream. Each removed file is logged as `[-] orphan removed: <rel_path>` and the final summary line includes an orphan count. The orphan check runs even when no files need downloading (e.g. timestamps match but the local folder has extras).
### GUI localization (`gui/locales.py`)
All user-facing strings are centralised in `gui/locales.py`. Two languages are supported: English (`"en"`) and Vietnamese (`"vi"`).
@@ -143,6 +166,10 @@ get_language() # → "vi"
Minimum is Python **3.9**. All files that use `X | Y` union type annotations **must** have `from __future__ import annotations` as the first import. Without it, the `|` syntax raises `TypeError` at runtime on Python < 3.10. Every module in `arma_modlist_tools/` already has it; any new CLI script you add must include it too.
### `fix_console_encoding` — `None` stdout guard
When the GUI is launched via `pythonw.exe` (no console window), Python sets `sys.stdout` and `sys.stderr` to `None`. `fix_console_encoding()` must check `if sys.stdout is None or sys.stderr is None: return` **before** accessing `.encoding`, otherwise it raises `AttributeError: 'NoneType' object has no attribute 'encoding'`. This error surfaces in the GUI as *"Failed to load pipeline"* because `run.py` calls `fix_console_encoding()` at module level and the exception is caught by the pipeline import guard.
## Test Suite
`test_suite.py` uses a custom harness (no pytest/unittest dependency). Structure:

104
README.md
View File

@@ -4,6 +4,31 @@ Python toolchain for managing Arma 3 mod presets: parse launcher exports, compar
---
## Quick Reference
```bash
# First time setup
cp config.template.json config.json # fill in server URL + credentials + arma_dir
python check_deps.py # verify dependencies
# Day-to-day: full pipeline
python run.py # parse → compare → migrate → download → link
# GUI (recommended)
python gui.py
# Maintenance
python clean_orphans.py --dry-run # find stale mod folders from old presets
python update_mods.py # re-download changed files (size-check)
python sync_missing.py # retry mods that were absent from server
python check_names.py --fix # fix folder name mismatches
# Testing
python test_suite.py # 158 tests (network tests auto-skip if offline)
```
---
## Table of Contents
1. [Prerequisites](#prerequisites)
@@ -20,6 +45,7 @@ Python toolchain for managing Arma 3 mod presets: parse launcher exports, compar
- [sync_missing.py](#sync_missingpy)
- [update_mods.py](#update_modspy)
- [check_names.py](#check_namespy)
- [clean_orphans.py](#clean_orphanspy)
- [run.py](#runpy)
- [gui.py](#guipy)
6. [Migrating Existing Mods](#migrating-existing-mods)
@@ -107,29 +133,34 @@ Place your Arma 3 Launcher preset exports (`.html`) into the `modlist_html/` fol
python run.py
```
This runs all four steps in sequence:
This runs all five steps in sequence:
```
Step 1/4: Parse presets — modlist_html/*.html -> modlist_json/*.json
Step 2/4: Compare presets — produces modlist_json/comparison.json
Step 3/4: Fetch mods — downloads from server -> downloads/
Step 4/4: Link mods creates junctions/symlinks in Arma 3 Server dir
Step 1/5: Parse presets — modlist_html/*.html -> modlist_json/*.json
Step 2/5: Compare presets — produces modlist_json/comparison.json
Step 3/5: Migrate mod groups — moves existing folders to match new group assignments
Step 4/5: Fetch mods — downloads from server -> downloads/
Step 5/5: Link mods — creates junctions/symlinks in Arma 3 Server dir
```
The **migrate step** avoids re-downloading mods that already exist on disk when you switch preset versions (e.g. `A``A_v1`). It matches mods by steam ID (via `meta.cpp`) and moves the folder to the correct group, removing any stale junction first so the link step can re-create it at the new path.
### Skip flags
```bash
python run.py --skip-fetch --skip-link # parse + compare only
python run.py --skip-fetch --skip-link # parse + compare + migrate only
python run.py --skip-parse --skip-compare --skip-fetch # link only
python run.py --skip-parse --skip-compare --skip-fetch --group shared
python run.py --skip-migrate # skip auto-migration
```
| Flag | Skips |
|------|-------|
| `--skip-parse` | Step 1 (HTML parsing) |
| `--skip-compare` | Step 2 (preset comparison) |
| `--skip-fetch` | Step 3 (downloading) |
| `--skip-link` | Step 4 (linking) |
| `--skip-migrate` | Step 3 (mod group migration) |
| `--skip-fetch` | Step 4 (downloading) |
| `--skip-link` | Step 5 (linking) |
| `--group GROUP` | Link step: only link this one group (e.g. `shared`) |
> **Safe to re-run.** Every step is idempotent — existing files are skipped, already-linked mods are skipped.
@@ -468,9 +499,39 @@ python check_names.py --fix --fix-ids
---
### clean_orphans.py
Find and optionally delete orphaned mod folders — `downloads/{group}/@ModName` folders that are no longer referenced in `comparison.json`. These accumulate when you switch presets and re-run the pipeline without cleaning up old downloads.
```bash
python clean_orphans.py # list orphans, prompt for confirmation
python clean_orphans.py --dry-run # list orphans, do not delete
python clean_orphans.py --yes # list and delete without prompting
```
```
Group Folder Size
---------------------------- -------------------------------- ----------
shared @OldMod 124.5 MB
150th_WW2_2026_V1.0 @SomeMod 88.2 MB
2 orphan(s) found — 212.7 MB total
Delete all orphans? [y/N]
```
- Matches by normalized name (same logic as the fetcher), so spacing/capitalization differences are handled correctly
- Junction-safe: uses `os.rmdir()` on junctions rather than `shutil.rmtree` to avoid deleting target files
**Requires:** `modlist_json/comparison.json` (run `run.py --skip-fetch --skip-link` first).
The same functionality is available as the **Clean Orphans** tab in the GUI.
---
### run.py
Orchestrator that chains all four pipeline steps. Described in [Quick Start](#quick-start--full-pipeline) above.
Orchestrator that chains all five pipeline steps. Described in [Quick Start](#quick-start--full-pipeline) above.
---
@@ -488,10 +549,12 @@ Opens a CustomTkinter desktop window with a sidebar navigation and the following
|------|---------|
| Dashboard | Overview: status, quick stats, recent activity |
| Mods | Browse and manage downloaded mods by group |
| Tools | Link/unlink, rename, sync missing, check server |
| Tools | Link/unlink, rename, sync missing, check server, **clean orphans** |
| Logs | Real-time log output from pipeline operations |
| Settings | Edit `config.json` (server URL, paths, credentials) |
The **Tools** view has five tabs: Check Names, Update Mods, Link Mods, Sync Missing / Report Missing, and **Clean Orphans** (find and delete stale mod folders from old presets).
On first launch (no `config.json`), a setup wizard walks you through creating one.
**Requires:** `customtkinter` (`pip install customtkinter`).
@@ -543,6 +606,8 @@ arma-modlist-tools/
| |- fetcher.py # Caddy server downloader
| |- linker.py # Junction/symlink manager
| |- reporter.py # Missing-mod report builder
| |- cleaner.py # Orphan folder detection
| |- migrator.py # Mod group migration (move folders to match comparison.json)
| |- config.py # config.json loader
| |- compat.py # OS detection + encoding fix
|
@@ -591,8 +656,9 @@ arma-modlist-tools/
|- sync_missing.py # Sync newly available missing mods
|- update_mods.py # Re-download updated mod files
|- check_names.py # Diagnose and fix folder name / steam_id issues
|- clean_orphans.py # Find and delete orphaned mod folders
|- check_deps.py # Dependency checker
|- test_suite.py # Test suite
|- test_suite.py # Test suite (158 tests)
```
---
@@ -631,7 +697,7 @@ arma-modlist-tools/
## Running Tests
The test suite covers all modules with 96 tests. No network connection required.
The test suite covers all modules with 158 tests. Network tests auto-skip when the server is unreachable.
```bash
python test_suite.py
@@ -639,19 +705,23 @@ python test_suite.py
```
------------------------------------------------------------
compat 6 tests
compat 11 tests
config 5 tests
parser 9 tests
compare 8 tests
fetcher 19 tests (pure functions, no network)
fetcher 24 tests (pure functions + mock)
reporter 8 tests
linker 12 tests (uses temp dirs)
__init__ 2 tests
check_names 16 tests
integration 2 tests
gui._io 11 tests (QueueWriter, no GUI required)
cleaner 8 tests
e2e — clean_orphans 6 tests (subprocess CLI)
coverage gaps 23 tests (mocked platform branches)
gui.views.mods 8 tests (_find_folder matching)
migrator 7 tests (group migration logic)
live server 9 tests (skipped if server unreachable)
------------------------------------------------------------
Results: 95 passed, 1 failed, 0 skipped (96 total)
Results: 158 passed, 0 failed, 0 skipped (158 total)
```
> The 1 failing test is a pre-existing comparison snapshot mismatch unrelated to the GUI changes.

View File

@@ -11,6 +11,8 @@ from .linker import (
from .config import load_config, Config
from .compat import is_windows, is_linux, get_os_label, fix_console_encoding
from .reporter import build_missing_report, save_missing_report
from .cleaner import find_orphan_folders, folder_size
from .migrator import migrate_mod_groups
__all__ = [
# parser
@@ -29,4 +31,8 @@ __all__ = [
"is_windows", "is_linux", "get_os_label", "fix_console_encoding",
# reporter
"build_missing_report", "save_missing_report",
# cleaner
"find_orphan_folders", "folder_size",
# migrator
"migrate_mod_groups",
]

View File

@@ -0,0 +1,80 @@
"""
arma_modlist_tools.cleaner
~~~~~~~~~~~~~~~~~~~~~~~~~~
Identify orphaned mod folders in the downloads directory.
An *orphan* is a downloaded ``@ModName`` folder that is no longer referenced
by any group in ``comparison.json``. This happens when the user swaps out a
modlist preset and re-runs the compare step — mods that were removed from the
preset remain on disk but are no longer tracked.
Typical usage::
from arma_modlist_tools.cleaner import find_orphan_folders
comparison = json.loads(Path("modlist_json/comparison.json").read_text())
orphans = find_orphan_folders(Path("downloads"), comparison)
for o in orphans:
print(o["group"], o["name"], o["size"])
"""
from __future__ import annotations
from pathlib import Path
from .fetcher import _normalize_name as _normalize
def folder_size(path: Path) -> int:
"""Return the total size in bytes of all files under *path* (recursive)."""
return sum(f.stat().st_size for f in path.rglob("*") if f.is_file())
def find_orphan_folders(
downloads: Path,
comparison: dict,
) -> list[dict]:
"""Return a list of orphan mod folder entries.
A folder ``downloads/{group}/@ModName`` is considered an orphan when its
normalised name does not match any mod in *comparison* under the same
group. Groups in ``downloads/`` that do not exist in *comparison* at all
are treated as entirely orphaned.
:param downloads: Path to the ``downloads/`` directory.
:param comparison: Parsed ``comparison.json`` dict (output of
:func:`~arma_modlist_tools.compare.compare_presets`).
:returns: List of dicts, each with:
- ``path`` — absolute :class:`~pathlib.Path` of the folder
- ``group`` — group name (e.g. ``"shared"``)
- ``name`` — folder name as it appears on disk (e.g. ``"@ace"``)
- ``size`` — total size in bytes (recursive)
"""
# Build group → set-of-normalised-mod-names from comparison data
known: dict[str, set[str]] = {}
for mod in comparison.get("shared", {}).get("mods", []):
known.setdefault("shared", set()).add(_normalize(mod["name"]))
for preset, pdata in comparison.get("unique", {}).items():
for mod in pdata.get("mods", []):
known.setdefault(preset, set()).add(_normalize(mod["name"]))
orphans: list[dict] = []
if not downloads.is_dir():
return orphans
for group_dir in sorted(downloads.iterdir()):
if not group_dir.is_dir():
continue
group_known = known.get(group_dir.name, set()) # empty → group removed
for mod_dir in sorted(group_dir.iterdir()):
if not mod_dir.is_dir() or not mod_dir.name.startswith("@"):
continue
if _normalize(mod_dir.name) not in group_known:
orphans.append({
"path": mod_dir,
"group": group_dir.name,
"name": mod_dir.name,
"size": folder_size(mod_dir),
})
return orphans

View File

@@ -102,6 +102,8 @@ def fix_console_encoding() -> None:
"""
if not is_windows():
return
if sys.stdout is None or sys.stderr is None:
return
if sys.stdout.encoding and sys.stdout.encoding.lower() == "utf-8":
return
sys.stdout = io.TextIOWrapper(sys.stdout.buffer, encoding="utf-8", errors="replace")

View File

@@ -81,7 +81,11 @@ def make_session(auth: tuple[str, str]) -> requests.Session:
return s
def build_server_index(base_url: str, auth: tuple[str, str]) -> dict:
def build_server_index(
base_url: str,
auth: tuple[str, str],
progress_fn: "Callable[[int, int, str], None] | None" = None,
) -> dict:
"""
Scan the root of the file server and build mod lookup maps.
@@ -90,6 +94,9 @@ def build_server_index(base_url: str, auth: tuple[str, str]) -> dict:
:param base_url: Root URL of the Caddy file server (trailing slash optional).
:param auth: ``(username, password)`` tuple for HTTP Basic Auth.
:param progress_fn: Optional callback called as ``progress_fn(current, total, name)``
after each folder is processed. Use it to report progress without
coupling the library to ``print`` or any specific I/O sink.
:returns: Dict with keys:
- ``by_steam_id`` — ``{steam_id: folder_url}``
@@ -100,11 +107,12 @@ def build_server_index(base_url: str, auth: tuple[str, str]) -> dict:
root = base_url.rstrip("/") + "/"
items = _list_dir(root, session)
folders = [it for it in items if it.get("is_dir")]
total = len(folders)
by_steam_id: dict[str, str] = {}
by_name: dict[str, str] = {}
for folder in folders:
for i, folder in enumerate(folders, 1):
name = folder["name"].strip("/")
url = _folder_url(root, name)
by_name[_normalize_name(name)] = url
@@ -118,6 +126,9 @@ def build_server_index(base_url: str, auth: tuple[str, str]) -> dict:
except requests.RequestException:
pass # meta.cpp missing or unreachable — name-based fallback still works
if progress_fn is not None:
progress_fn(i, total, name)
return {"by_steam_id": by_steam_id, "by_name": by_name, "folders": folders}

View File

@@ -0,0 +1,186 @@
"""
arma_modlist_tools.migrator
~~~~~~~~~~~~~~~~~~~~~~~~~~~
Move locally-downloaded mod folders to match the group assignments in
comparison.json, avoiding redundant re-downloads when presets are renamed
or reorganised.
A mod "needs migration" when it exists on disk under
``downloads/{old_group}/@FolderName`` but comparison.json now assigns it
to ``downloads/{new_group}/@FolderName``.
Algorithm
---------
1. Build a *local index*: scan every ``downloads/{group}/@ModDir`` and read
its ``meta.cpp`` to find its steam_id. Also record its normalised folder
name. Result: ``{steam_id: (group, path)}`` and
``{norm_name: (group, path)}``.
2. Build a *target list* from comparison.json: for every mod in every group,
record which group comparison now assigns it to.
3. For each target entry:
- Find the mod in the local index (steam_id first, normalised name fallback).
- If not found on disk: skip (step_fetch will download it).
- If already in the correct group: skip.
- If the destination already exists: skip (step_fetch handles file-level sync).
- Otherwise: remove any stale junction first, then move old → new.
"""
from __future__ import annotations
import shutil
from pathlib import Path
from .fetcher import _normalize_name, _parse_meta_cpp
from .linker import _is_junction, remove_junction
def _build_local_index(downloads: Path) -> dict:
"""Scan ``downloads/`` and return a two-key index of existing mod folders.
:returns: ``{"by_steam_id": {sid: (group, path)},
"by_norm_name": {norm: (group, path)}}``
First match wins for both keys (sorted directory order is deterministic).
"""
by_steam_id: dict[str, tuple[str, Path]] = {}
by_norm_name: dict[str, tuple[str, Path]] = {}
if not downloads.is_dir():
return {"by_steam_id": by_steam_id, "by_norm_name": by_norm_name}
for group_dir in sorted(downloads.iterdir()):
if not group_dir.is_dir():
continue
group_name = group_dir.name
for mod_dir in sorted(group_dir.iterdir()):
if not mod_dir.is_dir() or not mod_dir.name.startswith("@"):
continue
norm = _normalize_name(mod_dir.name)
if norm not in by_norm_name:
by_norm_name[norm] = (group_name, mod_dir)
meta = mod_dir / "meta.cpp"
if meta.exists():
try:
sid = _parse_meta_cpp(
meta.read_text(encoding="utf-8", errors="replace")
)
if sid and sid not in by_steam_id:
by_steam_id[sid] = (group_name, mod_dir)
except OSError:
pass
return {"by_steam_id": by_steam_id, "by_norm_name": by_norm_name}
def _build_target_list(comparison: dict) -> list[tuple[str, str | None, str]]:
"""Flatten comparison.json into ``[(new_group, steam_id_or_None, mod_name)]``.
Shared mods → group ``"shared"``; unique mods → group = preset name.
"""
entries: list[tuple[str, str | None, str]] = []
for mod in comparison.get("shared", {}).get("mods", []):
entries.append(("shared", mod.get("steam_id") or None, mod.get("name", "")))
for preset, pdata in comparison.get("unique", {}).items():
for mod in pdata.get("mods", []):
entries.append((preset, mod.get("steam_id") or None, mod.get("name", "")))
return entries
def migrate_mod_groups(
downloads: Path,
arma_dir: Path | None,
comparison: dict,
) -> dict:
"""Move locally-downloaded mod folders to match *comparison* group assignments.
:param downloads: Path to the ``downloads/`` directory.
:param arma_dir: Path to the Arma 3 server directory used for junction
cleanup. Pass ``None`` to skip junction removal.
:param comparison: Parsed ``comparison.json`` dict.
:returns: Result dict::
{
"moved": int,
"junction_removed": int,
"skipped_correct": int,
"skipped_dest_exists": int,
"skipped_not_found": int,
"errors": {mod_name: error_message},
}
"""
result: dict = {
"moved": 0,
"junction_removed": 0,
"skipped_correct": 0,
"skipped_dest_exists": 0,
"skipped_not_found": 0,
"errors": {},
}
local = _build_local_index(downloads)
for new_group, steam_id, mod_name in _build_target_list(comparison):
# 1. Locate on disk — steam_id first, normalised name fallback
entry: tuple[str, Path] | None = None
if steam_id:
entry = local["by_steam_id"].get(steam_id)
if entry is None:
entry = local["by_norm_name"].get(_normalize_name(mod_name))
if entry is None:
result["skipped_not_found"] += 1
continue
old_group, old_path = entry
# 2. Already in the correct group?
if old_group == new_group:
result["skipped_correct"] += 1
continue
# 3. Destination already present — let step_fetch handle file-level sync
new_path = downloads / new_group / old_path.name
if new_path.exists():
result["skipped_dest_exists"] += 1
continue
# 4. Remove stale junction so step_link can re-create it at the new path
if arma_dir is not None and arma_dir.is_dir():
link = arma_dir / old_path.name
if _is_junction(link):
ok, err = remove_junction(link)
if ok:
result["junction_removed"] += 1
else:
print(f" MIGRATE WARNING: cannot remove junction "
f"{link.name}: {err}")
# 5. Move folder (shutil.move handles cross-device gracefully)
try:
new_path.parent.mkdir(parents=True, exist_ok=True)
shutil.move(str(old_path), str(new_path))
print(f" MIGRATE moved: {old_path.name} {old_group} -> {new_group}")
result["moved"] += 1
# Update index so later targets don't re-match the now-moved path
if steam_id:
local["by_steam_id"][steam_id] = (new_group, new_path)
local["by_norm_name"][_normalize_name(old_path.name)] = (
new_group, new_path
)
except OSError as exc:
print(f" MIGRATE ERROR: {old_path.name}: {exc}")
result["errors"][mod_name] = str(exc)
print(
f" Moved: {result['moved']} "
f"Junctions removed: {result['junction_removed']} "
f"Already correct: {result['skipped_correct']} "
f"Dest exists: {result['skipped_dest_exists']} "
f"Not on disk: {result['skipped_not_found']}"
)
return result

127
clean_orphans.py Normal file
View File

@@ -0,0 +1,127 @@
#!/usr/bin/env python3
"""
CLI entry point: find and remove orphaned mod folders from downloads/.
An orphan is a downloads/{group}/@ModName folder that is no longer referenced
by any group in comparison.json. These accumulate when presets change and
the pipeline is re-run without cleaning up old folders.
Usage:
python clean_orphans.py # list orphans, ask for confirmation
python clean_orphans.py --dry-run # list orphans, do not delete
python clean_orphans.py --yes # list and delete without prompting
"""
from __future__ import annotations
import argparse
import json
import os
import shutil
import sys
from arma_modlist_tools.cleaner import find_orphan_folders
from arma_modlist_tools.compat import fix_console_encoding
from arma_modlist_tools.config import load_config
from arma_modlist_tools.linker import _is_junction, remove_junction
fix_console_encoding()
_UNITS = ("B", "KB", "MB", "GB", "TB")
def _fmt_size(n: int) -> str:
for unit in _UNITS:
if n < 1024:
return f"{n:.1f} {unit}"
n /= 1024
return f"{n:.1f} PB"
def main() -> None:
parser = argparse.ArgumentParser(
description="Find and remove orphaned mod folders from downloads/."
)
parser.add_argument(
"--dry-run",
action="store_true",
help="List orphans but do not delete anything.",
)
parser.add_argument(
"--yes", "-y",
action="store_true",
help="Delete without prompting for confirmation.",
)
args = parser.parse_args()
cfg = load_config()
if not cfg.comparison.exists():
print(f"ERROR: {cfg.comparison} not found. Run compare_modlists.py first.")
sys.exit(1)
comparison = json.loads(cfg.comparison.read_text(encoding="utf-8"))
print(f"\nScanning {cfg.downloads} for orphaned mod folders...\n")
orphans = find_orphan_folders(cfg.downloads, comparison)
if not orphans:
print(" No orphans found. Your downloads folder is clean.")
print()
return
total_size = sum(o["size"] for o in orphans)
print(f" {'Group':<28} {'Folder':<32} Size")
print(f" {'-'*28} {'-'*32} {'-'*10}")
for o in orphans:
print(f" {o['group']:<28} {o['name']:<32} {_fmt_size(o['size'])}")
print()
print(f" {len(orphans)} orphan(s) found — {_fmt_size(total_size)} total")
print()
if args.dry_run:
print(" --dry-run: nothing deleted.")
print()
return
if not args.yes:
answer = input(" Delete all orphans? [y/N] ").strip().lower()
if answer not in ("y", "yes"):
print(" Aborted.")
print()
return
deleted = 0
freed = 0
errors = 0
for o in orphans:
p = o["path"]
try:
if _is_junction(p):
# Safety: never rmtree a junction — use remove_junction() which
# calls os.rmdir() and removes only the pointer, not the target.
ok, err = remove_junction(p)
if not ok:
print(f" ERROR: could not remove junction {p.name}: {err}")
errors += 1
continue
else:
shutil.rmtree(p)
deleted += 1
freed += o["size"]
print(f" Deleted: {o['group']}/{o['name']}")
except Exception as e:
print(f" ERROR: {p.name}: {e}")
errors += 1
print()
print(f" Done: {deleted} deleted, freed {_fmt_size(freed)}"
+ (f", {errors} error(s)" if errors else ""))
print()
if errors:
sys.exit(1)
if __name__ == "__main__":
main()

View File

@@ -13,6 +13,11 @@ Tài liệu này dành cho người dùng **chưa biết gì** về dự án. B
5. [Tổng quan (Dashboard) — Quy trình cơ bản](#5-tổng-quan-dashboard--quy-trình-cơ-bản)
6. [Danh sách Mod](#6-danh-sách-mod)
7. [Công cụ nâng cao](#7-công-cụ-nâng-cao)
- [Check Names](#tab-check-names--kiểm-tra-tên-thư-mục)
- [Update Mods](#tab-update-mods--cập-nhật-mod)
- [Link Mods](#tab-link-mods--quản-lý-liên-kết)
- [Sync / Report Missing](#tab-sync-missing--đồng-bộ-mod-thiếu)
- [Clean Orphans](#tab-clean-orphans--dọn-dẹp-mod-thừa)
8. [Nhật ký (Logs)](#8-nhật-ký-logs)
9. [Cài đặt](#9-cài-đặt)
10. [Đổi sang giao diện tiếng Việt](#10-đổi-sang-giao-diện-tiếng-việt)
@@ -32,7 +37,8 @@ Tài liệu này dành cho người dùng **chưa biết gì** về dự án. B
Ứng dụng sẽ tự động:
- Đọc danh sách mod từ các preset
- So sánh, tìm mod dùng chung và mod riêng giữa các preset
- Tải mod từ máy chủ về máy tính của bạn
- **Di chuyển** các thư mục mod đã tải về sang đúng nhóm mới (tránh tải lại khi đổi phiên bản preset)
- Tải mod từ máy chủ về máy tính của bạn (chỉ những mod thực sự chưa có)
- Tạo liên kết (junction/symlink) để Arma 3 Server nhận ra các mod
---
@@ -148,13 +154,14 @@ Giao diện gồm thanh điều hướng bên trái và khu vực nội dung bê
### 5.3 Trạng thái Pipeline
Cột bên phải hiển thị 4 bước của quy trình:
Cột bên phải hiển thị 5 bước của quy trình:
| Bước | Mô tả | Dấu hiệu hoàn thành |
|------|-------|-------------------|
| Phân tích preset | Đọc danh sách mod từ tệp HTML | Có ≥ 2 preset được chọn |
| So sánh preset | Tìm mod chung và riêng | Tệp `comparison.json` tồn tại |
| Tải mod | Tải tệp mod từ máy chủ | Có thư mục mod trong `downloads/` |
| Di chuyển nhóm mod | Chuyển thư mục mod sẵn có sang đúng nhóm mới | Tự động (không cần tải lại) |
| Tải mod | Tải tệp mod thực sự còn thiếu từ máy chủ | Có thư mục mod trong `downloads/` |
| Liên kết với Arma | Tạo junction tới thư mục Arma 3 | Thư mục Arma 3 Server tồn tại |
Biểu tượng `✓` (xanh) = đã xong, `○` (xám) = chưa xong.
@@ -212,6 +219,14 @@ Gõ vào ô **Tìm kiếm:** để lọc mod theo tên trong tab đang xem.
Trang **Công cụ** có 5 tab phụ cho các tác vụ bảo trì. Mỗi tab đều có nút chạy ở góc phải phía dưới, output hiển thị trong **Nhật ký**.
| Tab | Chức năng tóm tắt |
|-----|-------------------|
| Check Names | Kiểm tra và sửa tên thư mục mod |
| Update Mods | Tải lại tệp mod đã thay đổi trên máy chủ |
| Link Mods | Tạo / xóa junction tới Arma 3 Server |
| Sync / Report Missing | Đồng bộ và báo cáo mod còn thiếu |
| **Clean Orphans** | Xóa thư mục mod thừa từ preset cũ |
### Tab "Check Names" — Kiểm tra tên thư mục
Quét thư mục mod trên máy tính và so sánh với máy chủ. Báo cáo các vấn đề:
@@ -254,6 +269,22 @@ Thử tải lại các mod bị thiếu từ lần chạy pipeline trước. H
Kiểm tra mod nào trong `comparison.json` chưa có trên máy chủ và lưu báo cáo vào `missing_report.json`. Dùng để theo dõi mod cần yêu cầu admin bổ sung.
### Tab "Clean Orphans" — Dọn dẹp mod thừa
Khi bạn đổi preset và chạy lại pipeline, các mod của preset cũ vẫn còn trong thư mục `downloads/` nhưng không được dùng nữa — gọi là **mod thừa** (orphan). Tab này giúp tìm và xóa chúng để giải phóng dung lượng ổ đĩa.
**Cách dùng:**
1. Nhấn **Quét mod thừa** — ứng dụng sẽ so sánh thư mục `downloads/` với `comparison.json` hiện tại
2. Danh sách mod thừa hiện ra kèm tên nhóm và dung lượng
3. Dùng **Chọn tất cả** hoặc tick thủ công từng mục
4. Nhấn **Xóa đã chọn** — xuất hiện hộp thoại xác nhận
5. Nhấn **Xác nhận xóa** để thực hiện; danh sách sẽ tự động quét lại sau khi xóa
> **Lưu ý an toàn:** Ứng dụng chỉ xóa thư mục `@ModName` trong `downloads/`, không đụng tới thư mục Arma 3 Server. Junction (liên kết) sẽ bị xóa đúng cách mà không làm mất tệp gốc.
**Yêu cầu:** Cần có `comparison.json` (chạy pipeline ít nhất một lần trước).
---
## 8. Nhật ký (Logs)
@@ -362,7 +393,7 @@ Hiển thị các đường dẫn đang dùng: URL máy chủ, thư mục Arma,
| Thuật ngữ | Giải thích |
|-----------|------------|
| **Preset** | Tệp HTML xuất từ Arma 3 Launcher chứa danh sách mod |
| **Pipeline** | Chuỗi 4 bước tự động: phân tích → so sánh → tải → liên kết |
| **Pipeline** | Chuỗi 5 bước tự động: phân tích → so sánh → di chuyển nhóm → tải → liên kết |
| **Junction / Symlink** | Liên kết thư mục ảo — Arma 3 thấy mod trong thư mục của mình nhưng tệp thực sự nằm ở `downloads/` |
| **Shared mods** | Mod xuất hiện trong tất cả preset đã chọn |
| **Unique mods** | Mod chỉ có trong một preset cụ thể |
@@ -372,8 +403,10 @@ Hiển thị các đường dẫn đang dùng: URL máy chủ, thư mục Arma,
| **comparison.json** | Tệp kết quả so sánh preset, lưu danh sách mod theo nhóm |
| **missing_report.json** | Báo cáo mod có trong preset nhưng chưa có trên máy chủ |
| **downloads/** | Thư mục chứa tệp mod đã tải về |
| **Di chuyển nhóm mod** | Bước tự động chuyển thư mục mod từ nhóm cũ sang nhóm mới theo `comparison.json` — tránh tải lại khi đổi phiên bản preset |
| **Mod thừa (Orphan)** | Thư mục mod còn trong `downloads/` nhưng không còn trong preset nào đang dùng |
| **config.json** | Tệp cấu hình lưu thông tin máy chủ và đường dẫn |
---
*Phiên bản tài liệu: 2026-04. Nếu có vấn đề, liên hệ người quản trị máy chủ.*
*Phiên bản tài liệu: 2026-04 (cập nhật: thêm bước Di chuyển nhóm mod, pipeline 5 bước). Nếu có vấn đề, liên hệ người quản trị máy chủ.*

View File

@@ -143,18 +143,29 @@ class ArmaModManagerApp(ctk.CTk):
self._pipeline_running = True
self._get_dashboard().set_pipeline_ui(running=True)
self.navigate_to("Logs")
# Post an immediate banner so the log is never blank after clicking Start.
_sep = "=" * 50
self.post_log(f"\n{_sep}\n {t('pipeline.starting')}\n{_sep}\n\n")
def worker() -> None:
# run.py calls fix_console_encoding() at import time, which needs
# the real sys.stdout.buffer. Import it before we redirect stdout.
from run import step_fetch, step_link
try:
from run import step_migrate, step_fetch, step_link
except Exception as _import_err:
self.after(0, lambda: self.post_log(
f"\n✗ Failed to load pipeline: {_import_err}\n"
))
self.after(0, self._pipeline_done)
return
self._redirect_output()
try:
from arma_modlist_tools.parser import parse_modlist_html
from arma_modlist_tools.compare import compare_presets
# Step 1 — Parse selected presets
_hdr("Step 1 / 4", t("pipeline.step1_name"))
_hdr("Step 1 / 5", t("pipeline.step1_name"))
cfg.modlist_json.mkdir(exist_ok=True)
presets = []
for fp in sorted(cfg.modlist_html.glob("*.html")):
@@ -171,7 +182,7 @@ class ArmaModManagerApp(ctk.CTk):
presets.append(preset)
# Step 2 — Compare
_hdr("Step 2 / 4", t("pipeline.step2_name"))
_hdr("Step 2 / 5", t("pipeline.step2_name"))
result = compare_presets(*presets)
cfg.comparison.write_text(
json.dumps(result, indent=2, ensure_ascii=False),
@@ -182,12 +193,16 @@ class ArmaModManagerApp(ctk.CTk):
print(f" Shared: {result['shared']['mod_count']} | "
f"Unique: {total_unique}")
# Step 3 — Fetch
_hdr("Step 3 / 4", t("pipeline.step3_name"))
# Step 3 — Migrate
_hdr("Step 3 / 5", t("pipeline.step3_name"))
step_migrate(cfg)
# Step 4 — Fetch
_hdr("Step 4 / 5", t("pipeline.step4_name"))
step_fetch(cfg)
# Step 4 — Link
_hdr("Step 4 / 4", t("pipeline.step4_name"))
# Step 5 — Link
_hdr("Step 5 / 5", t("pipeline.step5_name"))
groups = (
sorted(p.name for p in cfg.downloads.iterdir() if p.is_dir())
if cfg.downloads.is_dir() else []
@@ -216,11 +231,14 @@ class ArmaModManagerApp(ctk.CTk):
try:
env = os.environ.copy()
env["PYTHONUNBUFFERED"] = "1"
env["PYTHONUTF8"] = "1"
proc = subprocess.Popen(
[sys.executable, "-u", str(PROJECT_ROOT / script)] + extra,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT,
text=True,
encoding="utf-8",
errors="replace",
cwd=str(PROJECT_ROOT),
env=env,
)

View File

@@ -26,10 +26,12 @@ _EN: dict[str, str] = {
"nav.settings": "Settings",
# ── Pipeline step headers (printed to log) ───────────────────────────────
"pipeline.starting": "Pipeline started",
"pipeline.step1_name": "Parse presets",
"pipeline.step2_name": "Compare presets",
"pipeline.step3_name": "Download mods",
"pipeline.step4_name": "Link mods",
"pipeline.step3_name": "Migrate mod groups",
"pipeline.step4_name": "Download mods",
"pipeline.step5_name": "Link mods",
# ── app.py dialogs ────────────────────────────────────────────────────────
"app.dlg_presets_title": "Not enough presets selected",
@@ -224,6 +226,36 @@ _EN: dict[str, str] = {
"tools.rm_btn": "Generate Report",
"tools.rm_last": "Last generated: {ts}",
"tools.rm_none": "No report yet.",
# ── Tools — Clean Orphans ────────────────────────────────────────────────
"tools.oc_desc": (
"Scan the downloads folder for mod folders that are no longer "
"referenced in comparison.json. These orphans accumulate when you "
"remove mods from your presets and re-run the pipeline. "
"Select the ones you want to remove to free up disk space."
),
"tools.oc_warn": (
"⚠ Deleting orphans permanently removes mod files from disk. "
"This cannot be undone."
),
"tools.oc_scan_btn": "Scan for Orphans",
"tools.oc_scanning": "Scanning…",
"tools.oc_no_config": "No config found. Complete Setup first.",
"tools.oc_no_comparison": "No comparison.json found — run the pipeline first.",
"tools.oc_none_found": "No orphans found. Your downloads folder is clean.",
"tools.oc_found": "{count} orphan(s) found — {size} total",
"tools.oc_sel_all": "Select All",
"tools.oc_sel_none": "Deselect All",
"tools.oc_delete_btn": "Delete Selected",
"tools.oc_confirm_title": "Confirm Delete",
"tools.oc_confirm_body": (
"Permanently delete {count} orphan folder(s) ({size})?\n\n"
"This cannot be undone."
),
"tools.oc_done": "Deleted {count} folder(s), freed {size}.",
"tools.oc_error": "Error deleting {path}: {e}",
"tools.oc_error_title": "Delete errors",
"tools.oc_scan_error": "Scan error: {e}",
}
_VI: dict[str, str] = {
@@ -236,10 +268,12 @@ _VI: dict[str, str] = {
"nav.settings": "Cài đặt",
# ── Pipeline step headers ────────────────────────────────────────────────
"pipeline.starting": "Pipeline đã bắt đầu",
"pipeline.step1_name": "Phân tích preset",
"pipeline.step2_name": "So sánh preset",
"pipeline.step3_name": "Tải mod",
"pipeline.step4_name": "Liên kết mod",
"pipeline.step3_name": "Di chuyển nhóm mod",
"pipeline.step4_name": "Tải mod",
"pipeline.step5_name": "Liên kết mod",
# ── app.py dialogs ────────────────────────────────────────────────────────
"app.dlg_presets_title": "Chưa chọn đủ preset",
@@ -431,6 +465,36 @@ _VI: dict[str, str] = {
"tools.rm_btn": "Tạo báo cáo",
"tools.rm_last": "Tạo lần cuối: {ts}",
"tools.rm_none": "Chưa có báo cáo.",
# ── Tools — Clean Orphans ────────────────────────────────────────────────
"tools.oc_desc": (
"Quét thư mục downloads để tìm các thư mục mod không còn được "
"tham chiếu trong comparison.json. Các mod mồ côi này tích tụ khi "
"bạn xóa mod khỏi preset và chạy lại pipeline. "
"Chọn các thư mục muốn xóa để giải phóng dung lượng."
),
"tools.oc_warn": (
"⚠ Xóa mod mồ côi sẽ xóa vĩnh viễn tệp mod khỏi ổ đĩa. "
"Thao tác này không thể hoàn tác."
),
"tools.oc_scan_btn": "Quét mod mồ côi",
"tools.oc_scanning": "Đang quét…",
"tools.oc_no_config": "Chưa tìm thấy cấu hình. Vui lòng hoàn thành thiết lập.",
"tools.oc_no_comparison": "Chưa có comparison.json — hãy chạy pipeline trước.",
"tools.oc_none_found": "Không tìm thấy mod mồ côi. Thư mục downloads sạch.",
"tools.oc_found": "Tìm thấy {count} mod mồ côi — tổng {size}",
"tools.oc_sel_all": "Chọn tất cả",
"tools.oc_sel_none": "Bỏ chọn",
"tools.oc_delete_btn": "Xóa đã chọn",
"tools.oc_confirm_title": "Xác nhận xóa",
"tools.oc_confirm_body": (
"Xóa vĩnh viễn {count} thư mục mồ côi ({size})?\n\n"
"Thao tác này không thể hoàn tác."
),
"tools.oc_done": "Đã xóa {count} thư mục, giải phóng {size}.",
"tools.oc_error": "Lỗi khi xóa {path}: {e}",
"tools.oc_error_title": "Lỗi xóa",
"tools.oc_scan_error": "Lỗi quét: {e}",
}
# Guard: both dicts must have identical key sets

View File

@@ -7,7 +7,7 @@ from typing import TYPE_CHECKING, Optional
import customtkinter as ctk
from arma_modlist_tools.fetcher import _normalize_name
from arma_modlist_tools.fetcher import _normalize_name, _parse_meta_cpp
from gui._constants import COLOR_OK, COLOR_ERROR, COLOR_WARN, COLOR_RUNNING
from gui.locales import t
from gui.views.base import BaseView
@@ -16,13 +16,14 @@ if TYPE_CHECKING:
from gui.app import ArmaModManagerApp
def _find_folder(group_dir: Path, mod_name: str) -> Optional[Path]:
def _find_folder(group_dir: Path, mod_name: str, steam_id: str | None = None) -> Optional[Path]:
"""Return the local mod folder path, or None if not downloaded.
Matches in priority order:
1. Exact folder name ``@{mod_name}``
2. Case-insensitive name (handles ``@CBA_A3`` vs ``CBA_A3``)
3. Normalized name — strips non-alphanumeric (handles ``@cba_a3`` vs ``CBA A3``)
4. Steam ID match via ``meta.cpp`` ``publishedid`` field (folder name differs from modlist name)
"""
if not group_dir.is_dir():
return None
@@ -38,6 +39,18 @@ def _find_folder(group_dir: Path, mod_name: str) -> Optional[Path]:
return p
if _normalize_name(p.name) == target_norm:
return p
# Fallback: match by steam_id via meta.cpp when folder name diverges from modlist name
if steam_id:
for p in group_dir.iterdir():
if not p.is_dir():
continue
meta = p / "meta.cpp"
try:
sid = _parse_meta_cpp(meta.read_text(encoding="utf-8", errors="replace"))
if sid == steam_id:
return p
except OSError:
pass
return None
@@ -171,19 +184,22 @@ class ModsView(BaseView):
fg_color=("gray82", "gray22"), corner_radius=6)
col_hdr.grid(row=0, column=0, sticky="ew", padx=4, pady=(6, 2))
col_hdr.columnconfigure(0, weight=1)
for col, (w, lbl_key) in enumerate([
(0, "mods.col_name"),
(80, "mods.col_downloaded"),
(80, "mods.col_linked"),
(160, "mods.col_server"),
(80, ""),
for col, (w, lbl_key, anc) in enumerate([
(0, "mods.col_name", "w"),
(80, "mods.col_downloaded", "center"),
(80, "mods.col_linked", "center"),
(160, "mods.col_server", "w"),
(80, "", "center"),
]):
ctk.CTkLabel(col_hdr, text=t(lbl_key) if lbl_key else "",
font=ctk.CTkFont(weight="bold"),
anchor="w", width=w or 1).grid(
anchor=anc, width=w or 1).grid(
row=0, column=col,
padx=(10 if col == 0 else 4, 4), pady=5,
padx=(8 if col == 0 else 4, 4), pady=5,
sticky="ew" if col == 0 else "")
# Spacer compensates for CTkScrollableFrame's internal scrollbar width
# so the header columns line up with the data rows below.
ctk.CTkLabel(col_hdr, text="", width=16).grid(row=0, column=5, padx=0)
# Scrollable rows
scroll = ctk.CTkScrollableFrame(tab_frame)
@@ -215,7 +231,7 @@ class ModsView(BaseView):
link_map: dict[str, bool],
) -> None:
for i, mod in enumerate(sorted(mods, key=lambda m: m["name"].lower())):
folder_path = _find_folder(cfg.downloads / group, mod["name"])
folder_path = _find_folder(cfg.downloads / group, mod["name"], mod.get("steam_id"))
downloaded = folder_path is not None
linked = (link_map.get(folder_path.name.lower(), False)
if folder_path else False)
@@ -226,23 +242,23 @@ class ModsView(BaseView):
row.columnconfigure(0, weight=1)
# Mod name
name_lbl = ctk.CTkLabel(row, text=f" {mod['name']}", anchor="w")
name_lbl.grid(row=0, column=0, sticky="ew", padx=4, pady=3)
name_lbl = ctk.CTkLabel(row, text=mod["name"], anchor="w")
name_lbl.grid(row=0, column=0, sticky="ew", padx=(8, 4), pady=3)
# Downloaded
ctk.CTkLabel(
row,
text=" " if downloaded else " ",
text="" if downloaded else "",
text_color=COLOR_OK if downloaded else COLOR_ERROR,
width=80, anchor="w",
width=80, anchor="center",
).grid(row=0, column=1, padx=4)
# Linked
ctk.CTkLabel(
row,
text=" " if linked else (" " if not downloaded else " "),
text="" if linked else ("" if not downloaded else ""),
text_color=COLOR_OK if linked else "gray",
width=80, anchor="w",
width=80, anchor="center",
).grid(row=0, column=2, padx=4)
# Server status
@@ -254,7 +270,7 @@ class ModsView(BaseView):
# Update button (hidden until stale detected)
folder_name = folder_path.name if folder_path else None
update_btn = ctk.CTkButton(
row, text=t("mods.update_btn"), width=70,
row, text=t("mods.update_btn"), width=80,
command=(lambda g=group, fn=folder_name:
self._update_mod(g, fn)) if folder_name else None,
state="normal" if folder_name else "disabled",

View File

@@ -1,11 +1,15 @@
from __future__ import annotations
import json
import shutil
import threading
from tkinter import messagebox
from typing import TYPE_CHECKING, Optional
from typing import TYPE_CHECKING
import customtkinter as ctk
from arma_modlist_tools.cleaner import find_orphan_folders
from arma_modlist_tools.linker import _is_junction, remove_junction
from gui._constants import COLOR_WARN, PROJECT_ROOT
from gui.locales import t
from gui.views.base import BaseView
@@ -41,6 +45,7 @@ class ToolsView(BaseView):
self._build_link_mods_tab()
self._build_sync_missing_tab()
self._build_report_missing_tab()
self._build_clean_orphans_tab()
# =========================================================================
# Public
@@ -378,6 +383,208 @@ class ToolsView(BaseView):
pass
self._rm_info.configure(text=t("tools.rm_none"))
# -------------------------------------------------------------------------
def _build_clean_orphans_tab(self) -> None:
self._tab_view.add("Clean Orphans")
tab = self._tab_view.tab("Clean Orphans")
tab.grid_columnconfigure(0, weight=1)
tab.grid_rowconfigure(3, weight=1)
desc_lbl = _desc(tab, row=0, text=t("tools.oc_desc"))
self._translatable.append((desc_lbl, "tools.oc_desc"))
oc_warn = ctk.CTkLabel(tab, text=t("tools.oc_warn"),
text_color=_WARN_COLOR, anchor="w")
oc_warn.grid(row=1, column=0, padx=24, pady=(0, 4), sticky="w")
self._translatable.append((oc_warn, "tools.oc_warn"))
self._oc_status = ctk.CTkLabel(tab, text="", text_color="gray", anchor="w")
self._oc_status.grid(row=2, column=0, padx=24, pady=(0, 2), sticky="w")
# Scrollable list for results
self._oc_scroll = ctk.CTkScrollableFrame(tab)
self._oc_scroll.grid(row=3, column=0, sticky="nsew", padx=16, pady=(0, 4))
self._oc_scroll.grid_columnconfigure(0, weight=1)
# Bottom action bar
bot = ctk.CTkFrame(tab, fg_color="transparent")
bot.grid(row=4, column=0, sticky="ew", padx=16, pady=(4, 12))
self._oc_sel_all_btn = ctk.CTkButton(
bot, text=t("tools.oc_sel_all"), width=110,
command=self._oc_select_all,
)
self._oc_sel_all_btn.pack(side="left", padx=(0, 4))
self._translatable.append((self._oc_sel_all_btn, "tools.oc_sel_all"))
self._oc_sel_none_btn = ctk.CTkButton(
bot, text=t("tools.oc_sel_none"), width=110,
command=self._oc_deselect_all,
)
self._oc_sel_none_btn.pack(side="left", padx=4)
self._translatable.append((self._oc_sel_none_btn, "tools.oc_sel_none"))
self._oc_scan_btn = ctk.CTkButton(
bot, text=t("tools.oc_scan_btn"), width=150,
command=self._oc_scan,
)
self._oc_scan_btn.pack(side="right", padx=(4, 0))
self._translatable.append((self._oc_scan_btn, "tools.oc_scan_btn"))
self._oc_delete_btn = ctk.CTkButton(
bot, text=t("tools.oc_delete_btn"), width=150,
fg_color="darkred", hover_color="#8b0000",
command=self._oc_delete_selected,
state="disabled",
)
self._oc_delete_btn.pack(side="right", padx=4)
self._translatable.append((self._oc_delete_btn, "tools.oc_delete_btn"))
# Internal scan state
self._oc_orphans: list[dict] = []
self._oc_check_vars: list[ctk.BooleanVar] = []
self._oc_pending_done_msg: str | None = None
def _oc_scan(self) -> None:
cfg = self.app.cfg
if not cfg:
self._oc_status.configure(text=t("tools.oc_no_config"), text_color="gray")
return
if not cfg.comparison.exists():
self._oc_status.configure(text=t("tools.oc_no_comparison"), text_color="gray")
return
self._oc_scan_btn.configure(state="disabled", text=t("tools.oc_scanning"))
self._oc_delete_btn.configure(state="disabled")
self._oc_status.configure(text=t("tools.oc_scanning"), text_color="gray")
def _run() -> None:
try:
comparison = json.loads(cfg.comparison.read_text(encoding="utf-8"))
orphans = find_orphan_folders(cfg.downloads, comparison)
except Exception as e:
self.after(0, lambda: self._oc_scan_done(None, str(e)))
return
self.after(0, lambda: self._oc_scan_done(orphans, None))
threading.Thread(target=_run, daemon=True).start()
def _oc_scan_done(self, orphans: list[dict] | None, error: str | None) -> None:
self._oc_scan_btn.configure(state="normal", text=t("tools.oc_scan_btn"))
# Consume any pending success message from a previous delete operation
done_msg = self._oc_pending_done_msg
self._oc_pending_done_msg = None
# Clear previous results
for w in self._oc_scroll.winfo_children():
w.destroy()
self._oc_orphans = []
self._oc_check_vars = []
if error:
self._oc_status.configure(text=t("tools.oc_scan_error", e=error), text_color="red")
return
if not orphans:
msg = done_msg or t("tools.oc_none_found")
self._oc_status.configure(text=msg, text_color="gray")
return
total_size = sum(o["size"] for o in orphans)
self._oc_status.configure(
text=t("tools.oc_found", count=len(orphans), size=_fmt_size(total_size)),
text_color="gray",
)
self._oc_orphans = orphans
self._oc_delete_btn.configure(state="normal")
for i, orphan in enumerate(orphans):
var = ctk.BooleanVar(value=True)
self._oc_check_vars.append(var)
bg = ("gray90", "gray17") if i % 2 == 0 else ("gray86", "gray14")
row = ctk.CTkFrame(self._oc_scroll, fg_color=bg, corner_radius=4)
row.pack(fill="x", pady=1)
row.columnconfigure(1, weight=1)
ctk.CTkCheckBox(row, text="", variable=var, width=24).grid(
row=0, column=0, padx=(8, 4), pady=4,
)
ctk.CTkLabel(
row,
text=f" {orphan['group']} / {orphan['name']}",
anchor="w",
).grid(row=0, column=1, sticky="ew", padx=4)
ctk.CTkLabel(
row,
text=_fmt_size(orphan["size"]),
text_color="gray",
width=80,
anchor="e",
).grid(row=0, column=2, padx=(4, 12))
def _oc_select_all(self) -> None:
for var in self._oc_check_vars:
var.set(True)
def _oc_deselect_all(self) -> None:
for var in self._oc_check_vars:
var.set(False)
def _oc_delete_selected(self) -> None:
selected = [
self._oc_orphans[i]
for i, var in enumerate(self._oc_check_vars)
if var.get()
]
if not selected:
return
total_size = sum(o["size"] for o in selected)
confirmed = messagebox.askyesno(
t("tools.oc_confirm_title"),
t("tools.oc_confirm_body", count=len(selected), size=_fmt_size(total_size)),
)
if not confirmed:
return
self._oc_delete_btn.configure(state="disabled")
self._oc_scan_btn.configure(state="disabled")
def _run() -> None:
freed = 0
errors = []
for orphan in selected:
try:
p = orphan["path"]
if _is_junction(p):
# Safety: never rmtree a junction — it follows the
# reparse point and deletes the target's contents.
# Use remove_junction() which calls os.rmdir() instead.
ok, err = remove_junction(p)
if not ok:
errors.append(t("tools.oc_error", path=p.name, e=err))
continue
else:
shutil.rmtree(p)
freed += orphan["size"]
except Exception as e:
errors.append(t("tools.oc_error", path=orphan["path"].name, e=e))
self.after(0, lambda: self._oc_delete_done(len(selected), freed, errors))
threading.Thread(target=_run, daemon=True).start()
def _oc_delete_done(self, count: int, freed: int, errors: list[str]) -> None:
# Store success message so _oc_scan_done() can display it after the rescan
self._oc_pending_done_msg = (
None if errors
else t("tools.oc_done", count=count, size=_fmt_size(freed))
)
self._oc_scan_btn.configure(state="normal")
self._oc_scan()
if errors:
messagebox.showerror(t("tools.oc_error_title"), "\n".join(errors))
# =========================================================================
# Private — helpers
# =========================================================================
@@ -400,6 +607,21 @@ class ToolsView(BaseView):
self.app.run_tool(args)
# ---------------------------------------------------------------------------
# Size formatting helper
# ---------------------------------------------------------------------------
def _fmt_size(n: int) -> str:
"""Human-readable file size string."""
if n < 1024:
return f"{n} B"
if n < 1024 ** 2:
return f"{n / 1024:.1f} KB"
if n < 1024 ** 3:
return f"{n / 1024 ** 2:.1f} MB"
return f"{n / 1024 ** 3:.2f} GB"
# ---------------------------------------------------------------------------
# Layout helpers
# ---------------------------------------------------------------------------

33
run.py
View File

@@ -78,6 +78,22 @@ def step_compare(cfg) -> None:
print(f" -> {cfg.comparison}")
def step_migrate(cfg) -> None:
if not cfg.comparison.exists():
print(f" NOTE: {cfg.comparison} not found — skipping migration.")
return
if not cfg.downloads.is_dir():
print(f" NOTE: downloads dir missing ({cfg.downloads}) — nothing to migrate.")
return
from arma_modlist_tools.migrator import migrate_mod_groups
comparison = json.loads(cfg.comparison.read_text(encoding="utf-8"))
arma_dir = cfg.arma_dir if cfg.arma_dir.is_dir() else None
result = migrate_mod_groups(cfg.downloads, arma_dir, comparison)
if result["errors"]:
for name, err in result["errors"].items():
print(f" ERROR {name}: {err}")
def step_fetch(cfg) -> None:
if not cfg.comparison.exists():
print(f" ERROR: {cfg.comparison} not found. Run parse + compare first.")
@@ -91,9 +107,14 @@ def step_fetch(cfg) -> None:
for mod in data["mods"]:
queue.append((mod, preset_name))
def _index_progress(current: int, total: int, name: str) -> None:
if current == 1 or current % 25 == 0 or current == total:
print(f" Indexing {current}/{total}: {name}")
print(f" Building server index...")
index = build_server_index(cfg.server_url, cfg.server_auth)
print(f" Indexed {len(index['by_steam_id'])} mods\n")
index = build_server_index(cfg.server_url, cfg.server_auth, progress_fn=_index_progress)
print(f" Indexed {len(index['by_steam_id'])} mods by steam_id, "
f"{len(index['by_name'])} by name\n")
session = make_session(cfg.server_auth)
resolved = []
@@ -173,6 +194,7 @@ def main() -> None:
parser = argparse.ArgumentParser(description="Run the full mod management pipeline.")
parser.add_argument("--skip-parse", action="store_true")
parser.add_argument("--skip-compare", action="store_true")
parser.add_argument("--skip-migrate", action="store_true")
parser.add_argument("--skip-fetch", action="store_true")
parser.add_argument("--skip-link", action="store_true")
parser.add_argument("--group", "-g", metavar="GROUP",
@@ -182,13 +204,15 @@ def main() -> None:
steps = [
(not args.skip_parse, "Parse presets"),
(not args.skip_compare, "Compare presets"),
(not args.skip_migrate, "Migrate mods"),
(not args.skip_fetch, "Fetch mods"),
(not args.skip_link, "Link mods"),
]
active_count = sum(1 for run, _ in steps if run)
step_num = 0
if args.skip_parse and args.skip_compare and args.skip_fetch and args.skip_link:
if (args.skip_parse and args.skip_compare and args.skip_migrate
and args.skip_fetch and args.skip_link):
print("All steps skipped — nothing to do.")
sys.exit(0)
@@ -196,6 +220,7 @@ def main() -> None:
(cfg,),
(cfg,),
(cfg,),
(cfg,),
None, # handled separately
]):
if not run:
@@ -216,6 +241,8 @@ def main() -> None:
step_parse(cfg)
elif name == "Compare presets":
step_compare(cfg)
elif name == "Migrate mods":
step_migrate(cfg)
elif name == "Fetch mods":
step_fetch(cfg)

File diff suppressed because it is too large Load Diff

View File

@@ -102,6 +102,7 @@ def main() -> None:
COL_GROUP = 24
total_checked = total_updated = total_bytes = 0
total_removed = 0
not_on_server = []
for group, folder_name, mod_dir in targets:
@@ -123,13 +124,21 @@ def main() -> None:
all_files = list_mod_files(folder_url, session) if not args.force else stale
checked = len(all_files) if not args.force else len(stale)
if not stale:
# Find local files that no longer exist on the server (orphans)
server_rel = {rel for rel, _, _ in all_files}
orphans = [
f for f in mod_dir.rglob("*") if f.is_file()
and str(f.relative_to(mod_dir)).replace("\\", "/") not in server_rel
]
if not stale and not orphans:
print(f" [=] {folder_name:<{COL_MOD}} {group:<{COL_GROUP}} {checked} files up-to-date")
total_checked += checked
continue
# Download stale files
mod_bytes = 0
if stale:
with tqdm(
total=len(stale), unit="file",
desc=f" {folder_name[-COL_MOD:]:<{COL_MOD}}",
@@ -148,21 +157,32 @@ def main() -> None:
mod_bytes += n
file_bar.update(1)
# Remove orphan files
for orphan in orphans:
tqdm.write(f" [-] orphan removed: {orphan.relative_to(mod_dir)}")
orphan.unlink()
total_checked += checked
total_updated += len(stale)
total_bytes += mod_bytes
print(f" [+] {folder_name:<{COL_MOD}} {group:<{COL_GROUP}} "
f"{checked} files {len(stale)} updated ({_fmt_bytes(mod_bytes)})")
total_removed += len(orphans)
parts = [f"{checked} files"]
if stale:
parts.append(f"{len(stale)} updated ({_fmt_bytes(mod_bytes)})")
if orphans:
parts.append(f"{len(orphans)} orphan(s) removed")
print(f" [+] {folder_name:<{COL_MOD}} {group:<{COL_GROUP}} {' '.join(parts)}")
print(f"\n{'='*56}")
print(f" Total: {total_checked} files checked, "
f"{total_updated} updated, "
f"{total_removed} orphan(s) removed, "
f"{_fmt_bytes(total_bytes)} downloaded")
if not_on_server:
print(f" Not found on server ({len(not_on_server)}): {', '.join(not_on_server)}")
print(f"{'='*56}\n")
if total_updated == 0 and not not_on_server:
if total_updated == 0 and total_removed == 0 and not not_on_server:
print(" All mods are up-to-date.\n")