# Languard Servers Manager — Database Design ## Engine - **SQLite** via `SQLAlchemy Core` (sync for all access — routes and threads) - File: `languard.db` at project root (configurable via `LANGUARD_DB_PATH`) - WAL mode enabled: `PRAGMA journal_mode=WAL` — allows concurrent reads during writes - Foreign keys enabled: `PRAGMA foreign_keys=ON` - Busy timeout: `PRAGMA busy_timeout=5000` — prevents "database is locked" errors under concurrent thread writes - **Retry on exhaustion**: If busy_timeout is exceeded (5s), writes fail with `OperationalError("database is locked")`. Background threads retry with exponential backoff (1s, 2s, 4s), then skip the tick. API handlers retry up to 2 times with 1s backoff, then return 503. See THREADING.md for the full retry implementation. --- ## Design Philosophy The database uses a **hybrid approach**: core tables are fully normalized (game-agnostic), while game-specific config is stored as JSON blobs in a generic `game_configs` table, validated by adapter Pydantic models at the application layer. **Why this works for Languard:** - Config is always read/written as a whole section (nobody queries "find all servers where von_codec_quality > 20") - Each adapter section maps to a Pydantic model, so validation is enforced at the application layer - The JSON is opaque to the core, meaningful only to the adapter that owns it - Adding a new game requires **zero DB migration** — just a new adapter - Core queries across all games work naturally --- ## Schema ### Table: `users` Stores web UI admin accounts. ```sql CREATE TABLE users ( id INTEGER PRIMARY KEY AUTOINCREMENT, username TEXT NOT NULL UNIQUE, password_hash TEXT NOT NULL, -- bcrypt hash role TEXT NOT NULL DEFAULT 'viewer', -- 'admin' | 'viewer' CHECK (role IN ('admin', 'viewer')), created_at TEXT NOT NULL DEFAULT (datetime('now')), last_login TEXT ); ``` --- ### Table: `servers` One row per managed server instance. **Game-agnostic.** ```sql CREATE TABLE servers ( id INTEGER PRIMARY KEY AUTOINCREMENT, name TEXT NOT NULL, -- display name in UI description TEXT, game_type TEXT NOT NULL DEFAULT 'arma3', -- adapter lookup key status TEXT NOT NULL DEFAULT 'stopped', CHECK (status IN ('stopped', 'starting', 'running', 'stopping', 'crashed', 'error')), -- Process info pid INTEGER, -- OS process ID when running exe_path TEXT NOT NULL, -- path to server executable started_at TEXT, -- ISO datetime stopped_at TEXT, -- Network (core ports; adapter defines derived port conventions) game_port INTEGER NOT NULL, rcon_port INTEGER, -- NULL if game has no remote admin CHECK (game_port BETWEEN 1024 AND 65535), CHECK (rcon_port IS NULL OR (rcon_port BETWEEN 1024 AND 65535)), -- Auto-management auto_restart INTEGER NOT NULL DEFAULT 0, -- 1 = restart on crash max_restarts INTEGER NOT NULL DEFAULT 3, -- within restart_window_seconds restart_window_seconds INTEGER NOT NULL DEFAULT 300, restart_count INTEGER NOT NULL DEFAULT 0, last_restart_at TEXT, created_at TEXT NOT NULL DEFAULT (datetime('now')), updated_at TEXT NOT NULL DEFAULT (datetime('now')) ); CREATE INDEX idx_servers_status ON servers(status); CREATE INDEX idx_servers_game_type ON servers(game_type); CREATE INDEX idx_servers_game_port ON servers(game_port); ``` **Key changes from single-game design:** - Added `game_type` column (defaults to `'arma3'` for backward compatibility) - Removed Arma 3-specific defaults (`DEFAULT 2302`, `DEFAULT 2306`) — port defaults are now adapter-provided - Removed `steam_query_port` virtual column (Arma 3 convention moved to adapter) - `rcon_port` is now nullable (some games have no remote admin) --- ### Table: `game_configs` Stores all game-specific configuration as JSON blobs, keyed by section. **Replaces** the previous `server_configs`, `basic_configs`, `server_profiles`, `launch_params`, and `rcon_configs` tables. ```sql CREATE TABLE game_configs ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, game_type TEXT NOT NULL, -- for validation; matches servers.game_type section TEXT NOT NULL, -- e.g. 'server', 'basic', 'profile', 'launch', 'rcon' config_json TEXT NOT NULL DEFAULT '{}', -- JSON validated by adapter's Pydantic model config_version INTEGER NOT NULL DEFAULT 1, -- optimistic locking version; incremented on each write schema_version TEXT NOT NULL DEFAULT '1.0', -- adapter schema version at time of last write updated_at TEXT NOT NULL DEFAULT (datetime('now')), UNIQUE(server_id, section) ); CREATE INDEX idx_game_configs_server ON game_configs(server_id); CREATE INDEX idx_game_configs_type_section ON game_configs(game_type, section); ``` **How it works:** - Each adapter defines config sections via `ConfigGenerator.get_sections()` → `{section_name: PydanticModelClass}` - Core reads/writes JSON blobs; adapter validates them on write and parses them on read - Arma 3 has 5 sections: `server`, `basic`, `profile`, `launch`, `rcon` - Another game might have 2 sections or 8 — no migration needed - Sensitive fields within JSON (passwords, rcon_password) are encrypted at the application layer via Fernet before storage **Arma 3 config sections and their JSON structures:** Section `server` — maps to `server.cfg` parameters: ```json { "hostname": "My Arma 3 Server", "password": "encrypted:...", "password_admin": "encrypted:...", "server_command_password": "encrypted:...", "max_players": 40, "kick_duplicate": 1, "persistent": 1, "vote_threshold": 0.33, "vote_mission_players": 1, "vote_timeout": 60, "role_timeout": 90, "briefing_timeout": 60, "debriefing_timeout": 45, "lobby_idle_timeout": 300, "disable_von": 0, "von_codec": 1, "von_codec_quality": 20, "max_ping": 250, "max_packet_loss": 50, "max_desync": 200, "disconnect_timeout": 15, "kick_on_ping": 1, "kick_on_packet_loss": 1, "kick_on_desync": 1, "kick_on_timeout": 1, "battleye": 1, "verify_signatures": 2, "allowed_file_patching": 0, "forced_difficulty": "Regular", "timestamp_format": "short", "auto_select_mission": 0, "random_mission_order": 0, "missions_to_restart": 0, "missions_to_shutdown": 0, "log_file": "server_console.log", "skip_lobby": 0, "drawing_in_map": 1, "upnp": 0, "loopback": 0, "statistics_enabled": 1, "force_rotor_lib": 0, "required_build": 0, "steam_protocol_max_data_size": 1024, "motd_lines": ["Welcome!", "Have fun"], "motd_interval": 5.0, "on_user_connected": "", "on_user_disconnected": "", "on_unsigned_data": "kick (_this select 0)", "on_hacked_data": "kick (_this select 0)", "double_id_detected": "", "headless_clients": [], "local_clients": [], "admin_uids": [], "allowed_load_extensions": ["hpp","sqs","sqf","fsm","cpp","paa","txt","xml","inc","ext","sqm","ods","fxy","lip","csv","kb","bik","bikb","html","htm","biedi"], "allowed_preprocess_extensions": ["hpp","sqs","sqf","fsm","cpp","paa","txt","xml","inc","ext","sqm","ods","fxy","lip","csv","kb","bik","bikb","html","htm","biedi"], "allowed_html_extensions": ["htm","html","xml","txt"] } ``` Section `basic` — maps to `basic.cfg` parameters: ```json { "min_bandwidth": 800000, "max_bandwidth": 25000000, "max_msg_send": 384, "max_size_guaranteed": 512, "max_size_non_guaranteed": 256, "min_error_to_send": 0.003, "max_custom_file_size": 100000 } ``` Section `profile` — maps to `server.Arma3Profile` difficulty: ```json { "reduced_damage": 0, "group_indicators": 0, "friendly_tags": 0, "enemy_tags": 0, "detected_mines": 0, "commands": 1, "waypoints": 1, "tactical_ping": 0, "weapon_info": 2, "stance_indicator": 2, "stamina_bar": 0, "weapon_crosshair": 0, "vision_aid": 0, "third_person_view": 0, "camera_shake": 1, "score_table": 1, "death_messages": 1, "von_id": 1, "map_content_friendly": 0, "map_content_enemy": 0, "map_content_mines": 0, "auto_report": 0, "multiple_saves": 0, "ai_level_preset": 3, "skill_ai": 0.5, "precision_ai": 0.5 } ``` Section `launch` — maps to CLI launch parameters: ```json { "world": "empty", "extra_params": "", "limit_fps": 50, "auto_init": 0, "load_mission_to_memory": 0, "bandwidth_alg": null, "enable_ht": 0, "huge_pages": 0, "cpu_count": null, "ex_threads": 7, "max_mem": null, "no_logs": 0, "netlog": 0 } ``` Section `rcon` — BattlEye RCon settings: ```json { "rcon_password": "encrypted:...", "max_ping": 200, "enabled": 1 } ``` --- ### Table: `mods` Registered mods. Many-to-many with servers. Scoped by `game_type`. ```sql CREATE TABLE mods ( id INTEGER PRIMARY KEY AUTOINCREMENT, game_type TEXT NOT NULL, -- scope mods by game type name TEXT NOT NULL, folder_path TEXT NOT NULL, workshop_id TEXT, -- Steam Workshop ID if applicable description TEXT, game_data TEXT DEFAULT '{}', -- JSON for game-specific mod metadata created_at TEXT NOT NULL DEFAULT (datetime('now')), UNIQUE (game_type, folder_path) ); CREATE TABLE server_mods ( server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, mod_id INTEGER NOT NULL REFERENCES mods(id) ON DELETE CASCADE, is_server_mod INTEGER NOT NULL DEFAULT 0, -- 1 = server-side only (not broadcast to clients) sort_order INTEGER NOT NULL DEFAULT 0, game_data TEXT DEFAULT '{}', -- JSON for per-server mod overrides PRIMARY KEY (server_id, mod_id) ); CREATE INDEX idx_server_mods_server ON server_mods(server_id); ``` **Key changes:** Added `game_type` to `mods` (scopes the mod registry per game). Added `game_data` JSON columns for game-specific metadata that doesn't fit the common schema. --- ### Table: `missions` Mission/scenario files tracked per server. ```sql CREATE TABLE missions ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, filename TEXT NOT NULL, -- e.g. "MyMission.Altis.pbo" mission_name TEXT NOT NULL, -- parsed by adapter terrain TEXT, -- may be NULL for non-Arma games file_size INTEGER, -- bytes game_data TEXT DEFAULT '{}', -- JSON for game-specific mission metadata uploaded_at TEXT NOT NULL DEFAULT (datetime('now')), UNIQUE (server_id, filename) ); CREATE INDEX idx_missions_server ON missions(server_id); ``` **Key changes:** `terrain` is now nullable (not all games use the mission/terrain naming convention). Added `game_data` for adapter-specific metadata. --- ### Table: `mission_rotation` Ordered mission/scenario cycle for a server. ```sql CREATE TABLE mission_rotation ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, mission_id INTEGER NOT NULL REFERENCES missions(id) ON DELETE CASCADE, sort_order INTEGER NOT NULL DEFAULT 0, difficulty TEXT, -- game-specific (NULL for games without difficulty) params_json TEXT NOT NULL DEFAULT '{}', -- mission params as JSON game_data TEXT DEFAULT '{}', -- adapter-specific rotation metadata UNIQUE (server_id, sort_order) ); CREATE INDEX idx_mission_rotation_server ON mission_rotation(server_id); ``` **Key changes:** `difficulty` is now nullable. Added `game_data` for adapter-specific rotation config. --- ### Table: `players` Currently connected players (live state, refreshed by RemoteAdminPollerThread). ```sql CREATE TABLE players ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, slot_id TEXT NOT NULL, -- Game-specific slot identifier (was player_num int) name TEXT NOT NULL, guid TEXT, -- Game-specific identifier (BattlEye GUID, Steam ID, etc.) ip TEXT, ping INTEGER, game_data TEXT DEFAULT '{}', -- JSON: {verified, steam_uid, etc.} for Arma 3 joined_at TEXT NOT NULL DEFAULT (datetime('now')), updated_at TEXT NOT NULL DEFAULT (datetime('now')), UNIQUE (server_id, slot_id) ); CREATE INDEX idx_players_server ON players(server_id); ``` **Key changes:** `player_num` (INTEGER) renamed to `slot_id` (TEXT) for flexibility across games. `steam_uid` moved into `game_data` JSON (Arma 3 specific). `verified` moved into `game_data` JSON. --- ### Table: `player_history` Historical record of connections. Inserted when player disconnects. ```sql CREATE TABLE player_history ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, name TEXT NOT NULL, guid TEXT, ip TEXT, game_data TEXT DEFAULT '{}', -- JSON for game-specific historical data joined_at TEXT NOT NULL, left_at TEXT NOT NULL DEFAULT (datetime('now')), session_duration_seconds INTEGER ); CREATE INDEX idx_player_history_server ON player_history(server_id); CREATE INDEX idx_player_history_guid ON player_history(guid); ``` ### Player History Retention Cleanup (run daily via APScheduler, keep 90 days) ```sql DELETE FROM player_history WHERE left_at < datetime('now', '-90 days'); ``` --- ### Table: `bans` Ban records. Core concept is game-agnostic; ban file sync is adapter-specific. ```sql CREATE TABLE bans ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, guid TEXT, name TEXT, reason TEXT, banned_by TEXT, -- admin username banned_at TEXT NOT NULL DEFAULT (datetime('now')), expires_at TEXT, -- NULL = permanent is_active INTEGER NOT NULL DEFAULT 1, game_data TEXT DEFAULT '{}', -- JSON: {steam_uid, ip, etc.} CHECK (is_active IN (0, 1)) ); CREATE INDEX idx_bans_server ON bans(server_id); CREATE INDEX idx_bans_guid ON bans(guid); CREATE INDEX idx_bans_active ON bans(is_active); ``` **Key changes:** Removed `steam_uid` as a dedicated column (moved to `game_data`). Added `game_data` JSON. Ban file sync (e.g., `battleye/ban.txt`) is handled by the adapter's `BanManager`. --- ### Table: `logs` Parsed log lines (rolling retention, default 7 days). ```sql CREATE TABLE logs ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, timestamp TEXT NOT NULL, level TEXT NOT NULL DEFAULT 'info', -- 'info' | 'warning' | 'error' CHECK (level IN ('info', 'warning', 'error')), message TEXT NOT NULL, created_at TEXT NOT NULL DEFAULT (datetime('now')) ); CREATE INDEX idx_logs_server_ts ON logs(server_id, timestamp); CREATE INDEX idx_logs_level ON logs(level); CREATE INDEX idx_logs_created ON logs(created_at); ``` **No changes** — this table is fully game-agnostic. The adapter's `LogParser` converts game-specific log format into the standard `{timestamp, level, message}` tuple. --- ### Table: `metrics` Time-series CPU/RAM/player count snapshots. ```sql CREATE TABLE metrics ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, timestamp TEXT NOT NULL DEFAULT (datetime('now')), cpu_percent REAL, ram_mb REAL, player_count INTEGER ); CREATE INDEX idx_metrics_server_ts ON metrics(server_id, timestamp); ``` **No changes** — fully game-agnostic. --- ### Table: `server_events` Audit trail of all significant events. ```sql CREATE TABLE server_events ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, event_type TEXT NOT NULL, -- Core event types: -- 'started' | 'stopped' | 'crashed' | 'restarted' | 'config_updated' -- 'player_kicked' | 'player_banned' | 'admin_login' -- 'auto_restarted' | 'max_restarts_exceeded' -- Adapters may define additional event types actor TEXT, -- username or 'system' detail TEXT, -- JSON with event-specific data created_at TEXT NOT NULL DEFAULT (datetime('now')) ); CREATE INDEX idx_events_server ON server_events(server_id, created_at); ``` **No changes** — fully game-agnostic. Adapters can emit additional event types. --- ## Relationships Diagram ``` users (1) ──────────────────────────────────── (many) server_events.actor servers (1) ──┬── (many) game_configs ← JSON sections replace 5 Arma 3 tables ├── (many) server_mods ──── (many) mods (scoped by game_type) ├── (many) missions ├── (many) mission_rotation → missions ├── (many) players ├── (many) player_history ├── (many) bans ├── (many) logs ├── (many) metrics └── (many) server_events ``` --- ## Encryption Strategy Sensitive fields stored in `game_configs.config_json` are encrypted at the application layer before JSON serialization. The adapter declares which fields are sensitive via `ConfigGenerator.get_sensitive_fields(section) -> list[str]`. **How it works:** 1. On write: Core calls `adapter.get_config_generator().get_sensitive_fields(section)` to get a list of JSON keys 2. ConfigRepository replaces those values with `"encrypted:" + Fernet.encrypt(value)` tokens 3. On read: ConfigRepository detects `"encrypted:"` prefix and decrypts 4. The adapter's Pydantic model sees plaintext — encryption is transparent to the adapter **Arma 3 encrypted fields (declared by Arma3ConfigGenerator):** - Section `server`: `password`, `password_admin`, `server_command_password` - Section `rcon`: `rcon_password` Encryption uses Fernet (AES-256) with `LANGUARD_ENCRYPTION_KEY`. ## Optimistic Locking for Config Updates The `config_version` column in `game_configs` prevents lost updates when two admins edit simultaneously: 1. Client reads config section → gets `config_version: 3` 2. Client sends PUT with `config_version: 3` in request body 3. Server checks: `WHERE server_id = ? AND section = ? AND config_version = 3` 4. If match: increment version, write new JSON, return 200 5. If no match (version changed): return **409 Conflict** with current config for client-side merge ## Config Schema Migration When the adapter is updated and `get_config_version()` returns a newer version than what's stored in `game_configs.schema_version`, the core automatically migrates: 1. On read, detect `stored_schema_version != adapter.get_config_version()` 2. Call `adapter.migrate_config(stored_schema_version, config_json)` → returns migrated dict 3. Update the row: `SET config_json = ?, schema_version = ? WHERE id = ?` 4. On `ConfigMigrationError`: keep original config, log warning, server runs with old schema 5. Migration is per-section — each section can have a different stored version This ensures config data is always compatible with the current adapter without manual intervention. The `game_data` columns on `players`, `missions`, `mods`, and `bans` are validated by adapter-provided Pydantic models. Each capability protocol optionally provides a schema: | Table | game_data column | Protocol Method | Arma 3 Example | |-------|-----------------|-----------------|-----------------| | `players` | `game_data` | `RemoteAdmin.get_player_data_schema()` | `{verified: bool, steam_uid: str}` | | `missions` | `game_data` | `MissionManager.get_mission_data_schema()` | `{terrain: str}` | | `mods` | `game_data` | `ModManager.get_mod_data_schema()` | `{}` (empty for Arma 3) | | `bans` | `game_data` | `BanManager.get_ban_data_schema()` | `{steam_uid: str, ip: str}` | If an adapter doesn't define a game_data schema, the field accepts any JSON object (no validation). --- ## Maintenance Queries ### Log Retention Cleanup (run daily via APScheduler) ```sql DELETE FROM logs WHERE created_at < datetime('now', '-7 days'); ``` ### Metrics Retention Cleanup (keep 30 days) ```sql DELETE FROM metrics WHERE timestamp < datetime('now', '-30 days'); ``` ### Clear disconnected players on server stop ```sql DELETE FROM players WHERE server_id = ?; ``` ### Vacuum (run weekly) ```sql VACUUM; ``` --- ## Migration Strategy - Migrations are plain `.sql` files in `backend/core/migrations/` - Naming: `001_initial_schema.sql`, `002_add_game_type.sql`, etc. - Tracked in a `schema_migrations` table: ```sql CREATE TABLE schema_migrations ( version INTEGER PRIMARY KEY, applied_at TEXT NOT NULL DEFAULT (datetime('now')) ); ``` - Applied automatically at app startup by `database.py:run_migrations()` - **Adapter-specific migrations** go in `adapters//migrations/` and are applied by the adapter's initialization (if the adapter needs extra tables beyond `game_configs`) --- ## Migration from Single-Game Schema For existing deployments with the old Arma 3-specific schema: ### SQL Migration (002_multi_game_adapter.sql) ```sql -- 1. Add game_type to servers ALTER TABLE servers ADD COLUMN game_type TEXT NOT NULL DEFAULT 'arma3'; -- 2. Create game_configs table CREATE TABLE game_configs ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE, game_type TEXT NOT NULL, section TEXT NOT NULL, config_json TEXT NOT NULL DEFAULT '{}', config_version INTEGER NOT NULL DEFAULT 1, schema_version TEXT NOT NULL DEFAULT '1.0', updated_at TEXT NOT NULL DEFAULT (datetime('now')), UNIQUE(server_id, section) ); -- 3. Add game_data columns ALTER TABLE players ADD COLUMN game_data TEXT DEFAULT '{}'; ALTER TABLE missions ADD COLUMN game_data TEXT DEFAULT '{}'; ALTER TABLE mods ADD COLUMN game_type TEXT NOT NULL DEFAULT 'arma3'; ALTER TABLE mods ADD COLUMN game_data TEXT DEFAULT '{}'; ALTER TABLE server_mods ADD COLUMN game_data TEXT DEFAULT '{}'; ALTER TABLE bans ADD COLUMN game_data TEXT DEFAULT '{}'; ALTER TABLE mission_rotation ADD COLUMN game_data TEXT DEFAULT '{}'; -- 4. Migrate player_num → slot_id (SQLite 3.25.0+) ALTER TABLE players RENAME COLUMN player_num TO slot_id; -- Change slot_id type: SQLite doesn't support ALTER COLUMN TYPE. -- Workaround: create new table, copy data, drop old, rename. -- See Python migration script below. -- 5. Drop old Arma 3-specific tables (after Python migration confirms data copied) -- DROP TABLE server_configs; -- DROP TABLE basic_configs; -- DROP TABLE server_profiles; -- DROP TABLE launch_params; -- DROP TABLE rcon_configs; ``` ### Python Migration Script (002_migrate_arma3_config.py) This script reads old Arma 3-specific tables, serializes each row to JSON, and inserts into `game_configs`. It runs AFTER the SQL migration creates the new table structure. ```python """Migrate Arma 3 config data from normalized tables to game_configs JSON. Run after 002_multi_game_adapter.sql. This script: 1. Reads each old table row by row 2. Converts column values to JSON (handling type conversions) 3. Inserts into game_configs with proper section names 4. Migrates player data (player_num int → slot_id text, steam_uid → game_data) 5. Verifies row counts match before dropping old tables Type conversion rules: - INTEGER 0/1 → JSON boolean false/true (for boolean-like fields) - TEXT containing JSON arrays (motd_lines, headless_clients, etc.) → parsed JSON arrays - NULL values → omitted from JSON (Pydantic fills defaults on read) - Encrypted fields (password, password_admin, rcon_password) → copied as-is (already encrypted) Transaction: all inserts in a single transaction. On failure, rollback — old tables untouched. """ import json import sqlite3 COLUMN_TYPE_MAP = { # server_configs → section 'server' 'server_configs': { 'section': 'server', 'boolean_fields': [ 'kick_duplicate', 'persistent', 'disable_von', 'battleye', 'kick_on_ping', 'kick_on_packet_loss', 'kick_on_desync', 'kick_on_timeout', 'auto_select_mission', 'random_mission_order', 'skip_lobby', 'drawing_in_map', 'upnp', 'loopback', 'statistics_enabled', ], 'json_array_fields': [ 'motd_lines', 'headless_clients', 'local_clients', 'admin_uids', 'allowed_load_extensions', 'allowed_preprocess_extensions', 'allowed_html_extensions', ], 'encrypted_fields': ['password', 'password_admin', 'server_command_password'], }, # basic_configs → section 'basic' 'basic_configs': { 'section': 'basic', 'boolean_fields': [], 'json_array_fields': [], 'encrypted_fields': [], }, # server_profiles → section 'profile' 'server_profiles': { 'section': 'profile', 'boolean_fields': [ 'reduced_damage', 'camera_shake', 'score_table', 'death_messages', 'von_id', 'auto_report', 'multiple_saves', 'stamina_bar', 'weapon_crosshair', 'vision_aid', 'third_person_view', 'map_content_friendly', 'map_content_enemy', 'map_content_mines', ], 'json_array_fields': [], 'encrypted_fields': [], }, # launch_params → section 'launch' 'launch_params': { 'section': 'launch', 'boolean_fields': [ 'auto_init', 'load_mission_to_memory', 'enable_ht', 'huge_pages', 'no_logs', 'netlog', ], 'json_array_fields': [], 'encrypted_fields': [], }, # rcon_configs → section 'rcon' 'rcon_configs': { 'section': 'rcon', 'boolean_fields': ['enabled'], 'json_array_fields': [], 'encrypted_fields': ['rcon_password'], }, } def migrate_config_table(db: sqlite3.Connection, table: str, mapping: dict): """Read rows from old table, serialize to JSON, insert into game_configs.""" rows = db.execute(f"SELECT * FROM {table}").fetchall() columns = [desc[0] for desc in db.execute(f"SELECT * FROM {table} LIMIT 0").description] for row in rows: row_dict = dict(zip(columns, row)) server_id = row_dict.pop('id', None) or row_dict.pop('server_id', None) config = {} for col, value in row_dict.items(): if value is None: continue # omit NULLs; Pydantic fills defaults if col in mapping.get('boolean_fields', []): config[col] = bool(value) elif col in mapping.get('json_array_fields', []): config[col] = json.loads(value) if isinstance(value, str) else value elif col in mapping.get('encrypted_fields', []): config[col] = value # already encrypted, copy as-is elif col in ('updated_at', 'created_at', 'id', 'server_id'): continue # skip metadata columns else: config[col] = value db.execute(""" INSERT INTO game_configs (server_id, game_type, section, config_json, schema_version) VALUES (?, 'arma3', ?, ?, '1.0') """, (server_id, mapping['section'], json.dumps(config))) def migrate_player_data(db: sqlite3.Connection): """Convert player_num (int) to slot_id (text), move steam_uid to game_data.""" # Create new players table with slot_id as TEXT db.execute(""" CREATE TABLE players_new ( id INTEGER PRIMARY KEY AUTOINCREMENT, server_id INTEGER NOT NULL, slot_id TEXT NOT NULL, name TEXT NOT NULL, guid TEXT, ip TEXT, ping INTEGER, game_data TEXT DEFAULT '{}', joined_at TEXT NOT NULL DEFAULT (datetime('now')), updated_at TEXT NOT NULL DEFAULT (datetime('now')), UNIQUE (server_id, slot_id) ) """) rows = db.execute("SELECT * FROM players").fetchall() columns = [desc[0] for desc in db.execute("SELECT * FROM players LIMIT 0").description] for row in rows: d = dict(zip(columns, row)) game_data = {} if d.get('steam_uid'): game_data['steam_uid'] = d['steam_uid'] if d.get('verified') is not None: game_data['verified'] = bool(d['verified']) db.execute(""" INSERT INTO players_new (id, server_id, slot_id, name, guid, ip, ping, game_data, joined_at, updated_at) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?) """, ( d['id'], d['server_id'], str(d['player_num']), d['name'], d.get('guid'), d.get('ip'), d.get('ping'), json.dumps(game_data), d.get('joined_at'), d.get('updated_at'), )) db.execute("DROP TABLE players") db.execute("ALTER TABLE players_new RENAME TO players") db.execute("CREATE INDEX idx_players_server ON players(server_id)") def run_migration(db_path: str): db = sqlite3.connect(db_path) try: db.execute("BEGIN TRANSACTION") for table, mapping in COLUMN_TYPE_MAP.items(): count_before = db.execute(f"SELECT COUNT(*) FROM {table}").fetchone()[0] migrate_config_table(db, table, mapping) count_after = db.execute( "SELECT COUNT(*) FROM game_configs WHERE game_type='arma3' AND section=?", (mapping['section'],) ).fetchone()[0] assert count_after == count_before, ( f"Row count mismatch for {table}: {count_before} → {count_after}" ) migrate_player_data(db) db.execute("COMMIT") print("Migration successful. Old tables can now be dropped.") except Exception as e: db.execute("ROLLBACK") print(f"Migration FAILED (rolled back): {e}") raise finally: db.close() ```