languard-servers-manager/DATABASE.md

Languard Servers Manager — Database Design

Engine

• SQLite via SQLAlchemy Core (sync for all access — routes and threads)
• File: languard.db at project root (configurable via LANGUARD_DB_PATH)
• WAL mode enabled: PRAGMA journal_mode=WAL — allows concurrent reads during writes
• Foreign keys enabled: PRAGMA foreign_keys=ON
• Busy timeout: PRAGMA busy_timeout=5000 — prevents "database is locked" errors under concurrent thread writes
• Retry on exhaustion: If busy_timeout is exceeded (5s), writes fail with OperationalError("database is locked"). Background threads retry with exponential backoff (1s, 2s, 4s), then skip the tick. API handlers retry up to 2 times with 1s backoff, then return 503. See THREADING.md for the full retry implementation.

---

Design Philosophy

The database uses a hybrid approach: core tables are fully normalized (game-agnostic), while game-specific config is stored as JSON blobs in a generic game_configs table, validated by adapter Pydantic models at the application layer.

Why this works for Languard:

• Config is always read/written as a whole section (nobody queries "find all servers where von_codec_quality > 20")
• Each adapter section maps to a Pydantic model, so validation is enforced at the application layer
• The JSON is opaque to the core, meaningful only to the adapter that owns it
• Adding a new game requires zero DB migration — just a new adapter
• Core queries across all games work naturally

---

Schema

Table: users

Stores web UI admin accounts.

CREATE TABLE users (
    id           INTEGER PRIMARY KEY AUTOINCREMENT,
    username     TEXT    NOT NULL UNIQUE,
    password_hash TEXT   NOT NULL,              -- bcrypt hash
    role         TEXT    NOT NULL DEFAULT 'viewer', -- 'admin' | 'viewer'
    CHECK (role IN ('admin', 'viewer')),
    created_at   TEXT    NOT NULL DEFAULT (datetime('now')),
    last_login   TEXT
);

---

Table: servers

One row per managed server instance. Game-agnostic.

CREATE TABLE servers (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    name            TEXT    NOT NULL,              -- display name in UI
    description     TEXT,
    game_type       TEXT    NOT NULL DEFAULT 'arma3',  -- adapter lookup key
    status          TEXT    NOT NULL DEFAULT 'stopped',
    CHECK (status IN ('stopped', 'starting', 'running', 'stopping', 'crashed', 'error')),

    -- Process info
    pid             INTEGER,                        -- OS process ID when running
    exe_path        TEXT    NOT NULL,               -- path to server executable
    started_at      TEXT,                           -- ISO datetime
    stopped_at      TEXT,

    -- Network (core ports; adapter defines derived port conventions)
    game_port       INTEGER NOT NULL,
    rcon_port       INTEGER,                       -- NULL if game has no remote admin
    CHECK (game_port BETWEEN 1024 AND 65535),
    CHECK (rcon_port IS NULL OR (rcon_port BETWEEN 1024 AND 65535)),

    -- Auto-management
    auto_restart    INTEGER NOT NULL DEFAULT 0,     -- 1 = restart on crash
    max_restarts    INTEGER NOT NULL DEFAULT 3,     -- within restart_window_seconds
    restart_window_seconds INTEGER NOT NULL DEFAULT 300,
    restart_count   INTEGER NOT NULL DEFAULT 0,
    last_restart_at TEXT,

    created_at      TEXT    NOT NULL DEFAULT (datetime('now')),
    updated_at      TEXT    NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX idx_servers_status ON servers(status);
CREATE INDEX idx_servers_game_type ON servers(game_type);
CREATE INDEX idx_servers_game_port ON servers(game_port);

Key changes from single-game design:

• Added game_type column (defaults to 'arma3' for backward compatibility)
• Removed Arma 3-specific defaults (DEFAULT 2302, DEFAULT 2306) — port defaults are now adapter-provided
• Removed steam_query_port virtual column (Arma 3 convention moved to adapter)
• rcon_port is now nullable (some games have no remote admin)

---

Table: game_configs

Stores all game-specific configuration as JSON blobs, keyed by section. Replaces the previous server_configs, basic_configs, server_profiles, launch_params, and rcon_configs tables.

CREATE TABLE game_configs (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    game_type   TEXT    NOT NULL,              -- for validation; matches servers.game_type
    section     TEXT    NOT NULL,              -- e.g. 'server', 'basic', 'profile', 'launch', 'rcon'
    config_json TEXT    NOT NULL DEFAULT '{}', -- JSON validated by adapter's Pydantic model
    config_version INTEGER NOT NULL DEFAULT 1, -- optimistic locking version; incremented on each write
    schema_version TEXT    NOT NULL DEFAULT '1.0', -- adapter schema version at time of last write
    updated_at  TEXT    NOT NULL DEFAULT (datetime('now')),
    UNIQUE(server_id, section)
);

CREATE INDEX idx_game_configs_server ON game_configs(server_id);
CREATE INDEX idx_game_configs_type_section ON game_configs(game_type, section);

How it works:

• Each adapter defines config sections via ConfigGenerator.get_sections() → {section_name: PydanticModelClass}
• Core reads/writes JSON blobs; adapter validates them on write and parses them on read
• Arma 3 has 5 sections: server, basic, profile, launch, rcon
• Another game might have 2 sections or 8 — no migration needed
• Sensitive fields within JSON (passwords, rcon_password) are encrypted at the application layer via Fernet before storage

Arma 3 config sections and their JSON structures:

Section server — maps to server.cfg parameters:

{
  "hostname": "My Arma 3 Server",
  "password": "encrypted:...",
  "password_admin": "encrypted:...",
  "server_command_password": "encrypted:...",
  "max_players": 40,
  "kick_duplicate": 1,
  "persistent": 1,
  "vote_threshold": 0.33,
  "vote_mission_players": 1,
  "vote_timeout": 60,
  "role_timeout": 90,
  "briefing_timeout": 60,
  "debriefing_timeout": 45,
  "lobby_idle_timeout": 300,
  "disable_von": 0,
  "von_codec": 1,
  "von_codec_quality": 20,
  "max_ping": 250,
  "max_packet_loss": 50,
  "max_desync": 200,
  "disconnect_timeout": 15,
  "kick_on_ping": 1,
  "kick_on_packet_loss": 1,
  "kick_on_desync": 1,
  "kick_on_timeout": 1,
  "battleye": 1,
  "verify_signatures": 2,
  "allowed_file_patching": 0,
  "forced_difficulty": "Regular",
  "timestamp_format": "short",
  "auto_select_mission": 0,
  "random_mission_order": 0,
  "missions_to_restart": 0,
  "missions_to_shutdown": 0,
  "log_file": "server_console.log",
  "skip_lobby": 0,
  "drawing_in_map": 1,
  "upnp": 0,
  "loopback": 0,
  "statistics_enabled": 1,
  "force_rotor_lib": 0,
  "required_build": 0,
  "steam_protocol_max_data_size": 1024,
  "motd_lines": ["Welcome!", "Have fun"],
  "motd_interval": 5.0,
  "on_user_connected": "",
  "on_user_disconnected": "",
  "on_unsigned_data": "kick (_this select 0)",
  "on_hacked_data": "kick (_this select 0)",
  "double_id_detected": "",
  "headless_clients": [],
  "local_clients": [],
  "admin_uids": [],
  "allowed_load_extensions": ["hpp","sqs","sqf","fsm","cpp","paa","txt","xml","inc","ext","sqm","ods","fxy","lip","csv","kb","bik","bikb","html","htm","biedi"],
  "allowed_preprocess_extensions": ["hpp","sqs","sqf","fsm","cpp","paa","txt","xml","inc","ext","sqm","ods","fxy","lip","csv","kb","bik","bikb","html","htm","biedi"],
  "allowed_html_extensions": ["htm","html","xml","txt"]
}

Section basic — maps to basic.cfg parameters:

{
  "min_bandwidth": 800000,
  "max_bandwidth": 25000000,
  "max_msg_send": 384,
  "max_size_guaranteed": 512,
  "max_size_non_guaranteed": 256,
  "min_error_to_send": 0.003,
  "max_custom_file_size": 100000
}

Section profile — maps to server.Arma3Profile difficulty:

{
  "reduced_damage": 0,
  "group_indicators": 0,
  "friendly_tags": 0,
  "enemy_tags": 0,
  "detected_mines": 0,
  "commands": 1,
  "waypoints": 1,
  "tactical_ping": 0,
  "weapon_info": 2,
  "stance_indicator": 2,
  "stamina_bar": 0,
  "weapon_crosshair": 0,
  "vision_aid": 0,
  "third_person_view": 0,
  "camera_shake": 1,
  "score_table": 1,
  "death_messages": 1,
  "von_id": 1,
  "map_content_friendly": 0,
  "map_content_enemy": 0,
  "map_content_mines": 0,
  "auto_report": 0,
  "multiple_saves": 0,
  "ai_level_preset": 3,
  "skill_ai": 0.5,
  "precision_ai": 0.5
}

Section launch — maps to CLI launch parameters:

{
  "world": "empty",
  "extra_params": "",
  "limit_fps": 50,
  "auto_init": 0,
  "load_mission_to_memory": 0,
  "bandwidth_alg": null,
  "enable_ht": 0,
  "huge_pages": 0,
  "cpu_count": null,
  "ex_threads": 7,
  "max_mem": null,
  "no_logs": 0,
  "netlog": 0
}

Section rcon — BattlEye RCon settings:

{
  "rcon_password": "encrypted:...",
  "max_ping": 200,
  "enabled": 1
}

---

Table: mods

Registered mods. Many-to-many with servers. Scoped by game_type.

CREATE TABLE mods (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    game_type   TEXT    NOT NULL,              -- scope mods by game type
    name        TEXT    NOT NULL,
    folder_path TEXT    NOT NULL,
    workshop_id TEXT,                          -- Steam Workshop ID if applicable
    description TEXT,
    game_data   TEXT    DEFAULT '{}',          -- JSON for game-specific mod metadata
    created_at  TEXT    NOT NULL DEFAULT (datetime('now')),
    UNIQUE (game_type, folder_path)
);

CREATE TABLE server_mods (
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    mod_id      INTEGER NOT NULL REFERENCES mods(id) ON DELETE CASCADE,
    is_server_mod INTEGER NOT NULL DEFAULT 0,   -- 1 = server-side only (not broadcast to clients)
    sort_order  INTEGER NOT NULL DEFAULT 0,
    game_data   TEXT    DEFAULT '{}',          -- JSON for per-server mod overrides
    PRIMARY KEY (server_id, mod_id)
);

CREATE INDEX idx_server_mods_server ON server_mods(server_id);

Key changes: Added game_type to mods (scopes the mod registry per game). Added game_data JSON columns for game-specific metadata that doesn't fit the common schema.

---

Table: missions

Mission/scenario files tracked per server.

CREATE TABLE missions (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    filename    TEXT    NOT NULL,           -- e.g. "MyMission.Altis.pbo"
    mission_name TEXT   NOT NULL,           -- parsed by adapter
    terrain     TEXT,                       -- may be NULL for non-Arma games
    file_size   INTEGER,                    -- bytes
    game_data   TEXT    DEFAULT '{}',       -- JSON for game-specific mission metadata
    uploaded_at TEXT    NOT NULL DEFAULT (datetime('now')),
    UNIQUE (server_id, filename)
);

CREATE INDEX idx_missions_server ON missions(server_id);

Key changes: terrain is now nullable (not all games use the mission/terrain naming convention). Added game_data for adapter-specific metadata.

---

Table: mission_rotation

Ordered mission/scenario cycle for a server.

CREATE TABLE mission_rotation (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    mission_id  INTEGER NOT NULL REFERENCES missions(id) ON DELETE CASCADE,
    sort_order  INTEGER NOT NULL DEFAULT 0,
    difficulty  TEXT,                        -- game-specific (NULL for games without difficulty)
    params_json TEXT    NOT NULL DEFAULT '{}',  -- mission params as JSON
    game_data   TEXT    DEFAULT '{}',       -- adapter-specific rotation metadata
    UNIQUE (server_id, sort_order)
);

CREATE INDEX idx_mission_rotation_server ON mission_rotation(server_id);

Key changes: difficulty is now nullable. Added game_data for adapter-specific rotation config.

---

Table: players

Currently connected players (live state, refreshed by RemoteAdminPollerThread).

CREATE TABLE players (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    slot_id     TEXT    NOT NULL,           -- Game-specific slot identifier (was player_num int)
    name        TEXT    NOT NULL,
    guid        TEXT,                       -- Game-specific identifier (BattlEye GUID, Steam ID, etc.)
    ip          TEXT,
    ping        INTEGER,
    game_data   TEXT    DEFAULT '{}',       -- JSON: {verified, steam_uid, etc.} for Arma 3
    joined_at   TEXT    NOT NULL DEFAULT (datetime('now')),
    updated_at  TEXT    NOT NULL DEFAULT (datetime('now')),
    UNIQUE (server_id, slot_id)
);

CREATE INDEX idx_players_server ON players(server_id);

Key changes: player_num (INTEGER) renamed to slot_id (TEXT) for flexibility across games. steam_uid moved into game_data JSON (Arma 3 specific). verified moved into game_data JSON.

---

Table: player_history

Historical record of connections. Inserted when player disconnects.

CREATE TABLE player_history (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    name        TEXT    NOT NULL,
    guid        TEXT,
    ip          TEXT,
    game_data   TEXT    DEFAULT '{}',       -- JSON for game-specific historical data
    joined_at   TEXT    NOT NULL,
    left_at     TEXT    NOT NULL DEFAULT (datetime('now')),
    session_duration_seconds INTEGER
);

CREATE INDEX idx_player_history_server ON player_history(server_id);
CREATE INDEX idx_player_history_guid ON player_history(guid);

Player History Retention Cleanup (run daily via APScheduler, keep 90 days)

DELETE FROM player_history
WHERE left_at < datetime('now', '-90 days');

---

Table: bans

Ban records. Core concept is game-agnostic; ban file sync is adapter-specific.

CREATE TABLE bans (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    guid        TEXT,
    name        TEXT,
    reason      TEXT,
    banned_by   TEXT,                       -- admin username
    banned_at   TEXT    NOT NULL DEFAULT (datetime('now')),
    expires_at  TEXT,                       -- NULL = permanent
    is_active   INTEGER NOT NULL DEFAULT 1,
    game_data   TEXT    DEFAULT '{}',       -- JSON: {steam_uid, ip, etc.}
    CHECK (is_active IN (0, 1))
);

CREATE INDEX idx_bans_server ON bans(server_id);
CREATE INDEX idx_bans_guid ON bans(guid);
CREATE INDEX idx_bans_active ON bans(is_active);

Key changes: Removed steam_uid as a dedicated column (moved to game_data). Added game_data JSON. Ban file sync (e.g., battleye/ban.txt) is handled by the adapter's BanManager.

---

Table: logs

Parsed log lines (rolling retention, default 7 days).

CREATE TABLE logs (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    timestamp   TEXT    NOT NULL,
    level       TEXT    NOT NULL DEFAULT 'info',  -- 'info' | 'warning' | 'error'
    CHECK (level IN ('info', 'warning', 'error')),
    message     TEXT    NOT NULL,
    created_at  TEXT    NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX idx_logs_server_ts ON logs(server_id, timestamp);
CREATE INDEX idx_logs_level ON logs(level);
CREATE INDEX idx_logs_created ON logs(created_at);

No changes — this table is fully game-agnostic. The adapter's LogParser converts game-specific log format into the standard {timestamp, level, message} tuple.

---

Table: metrics

Time-series CPU/RAM/player count snapshots.

CREATE TABLE metrics (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id       INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    timestamp       TEXT    NOT NULL DEFAULT (datetime('now')),
    cpu_percent     REAL,
    ram_mb          REAL,
    player_count    INTEGER
);

CREATE INDEX idx_metrics_server_ts ON metrics(server_id, timestamp);

No changes — fully game-agnostic.

---

Table: server_events

Audit trail of all significant events.

CREATE TABLE server_events (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    event_type  TEXT    NOT NULL,
    -- Core event types:
    --   'started' | 'stopped' | 'crashed' | 'restarted' | 'config_updated'
    --   'player_kicked' | 'player_banned' | 'admin_login'
    --   'auto_restarted' | 'max_restarts_exceeded'
    -- Adapters may define additional event types
    actor       TEXT,                       -- username or 'system'
    detail      TEXT,                       -- JSON with event-specific data
    created_at  TEXT    NOT NULL DEFAULT (datetime('now'))
);

CREATE INDEX idx_events_server ON server_events(server_id, created_at);

No changes — fully game-agnostic. Adapters can emit additional event types.

---

Relationships Diagram

users (1) ──────────────────────────────────── (many) server_events.actor

servers (1) ──┬── (many) game_configs           ← JSON sections replace 5 Arma 3 tables
              ├── (many) server_mods ──── (many) mods (scoped by game_type)
              ├── (many) missions
              ├── (many) mission_rotation → missions
              ├── (many) players
              ├── (many) player_history
              ├── (many) bans
              ├── (many) logs
              ├── (many) metrics
              └── (many) server_events

---

Encryption Strategy

Sensitive fields stored in game_configs.config_json are encrypted at the application layer before JSON serialization. The adapter declares which fields are sensitive via ConfigGenerator.get_sensitive_fields(section) -> list[str].

How it works:

1. On write: Core calls adapter.get_config_generator().get_sensitive_fields(section) to get a list of JSON keys
2. ConfigRepository replaces those values with "encrypted:" + Fernet.encrypt(value) tokens
3. On read: ConfigRepository detects "encrypted:" prefix and decrypts
4. The adapter's Pydantic model sees plaintext — encryption is transparent to the adapter

Arma 3 encrypted fields (declared by Arma3ConfigGenerator):

• Section server: password, password_admin, server_command_password
• Section rcon: rcon_password

Encryption uses Fernet (AES-256) with LANGUARD_ENCRYPTION_KEY.

Optimistic Locking for Config Updates

The config_version column in game_configs prevents lost updates when two admins edit simultaneously:

1. Client reads config section → gets config_version: 3
2. Client sends PUT with config_version: 3 in request body
3. Server checks: WHERE server_id = ? AND section = ? AND config_version = 3
4. If match: increment version, write new JSON, return 200
5. If no match (version changed): return 409 Conflict with current config for client-side merge

Config Schema Migration

When the adapter is updated and get_config_version() returns a newer version than what's stored in game_configs.schema_version, the core automatically migrates:

1. On read, detect stored_schema_version != adapter.get_config_version()
2. Call adapter.migrate_config(stored_schema_version, config_json) → returns migrated dict
3. Update the row: SET config_json = ?, schema_version = ? WHERE id = ?
4. On ConfigMigrationError: keep original config, log warning, server runs with old schema
5. Migration is per-section — each section can have a different stored version

This ensures config data is always compatible with the current adapter without manual intervention.

The game_data columns on players, missions, mods, and bans are validated by adapter-provided Pydantic models. Each capability protocol optionally provides a schema:

Table	game_data column	Protocol Method	Arma 3 Example
players	game_data	RemoteAdmin.get_player_data_schema()	{verified: bool, steam_uid: str}
missions	game_data	MissionManager.get_mission_data_schema()	{terrain: str}
mods	game_data	ModManager.get_mod_data_schema()	{} (empty for Arma 3)
bans	game_data	BanManager.get_ban_data_schema()	{steam_uid: str, ip: str}

If an adapter doesn't define a game_data schema, the field accepts any JSON object (no validation).

---

Maintenance Queries

Log Retention Cleanup (run daily via APScheduler)

DELETE FROM logs
WHERE created_at < datetime('now', '-7 days');

Metrics Retention Cleanup (keep 30 days)

DELETE FROM metrics
WHERE timestamp < datetime('now', '-30 days');

Clear disconnected players on server stop

DELETE FROM players WHERE server_id = ?;

Vacuum (run weekly)

VACUUM;

---

Migration Strategy

• Migrations are plain .sql files in backend/core/migrations/
• Naming: 001_initial_schema.sql, 002_add_game_type.sql, etc.
• Tracked in a schema_migrations table:

  CREATE TABLE schema_migrations (
      version  INTEGER PRIMARY KEY,
      applied_at TEXT NOT NULL DEFAULT (datetime('now'))
  );
  

• Applied automatically at app startup by database.py:run_migrations()
• Adapter-specific migrations go in adapters/<game_type>/migrations/ and are applied by the adapter's initialization (if the adapter needs extra tables beyond game_configs)

---

Migration from Single-Game Schema

For existing deployments with the old Arma 3-specific schema:

SQL Migration (002_multi_game_adapter.sql)

-- 1. Add game_type to servers
ALTER TABLE servers ADD COLUMN game_type TEXT NOT NULL DEFAULT 'arma3';

-- 2. Create game_configs table
CREATE TABLE game_configs (
    id          INTEGER PRIMARY KEY AUTOINCREMENT,
    server_id   INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
    game_type   TEXT    NOT NULL,
    section     TEXT    NOT NULL,
    config_json TEXT    NOT NULL DEFAULT '{}',
    config_version INTEGER NOT NULL DEFAULT 1,
    schema_version TEXT    NOT NULL DEFAULT '1.0',
    updated_at  TEXT    NOT NULL DEFAULT (datetime('now')),
    UNIQUE(server_id, section)
);

-- 3. Add game_data columns
ALTER TABLE players ADD COLUMN game_data TEXT DEFAULT '{}';
ALTER TABLE missions ADD COLUMN game_data TEXT DEFAULT '{}';
ALTER TABLE mods ADD COLUMN game_type TEXT NOT NULL DEFAULT 'arma3';
ALTER TABLE mods ADD COLUMN game_data TEXT DEFAULT '{}';
ALTER TABLE server_mods ADD COLUMN game_data TEXT DEFAULT '{}';
ALTER TABLE bans ADD COLUMN game_data TEXT DEFAULT '{}';
ALTER TABLE mission_rotation ADD COLUMN game_data TEXT DEFAULT '{}';

-- 4. Migrate player_num → slot_id (SQLite 3.25.0+)
ALTER TABLE players RENAME COLUMN player_num TO slot_id;
-- Change slot_id type: SQLite doesn't support ALTER COLUMN TYPE.
-- Workaround: create new table, copy data, drop old, rename.
-- See Python migration script below.

-- 5. Drop old Arma 3-specific tables (after Python migration confirms data copied)
-- DROP TABLE server_configs;
-- DROP TABLE basic_configs;
-- DROP TABLE server_profiles;
-- DROP TABLE launch_params;
-- DROP TABLE rcon_configs;

Python Migration Script (002_migrate_arma3_config.py)

This script reads old Arma 3-specific tables, serializes each row to JSON, and inserts into game_configs. It runs AFTER the SQL migration creates the new table structure.

"""Migrate Arma 3 config data from normalized tables to game_configs JSON.

Run after 002_multi_game_adapter.sql. This script:
1. Reads each old table row by row
2. Converts column values to JSON (handling type conversions)
3. Inserts into game_configs with proper section names
4. Migrates player data (player_num int → slot_id text, steam_uid → game_data)
5. Verifies row counts match before dropping old tables

Type conversion rules:
  - INTEGER 0/1 → JSON boolean false/true (for boolean-like fields)
  - TEXT containing JSON arrays (motd_lines, headless_clients, etc.) → parsed JSON arrays
  - NULL values → omitted from JSON (Pydantic fills defaults on read)
  - Encrypted fields (password, password_admin, rcon_password) → copied as-is (already encrypted)

Transaction: all inserts in a single transaction. On failure, rollback — old tables untouched.
"""

import json
import sqlite3

COLUMN_TYPE_MAP = {
    # server_configs → section 'server'
    'server_configs': {
        'section': 'server',
        'boolean_fields': [
            'kick_duplicate', 'persistent', 'disable_von', 'battleye',
            'kick_on_ping', 'kick_on_packet_loss', 'kick_on_desync', 'kick_on_timeout',
            'auto_select_mission', 'random_mission_order', 'skip_lobby',
            'drawing_in_map', 'upnp', 'loopback', 'statistics_enabled',
        ],
        'json_array_fields': [
            'motd_lines', 'headless_clients', 'local_clients', 'admin_uids',
            'allowed_load_extensions', 'allowed_preprocess_extensions',
            'allowed_html_extensions',
        ],
        'encrypted_fields': ['password', 'password_admin', 'server_command_password'],
    },
    # basic_configs → section 'basic'
    'basic_configs': {
        'section': 'basic',
        'boolean_fields': [],
        'json_array_fields': [],
        'encrypted_fields': [],
    },
    # server_profiles → section 'profile'
    'server_profiles': {
        'section': 'profile',
        'boolean_fields': [
            'reduced_damage', 'camera_shake', 'score_table', 'death_messages',
            'von_id', 'auto_report', 'multiple_saves', 'stamina_bar',
            'weapon_crosshair', 'vision_aid', 'third_person_view',
            'map_content_friendly', 'map_content_enemy', 'map_content_mines',
        ],
        'json_array_fields': [],
        'encrypted_fields': [],
    },
    # launch_params → section 'launch'
    'launch_params': {
        'section': 'launch',
        'boolean_fields': [
            'auto_init', 'load_mission_to_memory', 'enable_ht',
            'huge_pages', 'no_logs', 'netlog',
        ],
        'json_array_fields': [],
        'encrypted_fields': [],
    },
    # rcon_configs → section 'rcon'
    'rcon_configs': {
        'section': 'rcon',
        'boolean_fields': ['enabled'],
        'json_array_fields': [],
        'encrypted_fields': ['rcon_password'],
    },
}

def migrate_config_table(db: sqlite3.Connection, table: str, mapping: dict):
    """Read rows from old table, serialize to JSON, insert into game_configs."""
    rows = db.execute(f"SELECT * FROM {table}").fetchall()
    columns = [desc[0] for desc in db.execute(f"SELECT * FROM {table} LIMIT 0").description]

    for row in rows:
        row_dict = dict(zip(columns, row))
        server_id = row_dict.pop('id', None) or row_dict.pop('server_id', None)

        config = {}
        for col, value in row_dict.items():
            if value is None:
                continue  # omit NULLs; Pydantic fills defaults
            if col in mapping.get('boolean_fields', []):
                config[col] = bool(value)
            elif col in mapping.get('json_array_fields', []):
                config[col] = json.loads(value) if isinstance(value, str) else value
            elif col in mapping.get('encrypted_fields', []):
                config[col] = value  # already encrypted, copy as-is
            elif col in ('updated_at', 'created_at', 'id', 'server_id'):
                continue  # skip metadata columns
            else:
                config[col] = value

        db.execute("""
            INSERT INTO game_configs (server_id, game_type, section, config_json, schema_version)
            VALUES (?, 'arma3', ?, ?, '1.0')
        """, (server_id, mapping['section'], json.dumps(config)))


def migrate_player_data(db: sqlite3.Connection):
    """Convert player_num (int) to slot_id (text), move steam_uid to game_data."""
    # Create new players table with slot_id as TEXT
    db.execute("""
        CREATE TABLE players_new (
            id          INTEGER PRIMARY KEY AUTOINCREMENT,
            server_id   INTEGER NOT NULL,
            slot_id     TEXT    NOT NULL,
            name        TEXT    NOT NULL,
            guid        TEXT,
            ip          TEXT,
            ping        INTEGER,
            game_data   TEXT    DEFAULT '{}',
            joined_at   TEXT    NOT NULL DEFAULT (datetime('now')),
            updated_at  TEXT    NOT NULL DEFAULT (datetime('now')),
            UNIQUE (server_id, slot_id)
        )
    """)
    rows = db.execute("SELECT * FROM players").fetchall()
    columns = [desc[0] for desc in db.execute("SELECT * FROM players LIMIT 0").description]
    for row in rows:
        d = dict(zip(columns, row))
        game_data = {}
        if d.get('steam_uid'):
            game_data['steam_uid'] = d['steam_uid']
        if d.get('verified') is not None:
            game_data['verified'] = bool(d['verified'])
        db.execute("""
            INSERT INTO players_new (id, server_id, slot_id, name, guid, ip, ping,
                                      game_data, joined_at, updated_at)
            VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
        """, (
            d['id'], d['server_id'], str(d['player_num']),
            d['name'], d.get('guid'), d.get('ip'), d.get('ping'),
            json.dumps(game_data), d.get('joined_at'), d.get('updated_at'),
        ))
    db.execute("DROP TABLE players")
    db.execute("ALTER TABLE players_new RENAME TO players")
    db.execute("CREATE INDEX idx_players_server ON players(server_id)")


def run_migration(db_path: str):
    db = sqlite3.connect(db_path)
    try:
        db.execute("BEGIN TRANSACTION")
        for table, mapping in COLUMN_TYPE_MAP.items():
            count_before = db.execute(f"SELECT COUNT(*) FROM {table}").fetchone()[0]
            migrate_config_table(db, table, mapping)
            count_after = db.execute(
                "SELECT COUNT(*) FROM game_configs WHERE game_type='arma3' AND section=?",
                (mapping['section'],)
            ).fetchone()[0]
            assert count_after == count_before, (
                f"Row count mismatch for {table}: {count_before} → {count_after}"
            )
        migrate_player_data(db)
        db.execute("COMMIT")
        print("Migration successful. Old tables can now be dropped.")
    except Exception as e:
        db.execute("ROLLBACK")
        print(f"Migration FAILED (rolled back): {e}")
        raise
    finally:
        db.close()