- Revamp architecture for modular game server support (Arma 3 first, extensible) - Merge ConfigSchema into ConfigGenerator per council decision (8→7 protocols) - Add has_capability() method to GameAdapter protocol for explicit capability probing - Add FRONTEND.md: production-grade dark neumorphism design with amber/orange palette - Update all docs (ARCHITECTURE, MODULES, DATABASE, API, IMPLEMENTATION_PLAN, THREADING) to reflect protocol merge and multi-game adapter patterns
818 lines
29 KiB
Markdown
818 lines
29 KiB
Markdown
# Languard Servers Manager — Database Design
|
|
|
|
## Engine
|
|
- **SQLite** via `SQLAlchemy Core` (sync for all access — routes and threads)
|
|
- File: `languard.db` at project root (configurable via `LANGUARD_DB_PATH`)
|
|
- WAL mode enabled: `PRAGMA journal_mode=WAL` — allows concurrent reads during writes
|
|
- Foreign keys enabled: `PRAGMA foreign_keys=ON`
|
|
- Busy timeout: `PRAGMA busy_timeout=5000` — prevents "database is locked" errors under concurrent thread writes
|
|
|
|
---
|
|
|
|
## Design Philosophy
|
|
|
|
The database uses a **hybrid approach**: core tables are fully normalized (game-agnostic), while game-specific config is stored as JSON blobs in a generic `game_configs` table, validated by adapter Pydantic models at the application layer.
|
|
|
|
**Why this works for Languard:**
|
|
- Config is always read/written as a whole section (nobody queries "find all servers where von_codec_quality > 20")
|
|
- Each adapter section maps to a Pydantic model, so validation is enforced at the application layer
|
|
- The JSON is opaque to the core, meaningful only to the adapter that owns it
|
|
- Adding a new game requires **zero DB migration** — just a new adapter
|
|
- Core queries across all games work naturally
|
|
|
|
---
|
|
|
|
## Schema
|
|
|
|
### Table: `users`
|
|
|
|
Stores web UI admin accounts.
|
|
|
|
```sql
|
|
CREATE TABLE users (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
username TEXT NOT NULL UNIQUE,
|
|
password_hash TEXT NOT NULL, -- bcrypt hash
|
|
role TEXT NOT NULL DEFAULT 'viewer', -- 'admin' | 'viewer'
|
|
CHECK (role IN ('admin', 'viewer')),
|
|
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
last_login TEXT
|
|
);
|
|
```
|
|
|
|
---
|
|
|
|
### Table: `servers`
|
|
|
|
One row per managed server instance. **Game-agnostic.**
|
|
|
|
```sql
|
|
CREATE TABLE servers (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
name TEXT NOT NULL, -- display name in UI
|
|
description TEXT,
|
|
game_type TEXT NOT NULL DEFAULT 'arma3', -- adapter lookup key
|
|
status TEXT NOT NULL DEFAULT 'stopped',
|
|
CHECK (status IN ('stopped', 'starting', 'running', 'stopping', 'crashed', 'error')),
|
|
|
|
-- Process info
|
|
pid INTEGER, -- OS process ID when running
|
|
exe_path TEXT NOT NULL, -- path to server executable
|
|
started_at TEXT, -- ISO datetime
|
|
stopped_at TEXT,
|
|
|
|
-- Network (core ports; adapter defines derived port conventions)
|
|
game_port INTEGER NOT NULL,
|
|
rcon_port INTEGER, -- NULL if game has no remote admin
|
|
CHECK (game_port BETWEEN 1024 AND 65535),
|
|
CHECK (rcon_port IS NULL OR (rcon_port BETWEEN 1024 AND 65535)),
|
|
|
|
-- Auto-management
|
|
auto_restart INTEGER NOT NULL DEFAULT 0, -- 1 = restart on crash
|
|
max_restarts INTEGER NOT NULL DEFAULT 3, -- within restart_window_seconds
|
|
restart_window_seconds INTEGER NOT NULL DEFAULT 300,
|
|
restart_count INTEGER NOT NULL DEFAULT 0,
|
|
last_restart_at TEXT,
|
|
|
|
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
|
);
|
|
|
|
CREATE INDEX idx_servers_status ON servers(status);
|
|
CREATE INDEX idx_servers_game_type ON servers(game_type);
|
|
CREATE INDEX idx_servers_game_port ON servers(game_port);
|
|
```
|
|
|
|
**Key changes from single-game design:**
|
|
- Added `game_type` column (defaults to `'arma3'` for backward compatibility)
|
|
- Removed Arma 3-specific defaults (`DEFAULT 2302`, `DEFAULT 2306`) — port defaults are now adapter-provided
|
|
- Removed `steam_query_port` virtual column (Arma 3 convention moved to adapter)
|
|
- `rcon_port` is now nullable (some games have no remote admin)
|
|
|
|
---
|
|
|
|
### Table: `game_configs`
|
|
|
|
Stores all game-specific configuration as JSON blobs, keyed by section. **Replaces** the previous `server_configs`, `basic_configs`, `server_profiles`, `launch_params`, and `rcon_configs` tables.
|
|
|
|
```sql
|
|
CREATE TABLE game_configs (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
game_type TEXT NOT NULL, -- for validation; matches servers.game_type
|
|
section TEXT NOT NULL, -- e.g. 'server', 'basic', 'profile', 'launch', 'rcon'
|
|
config_json TEXT NOT NULL DEFAULT '{}', -- JSON validated by adapter's Pydantic model
|
|
config_version INTEGER NOT NULL DEFAULT 1, -- optimistic locking version; incremented on each write
|
|
schema_version TEXT NOT NULL DEFAULT '1.0', -- adapter schema version at time of last write
|
|
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
UNIQUE(server_id, section)
|
|
);
|
|
|
|
CREATE INDEX idx_game_configs_server ON game_configs(server_id);
|
|
CREATE INDEX idx_game_configs_type_section ON game_configs(game_type, section);
|
|
```
|
|
|
|
**How it works:**
|
|
- Each adapter defines config sections via `ConfigGenerator.get_sections()` → `{section_name: PydanticModelClass}`
|
|
- Core reads/writes JSON blobs; adapter validates them on write and parses them on read
|
|
- Arma 3 has 5 sections: `server`, `basic`, `profile`, `launch`, `rcon`
|
|
- Another game might have 2 sections or 8 — no migration needed
|
|
- Sensitive fields within JSON (passwords, rcon_password) are encrypted at the application layer via Fernet before storage
|
|
|
|
**Arma 3 config sections and their JSON structures:**
|
|
|
|
Section `server` — maps to `server.cfg` parameters:
|
|
```json
|
|
{
|
|
"hostname": "My Arma 3 Server",
|
|
"password": "encrypted:...",
|
|
"password_admin": "encrypted:...",
|
|
"server_command_password": "encrypted:...",
|
|
"max_players": 40,
|
|
"kick_duplicate": 1,
|
|
"persistent": 1,
|
|
"vote_threshold": 0.33,
|
|
"vote_mission_players": 1,
|
|
"vote_timeout": 60,
|
|
"role_timeout": 90,
|
|
"briefing_timeout": 60,
|
|
"debriefing_timeout": 45,
|
|
"lobby_idle_timeout": 300,
|
|
"disable_von": 0,
|
|
"von_codec": 1,
|
|
"von_codec_quality": 20,
|
|
"max_ping": 250,
|
|
"max_packet_loss": 50,
|
|
"max_desync": 200,
|
|
"disconnect_timeout": 15,
|
|
"kick_on_ping": 1,
|
|
"kick_on_packet_loss": 1,
|
|
"kick_on_desync": 1,
|
|
"kick_on_timeout": 1,
|
|
"battleye": 1,
|
|
"verify_signatures": 2,
|
|
"allowed_file_patching": 0,
|
|
"forced_difficulty": "Regular",
|
|
"timestamp_format": "short",
|
|
"auto_select_mission": 0,
|
|
"random_mission_order": 0,
|
|
"missions_to_restart": 0,
|
|
"missions_to_shutdown": 0,
|
|
"log_file": "server_console.log",
|
|
"skip_lobby": 0,
|
|
"drawing_in_map": 1,
|
|
"upnp": 0,
|
|
"loopback": 0,
|
|
"statistics_enabled": 1,
|
|
"force_rotor_lib": 0,
|
|
"required_build": 0,
|
|
"steam_protocol_max_data_size": 1024,
|
|
"motd_lines": ["Welcome!", "Have fun"],
|
|
"motd_interval": 5.0,
|
|
"on_user_connected": "",
|
|
"on_user_disconnected": "",
|
|
"on_unsigned_data": "kick (_this select 0)",
|
|
"on_hacked_data": "kick (_this select 0)",
|
|
"double_id_detected": "",
|
|
"headless_clients": [],
|
|
"local_clients": [],
|
|
"admin_uids": [],
|
|
"allowed_load_extensions": ["hpp","sqs","sqf","fsm","cpp","paa","txt","xml","inc","ext","sqm","ods","fxy","lip","csv","kb","bik","bikb","html","htm","biedi"],
|
|
"allowed_preprocess_extensions": ["hpp","sqs","sqf","fsm","cpp","paa","txt","xml","inc","ext","sqm","ods","fxy","lip","csv","kb","bik","bikb","html","htm","biedi"],
|
|
"allowed_html_extensions": ["htm","html","xml","txt"]
|
|
}
|
|
```
|
|
|
|
Section `basic` — maps to `basic.cfg` parameters:
|
|
```json
|
|
{
|
|
"min_bandwidth": 800000,
|
|
"max_bandwidth": 25000000,
|
|
"max_msg_send": 384,
|
|
"max_size_guaranteed": 512,
|
|
"max_size_non_guaranteed": 256,
|
|
"min_error_to_send": 0.003,
|
|
"max_custom_file_size": 100000
|
|
}
|
|
```
|
|
|
|
Section `profile` — maps to `server.Arma3Profile` difficulty:
|
|
```json
|
|
{
|
|
"reduced_damage": 0,
|
|
"group_indicators": 0,
|
|
"friendly_tags": 0,
|
|
"enemy_tags": 0,
|
|
"detected_mines": 0,
|
|
"commands": 1,
|
|
"waypoints": 1,
|
|
"tactical_ping": 0,
|
|
"weapon_info": 2,
|
|
"stance_indicator": 2,
|
|
"stamina_bar": 0,
|
|
"weapon_crosshair": 0,
|
|
"vision_aid": 0,
|
|
"third_person_view": 0,
|
|
"camera_shake": 1,
|
|
"score_table": 1,
|
|
"death_messages": 1,
|
|
"von_id": 1,
|
|
"map_content_friendly": 0,
|
|
"map_content_enemy": 0,
|
|
"map_content_mines": 0,
|
|
"auto_report": 0,
|
|
"multiple_saves": 0,
|
|
"ai_level_preset": 3,
|
|
"skill_ai": 0.5,
|
|
"precision_ai": 0.5
|
|
}
|
|
```
|
|
|
|
Section `launch` — maps to CLI launch parameters:
|
|
```json
|
|
{
|
|
"world": "empty",
|
|
"extra_params": "",
|
|
"limit_fps": 50,
|
|
"auto_init": 0,
|
|
"load_mission_to_memory": 0,
|
|
"bandwidth_alg": null,
|
|
"enable_ht": 0,
|
|
"huge_pages": 0,
|
|
"cpu_count": null,
|
|
"ex_threads": 7,
|
|
"max_mem": null,
|
|
"no_logs": 0,
|
|
"netlog": 0
|
|
}
|
|
```
|
|
|
|
Section `rcon` — BattlEye RCon settings:
|
|
```json
|
|
{
|
|
"rcon_password": "encrypted:...",
|
|
"max_ping": 200,
|
|
"enabled": 1
|
|
}
|
|
```
|
|
|
|
---
|
|
|
|
### Table: `mods`
|
|
|
|
Registered mods. Many-to-many with servers. Scoped by `game_type`.
|
|
|
|
```sql
|
|
CREATE TABLE mods (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
game_type TEXT NOT NULL, -- scope mods by game type
|
|
name TEXT NOT NULL,
|
|
folder_path TEXT NOT NULL,
|
|
workshop_id TEXT, -- Steam Workshop ID if applicable
|
|
description TEXT,
|
|
game_data TEXT DEFAULT '{}', -- JSON for game-specific mod metadata
|
|
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
UNIQUE (game_type, folder_path)
|
|
);
|
|
|
|
CREATE TABLE server_mods (
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
mod_id INTEGER NOT NULL REFERENCES mods(id) ON DELETE CASCADE,
|
|
is_server_mod INTEGER NOT NULL DEFAULT 0, -- 1 = server-side only (not broadcast to clients)
|
|
sort_order INTEGER NOT NULL DEFAULT 0,
|
|
game_data TEXT DEFAULT '{}', -- JSON for per-server mod overrides
|
|
PRIMARY KEY (server_id, mod_id)
|
|
);
|
|
|
|
CREATE INDEX idx_server_mods_server ON server_mods(server_id);
|
|
```
|
|
|
|
**Key changes:** Added `game_type` to `mods` (scopes the mod registry per game). Added `game_data` JSON columns for game-specific metadata that doesn't fit the common schema.
|
|
|
|
---
|
|
|
|
### Table: `missions`
|
|
|
|
Mission/scenario files tracked per server.
|
|
|
|
```sql
|
|
CREATE TABLE missions (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
filename TEXT NOT NULL, -- e.g. "MyMission.Altis.pbo"
|
|
mission_name TEXT NOT NULL, -- parsed by adapter
|
|
terrain TEXT, -- may be NULL for non-Arma games
|
|
file_size INTEGER, -- bytes
|
|
game_data TEXT DEFAULT '{}', -- JSON for game-specific mission metadata
|
|
uploaded_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
UNIQUE (server_id, filename)
|
|
);
|
|
|
|
CREATE INDEX idx_missions_server ON missions(server_id);
|
|
```
|
|
|
|
**Key changes:** `terrain` is now nullable (not all games use the mission/terrain naming convention). Added `game_data` for adapter-specific metadata.
|
|
|
|
---
|
|
|
|
### Table: `mission_rotation`
|
|
|
|
Ordered mission/scenario cycle for a server.
|
|
|
|
```sql
|
|
CREATE TABLE mission_rotation (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
mission_id INTEGER NOT NULL REFERENCES missions(id) ON DELETE CASCADE,
|
|
sort_order INTEGER NOT NULL DEFAULT 0,
|
|
difficulty TEXT, -- game-specific (NULL for games without difficulty)
|
|
params_json TEXT NOT NULL DEFAULT '{}', -- mission params as JSON
|
|
game_data TEXT DEFAULT '{}', -- adapter-specific rotation metadata
|
|
UNIQUE (server_id, sort_order)
|
|
);
|
|
|
|
CREATE INDEX idx_mission_rotation_server ON mission_rotation(server_id);
|
|
```
|
|
|
|
**Key changes:** `difficulty` is now nullable. Added `game_data` for adapter-specific rotation config.
|
|
|
|
---
|
|
|
|
### Table: `players`
|
|
|
|
Currently connected players (live state, refreshed by RemoteAdminPollerThread).
|
|
|
|
```sql
|
|
CREATE TABLE players (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
slot_id TEXT NOT NULL, -- Game-specific slot identifier (was player_num int)
|
|
name TEXT NOT NULL,
|
|
guid TEXT, -- Game-specific identifier (BattlEye GUID, Steam ID, etc.)
|
|
ip TEXT,
|
|
ping INTEGER,
|
|
game_data TEXT DEFAULT '{}', -- JSON: {verified, steam_uid, etc.} for Arma 3
|
|
joined_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
UNIQUE (server_id, slot_id)
|
|
);
|
|
|
|
CREATE INDEX idx_players_server ON players(server_id);
|
|
```
|
|
|
|
**Key changes:** `player_num` (INTEGER) renamed to `slot_id` (TEXT) for flexibility across games. `steam_uid` moved into `game_data` JSON (Arma 3 specific). `verified` moved into `game_data` JSON.
|
|
|
|
---
|
|
|
|
### Table: `player_history`
|
|
|
|
Historical record of connections. Inserted when player disconnects.
|
|
|
|
```sql
|
|
CREATE TABLE player_history (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
name TEXT NOT NULL,
|
|
guid TEXT,
|
|
ip TEXT,
|
|
game_data TEXT DEFAULT '{}', -- JSON for game-specific historical data
|
|
joined_at TEXT NOT NULL,
|
|
left_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
session_duration_seconds INTEGER
|
|
);
|
|
|
|
CREATE INDEX idx_player_history_server ON player_history(server_id);
|
|
CREATE INDEX idx_player_history_guid ON player_history(guid);
|
|
```
|
|
|
|
### Player History Retention Cleanup (run daily via APScheduler, keep 90 days)
|
|
```sql
|
|
DELETE FROM player_history
|
|
WHERE left_at < datetime('now', '-90 days');
|
|
```
|
|
|
|
---
|
|
|
|
### Table: `bans`
|
|
|
|
Ban records. Core concept is game-agnostic; ban file sync is adapter-specific.
|
|
|
|
```sql
|
|
CREATE TABLE bans (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
guid TEXT,
|
|
name TEXT,
|
|
reason TEXT,
|
|
banned_by TEXT, -- admin username
|
|
banned_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
expires_at TEXT, -- NULL = permanent
|
|
is_active INTEGER NOT NULL DEFAULT 1,
|
|
game_data TEXT DEFAULT '{}', -- JSON: {steam_uid, ip, etc.}
|
|
CHECK (is_active IN (0, 1))
|
|
);
|
|
|
|
CREATE INDEX idx_bans_server ON bans(server_id);
|
|
CREATE INDEX idx_bans_guid ON bans(guid);
|
|
CREATE INDEX idx_bans_active ON bans(is_active);
|
|
```
|
|
|
|
**Key changes:** Removed `steam_uid` as a dedicated column (moved to `game_data`). Added `game_data` JSON. Ban file sync (e.g., `battleye/ban.txt`) is handled by the adapter's `BanManager`.
|
|
|
|
---
|
|
|
|
### Table: `logs`
|
|
|
|
Parsed log lines (rolling retention, default 7 days).
|
|
|
|
```sql
|
|
CREATE TABLE logs (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
timestamp TEXT NOT NULL,
|
|
level TEXT NOT NULL DEFAULT 'info', -- 'info' | 'warning' | 'error'
|
|
CHECK (level IN ('info', 'warning', 'error')),
|
|
message TEXT NOT NULL,
|
|
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
|
);
|
|
|
|
CREATE INDEX idx_logs_server_ts ON logs(server_id, timestamp);
|
|
CREATE INDEX idx_logs_level ON logs(level);
|
|
CREATE INDEX idx_logs_created ON logs(created_at);
|
|
```
|
|
|
|
**No changes** — this table is fully game-agnostic. The adapter's `LogParser` converts game-specific log format into the standard `{timestamp, level, message}` tuple.
|
|
|
|
---
|
|
|
|
### Table: `metrics`
|
|
|
|
Time-series CPU/RAM/player count snapshots.
|
|
|
|
```sql
|
|
CREATE TABLE metrics (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
timestamp TEXT NOT NULL DEFAULT (datetime('now')),
|
|
cpu_percent REAL,
|
|
ram_mb REAL,
|
|
player_count INTEGER
|
|
);
|
|
|
|
CREATE INDEX idx_metrics_server_ts ON metrics(server_id, timestamp);
|
|
```
|
|
|
|
**No changes** — fully game-agnostic.
|
|
|
|
---
|
|
|
|
### Table: `server_events`
|
|
|
|
Audit trail of all significant events.
|
|
|
|
```sql
|
|
CREATE TABLE server_events (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
event_type TEXT NOT NULL,
|
|
-- Core event types:
|
|
-- 'started' | 'stopped' | 'crashed' | 'restarted' | 'config_updated'
|
|
-- 'player_kicked' | 'player_banned' | 'admin_login'
|
|
-- 'auto_restarted' | 'max_restarts_exceeded'
|
|
-- Adapters may define additional event types
|
|
actor TEXT, -- username or 'system'
|
|
detail TEXT, -- JSON with event-specific data
|
|
created_at TEXT NOT NULL DEFAULT (datetime('now'))
|
|
);
|
|
|
|
CREATE INDEX idx_events_server ON server_events(server_id, created_at);
|
|
```
|
|
|
|
**No changes** — fully game-agnostic. Adapters can emit additional event types.
|
|
|
|
---
|
|
|
|
## Relationships Diagram
|
|
|
|
```
|
|
users (1) ──────────────────────────────────── (many) server_events.actor
|
|
|
|
servers (1) ──┬── (many) game_configs ← JSON sections replace 5 Arma 3 tables
|
|
├── (many) server_mods ──── (many) mods (scoped by game_type)
|
|
├── (many) missions
|
|
├── (many) mission_rotation → missions
|
|
├── (many) players
|
|
├── (many) player_history
|
|
├── (many) bans
|
|
├── (many) logs
|
|
├── (many) metrics
|
|
└── (many) server_events
|
|
```
|
|
|
|
---
|
|
|
|
## Encryption Strategy
|
|
|
|
Sensitive fields stored in `game_configs.config_json` are encrypted at the application layer before JSON serialization. The adapter declares which fields are sensitive via `ConfigGenerator.get_sensitive_fields(section) -> list[str]`.
|
|
|
|
**How it works:**
|
|
1. On write: Core calls `adapter.get_config_generator().get_sensitive_fields(section)` to get a list of JSON keys
|
|
2. ConfigRepository replaces those values with `"encrypted:" + Fernet.encrypt(value)` tokens
|
|
3. On read: ConfigRepository detects `"encrypted:"` prefix and decrypts
|
|
4. The adapter's Pydantic model sees plaintext — encryption is transparent to the adapter
|
|
|
|
**Arma 3 encrypted fields (declared by Arma3ConfigGenerator):**
|
|
- Section `server`: `password`, `password_admin`, `server_command_password`
|
|
- Section `rcon`: `rcon_password`
|
|
|
|
Encryption uses Fernet (AES-256) with `LANGUARD_ENCRYPTION_KEY`.
|
|
|
|
## Optimistic Locking for Config Updates
|
|
|
|
The `config_version` column in `game_configs` prevents lost updates when two admins edit simultaneously:
|
|
|
|
1. Client reads config section → gets `config_version: 3`
|
|
2. Client sends PUT with `config_version: 3` in request body
|
|
3. Server checks: `WHERE server_id = ? AND section = ? AND config_version = 3`
|
|
4. If match: increment version, write new JSON, return 200
|
|
5. If no match (version changed): return **409 Conflict** with current config for client-side merge
|
|
|
|
## game_data JSON Schema
|
|
|
|
The `game_data` columns on `players`, `missions`, `mods`, and `bans` are validated by adapter-provided Pydantic models. Each capability protocol optionally provides a schema:
|
|
|
|
| Table | game_data column | Protocol Method | Arma 3 Example |
|
|
|-------|-----------------|-----------------|-----------------|
|
|
| `players` | `game_data` | `RemoteAdmin.get_player_data_schema()` | `{verified: bool, steam_uid: str}` |
|
|
| `missions` | `game_data` | `MissionManager.get_mission_data_schema()` | `{terrain: str}` |
|
|
| `mods` | `game_data` | `ModManager.get_mod_data_schema()` | `{}` (empty for Arma 3) |
|
|
| `bans` | `game_data` | `BanManager.get_ban_data_schema()` | `{steam_uid: str, ip: str}` |
|
|
|
|
If an adapter doesn't define a game_data schema, the field accepts any JSON object (no validation).
|
|
|
|
---
|
|
|
|
## Maintenance Queries
|
|
|
|
### Log Retention Cleanup (run daily via APScheduler)
|
|
```sql
|
|
DELETE FROM logs
|
|
WHERE created_at < datetime('now', '-7 days');
|
|
```
|
|
|
|
### Metrics Retention Cleanup (keep 30 days)
|
|
```sql
|
|
DELETE FROM metrics
|
|
WHERE timestamp < datetime('now', '-30 days');
|
|
```
|
|
|
|
### Clear disconnected players on server stop
|
|
```sql
|
|
DELETE FROM players WHERE server_id = ?;
|
|
```
|
|
|
|
### Vacuum (run weekly)
|
|
```sql
|
|
VACUUM;
|
|
```
|
|
|
|
---
|
|
|
|
## Migration Strategy
|
|
|
|
- Migrations are plain `.sql` files in `backend/core/migrations/`
|
|
- Naming: `001_initial_schema.sql`, `002_add_game_type.sql`, etc.
|
|
- Tracked in a `schema_migrations` table:
|
|
```sql
|
|
CREATE TABLE schema_migrations (
|
|
version INTEGER PRIMARY KEY,
|
|
applied_at TEXT NOT NULL DEFAULT (datetime('now'))
|
|
);
|
|
```
|
|
- Applied automatically at app startup by `database.py:run_migrations()`
|
|
- **Adapter-specific migrations** go in `adapters/<game_type>/migrations/` and are applied by the adapter's initialization (if the adapter needs extra tables beyond `game_configs`)
|
|
|
|
---
|
|
|
|
## Migration from Single-Game Schema
|
|
|
|
For existing deployments with the old Arma 3-specific schema:
|
|
|
|
### SQL Migration (002_multi_game_adapter.sql)
|
|
|
|
```sql
|
|
-- 1. Add game_type to servers
|
|
ALTER TABLE servers ADD COLUMN game_type TEXT NOT NULL DEFAULT 'arma3';
|
|
|
|
-- 2. Create game_configs table
|
|
CREATE TABLE game_configs (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
|
game_type TEXT NOT NULL,
|
|
section TEXT NOT NULL,
|
|
config_json TEXT NOT NULL DEFAULT '{}',
|
|
config_version INTEGER NOT NULL DEFAULT 1,
|
|
schema_version TEXT NOT NULL DEFAULT '1.0',
|
|
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
UNIQUE(server_id, section)
|
|
);
|
|
|
|
-- 3. Add game_data columns
|
|
ALTER TABLE players ADD COLUMN game_data TEXT DEFAULT '{}';
|
|
ALTER TABLE missions ADD COLUMN game_data TEXT DEFAULT '{}';
|
|
ALTER TABLE mods ADD COLUMN game_type TEXT NOT NULL DEFAULT 'arma3';
|
|
ALTER TABLE mods ADD COLUMN game_data TEXT DEFAULT '{}';
|
|
ALTER TABLE server_mods ADD COLUMN game_data TEXT DEFAULT '{}';
|
|
ALTER TABLE bans ADD COLUMN game_data TEXT DEFAULT '{}';
|
|
ALTER TABLE mission_rotation ADD COLUMN game_data TEXT DEFAULT '{}';
|
|
|
|
-- 4. Migrate player_num → slot_id (SQLite 3.25.0+)
|
|
ALTER TABLE players RENAME COLUMN player_num TO slot_id;
|
|
-- Change slot_id type: SQLite doesn't support ALTER COLUMN TYPE.
|
|
-- Workaround: create new table, copy data, drop old, rename.
|
|
-- See Python migration script below.
|
|
|
|
-- 5. Drop old Arma 3-specific tables (after Python migration confirms data copied)
|
|
-- DROP TABLE server_configs;
|
|
-- DROP TABLE basic_configs;
|
|
-- DROP TABLE server_profiles;
|
|
-- DROP TABLE launch_params;
|
|
-- DROP TABLE rcon_configs;
|
|
```
|
|
|
|
### Python Migration Script (002_migrate_arma3_config.py)
|
|
|
|
This script reads old Arma 3-specific tables, serializes each row to JSON, and inserts into `game_configs`. It runs AFTER the SQL migration creates the new table structure.
|
|
|
|
```python
|
|
"""Migrate Arma 3 config data from normalized tables to game_configs JSON.
|
|
|
|
Run after 002_multi_game_adapter.sql. This script:
|
|
1. Reads each old table row by row
|
|
2. Converts column values to JSON (handling type conversions)
|
|
3. Inserts into game_configs with proper section names
|
|
4. Migrates player data (player_num int → slot_id text, steam_uid → game_data)
|
|
5. Verifies row counts match before dropping old tables
|
|
|
|
Type conversion rules:
|
|
- INTEGER 0/1 → JSON boolean false/true (for boolean-like fields)
|
|
- TEXT containing JSON arrays (motd_lines, headless_clients, etc.) → parsed JSON arrays
|
|
- NULL values → omitted from JSON (Pydantic fills defaults on read)
|
|
- Encrypted fields (password, password_admin, rcon_password) → copied as-is (already encrypted)
|
|
|
|
Transaction: all inserts in a single transaction. On failure, rollback — old tables untouched.
|
|
"""
|
|
|
|
import json
|
|
import sqlite3
|
|
|
|
COLUMN_TYPE_MAP = {
|
|
# server_configs → section 'server'
|
|
'server_configs': {
|
|
'section': 'server',
|
|
'boolean_fields': [
|
|
'kick_duplicate', 'persistent', 'disable_von', 'battleye',
|
|
'kick_on_ping', 'kick_on_packet_loss', 'kick_on_desync', 'kick_on_timeout',
|
|
'auto_select_mission', 'random_mission_order', 'skip_lobby',
|
|
'drawing_in_map', 'upnp', 'loopback', 'statistics_enabled',
|
|
],
|
|
'json_array_fields': [
|
|
'motd_lines', 'headless_clients', 'local_clients', 'admin_uids',
|
|
'allowed_load_extensions', 'allowed_preprocess_extensions',
|
|
'allowed_html_extensions',
|
|
],
|
|
'encrypted_fields': ['password', 'password_admin', 'server_command_password'],
|
|
},
|
|
# basic_configs → section 'basic'
|
|
'basic_configs': {
|
|
'section': 'basic',
|
|
'boolean_fields': [],
|
|
'json_array_fields': [],
|
|
'encrypted_fields': [],
|
|
},
|
|
# server_profiles → section 'profile'
|
|
'server_profiles': {
|
|
'section': 'profile',
|
|
'boolean_fields': [
|
|
'reduced_damage', 'camera_shake', 'score_table', 'death_messages',
|
|
'von_id', 'auto_report', 'multiple_saves', 'stamina_bar',
|
|
'weapon_crosshair', 'vision_aid', 'third_person_view',
|
|
'map_content_friendly', 'map_content_enemy', 'map_content_mines',
|
|
],
|
|
'json_array_fields': [],
|
|
'encrypted_fields': [],
|
|
},
|
|
# launch_params → section 'launch'
|
|
'launch_params': {
|
|
'section': 'launch',
|
|
'boolean_fields': [
|
|
'auto_init', 'load_mission_to_memory', 'enable_ht',
|
|
'huge_pages', 'no_logs', 'netlog',
|
|
],
|
|
'json_array_fields': [],
|
|
'encrypted_fields': [],
|
|
},
|
|
# rcon_configs → section 'rcon'
|
|
'rcon_configs': {
|
|
'section': 'rcon',
|
|
'boolean_fields': ['enabled'],
|
|
'json_array_fields': [],
|
|
'encrypted_fields': ['rcon_password'],
|
|
},
|
|
}
|
|
|
|
def migrate_config_table(db: sqlite3.Connection, table: str, mapping: dict):
|
|
"""Read rows from old table, serialize to JSON, insert into game_configs."""
|
|
rows = db.execute(f"SELECT * FROM {table}").fetchall()
|
|
columns = [desc[0] for desc in db.execute(f"SELECT * FROM {table} LIMIT 0").description]
|
|
|
|
for row in rows:
|
|
row_dict = dict(zip(columns, row))
|
|
server_id = row_dict.pop('id', None) or row_dict.pop('server_id', None)
|
|
|
|
config = {}
|
|
for col, value in row_dict.items():
|
|
if value is None:
|
|
continue # omit NULLs; Pydantic fills defaults
|
|
if col in mapping.get('boolean_fields', []):
|
|
config[col] = bool(value)
|
|
elif col in mapping.get('json_array_fields', []):
|
|
config[col] = json.loads(value) if isinstance(value, str) else value
|
|
elif col in mapping.get('encrypted_fields', []):
|
|
config[col] = value # already encrypted, copy as-is
|
|
elif col in ('updated_at', 'created_at', 'id', 'server_id'):
|
|
continue # skip metadata columns
|
|
else:
|
|
config[col] = value
|
|
|
|
db.execute("""
|
|
INSERT INTO game_configs (server_id, game_type, section, config_json, schema_version)
|
|
VALUES (?, 'arma3', ?, ?, '1.0')
|
|
""", (server_id, mapping['section'], json.dumps(config)))
|
|
|
|
|
|
def migrate_player_data(db: sqlite3.Connection):
|
|
"""Convert player_num (int) to slot_id (text), move steam_uid to game_data."""
|
|
# Create new players table with slot_id as TEXT
|
|
db.execute("""
|
|
CREATE TABLE players_new (
|
|
id INTEGER PRIMARY KEY AUTOINCREMENT,
|
|
server_id INTEGER NOT NULL,
|
|
slot_id TEXT NOT NULL,
|
|
name TEXT NOT NULL,
|
|
guid TEXT,
|
|
ip TEXT,
|
|
ping INTEGER,
|
|
game_data TEXT DEFAULT '{}',
|
|
joined_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
|
|
UNIQUE (server_id, slot_id)
|
|
)
|
|
""")
|
|
rows = db.execute("SELECT * FROM players").fetchall()
|
|
columns = [desc[0] for desc in db.execute("SELECT * FROM players LIMIT 0").description]
|
|
for row in rows:
|
|
d = dict(zip(columns, row))
|
|
game_data = {}
|
|
if d.get('steam_uid'):
|
|
game_data['steam_uid'] = d['steam_uid']
|
|
if d.get('verified') is not None:
|
|
game_data['verified'] = bool(d['verified'])
|
|
db.execute("""
|
|
INSERT INTO players_new (id, server_id, slot_id, name, guid, ip, ping,
|
|
game_data, joined_at, updated_at)
|
|
VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
|
|
""", (
|
|
d['id'], d['server_id'], str(d['player_num']),
|
|
d['name'], d.get('guid'), d.get('ip'), d.get('ping'),
|
|
json.dumps(game_data), d.get('joined_at'), d.get('updated_at'),
|
|
))
|
|
db.execute("DROP TABLE players")
|
|
db.execute("ALTER TABLE players_new RENAME TO players")
|
|
db.execute("CREATE INDEX idx_players_server ON players(server_id)")
|
|
|
|
|
|
def run_migration(db_path: str):
|
|
db = sqlite3.connect(db_path)
|
|
try:
|
|
db.execute("BEGIN TRANSACTION")
|
|
for table, mapping in COLUMN_TYPE_MAP.items():
|
|
count_before = db.execute(f"SELECT COUNT(*) FROM {table}").fetchone()[0]
|
|
migrate_config_table(db, table, mapping)
|
|
count_after = db.execute(
|
|
"SELECT COUNT(*) FROM game_configs WHERE game_type='arma3' AND section=?",
|
|
(mapping['section'],)
|
|
).fetchone()[0]
|
|
assert count_after == count_before, (
|
|
f"Row count mismatch for {table}: {count_before} → {count_after}"
|
|
)
|
|
migrate_player_data(db)
|
|
db.execute("COMMIT")
|
|
print("Migration successful. Old tables can now be dropped.")
|
|
except Exception as e:
|
|
db.execute("ROLLBACK")
|
|
print(f"Migration FAILED (rolled back): {e}")
|
|
raise
|
|
finally:
|
|
db.close()
|
|
``` |