commit e02db3ddde0eb2e1c2fd200fb9533d54f1992c6b Author: Khoa (Revenovich) Tran Gia Date: Tue Apr 14 15:06:56 2026 +0700 feat: add Java→Python migration plan with 9 self-contained phase files Converts Spring Boot 4.0.3 ARMA Server Web GUI to FastAPI/Python. Each phase file is fully self-contained: lists Java source files to read, output files to create, implementation patterns, REST endpoint contracts, and a completion checklist. A future agent can execute any single phase without rescanning the Java project. Phases: - 01: Foundation — SQLAlchemy models, Alembic, settings, base schemas - 02: Auth & Users — JWT middleware, RBAC, user CRUD - 03: CFG parser + server process — server.cfg round-trip, start/stop - 04: Server settings — general/network/logging/security/difficulty - 05: Mod management — mod CRUD, presets, settings, WebSocket progress - 06: Steam integration — SteamCMD queue, Workshop API, python-a2s - 07: Missions, CDLC, Discord, APScheduler jobs - 08: Middleware & polish — global exception handler, SPA redirect, structlog - 09: Testing — pytest-asyncio, respx, 80% coverage target diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 0000000..699176f --- /dev/null +++ b/PLAN.md @@ -0,0 +1,133 @@ +# ASWG Python Conversion — Master Plan + +**Source project**: `E:\TestScript\ARMA-Server-Web-Gui` (Java 25 / Spring Boot 4.0.3) +**Target project**: `E:\TestScript\Arma_Server_Web_Manager` (Python / FastAPI) +**Goal**: Identical REST API so the Angular 21 frontend works without any changes. +**Server port**: 8085 (same as Java app) + +--- + +## Phase Status Tracker + +Update each phase's Status field when starting (`IN_PROGRESS`) and finishing (`DONE`). +Each phase file is self-contained — read only the phase file for the current phase. + +| Phase | File | Status | +|-------|------|--------| +| 1 — Skeleton + DB | `phases/phase-01-skeleton-db.md` | PENDING | +| 2 — Auth + Security | `phases/phase-02-auth-security.md` | PENDING | +| 3 — CFG Parser + Server Process | `phases/phase-03-cfg-parser-process.md` | PENDING | +| 4 — Server Settings Subdomains | `phases/phase-04-server-settings.md` | PENDING | +| 5 — Mod Management | `phases/phase-05-mod-management.md` | PENDING | +| 6 — Steam Integration | `phases/phase-06-steam-integration.md` | PENDING | +| 7 — Missions, CDLC, Discord, Jobs | `phases/phase-07-missions-cdlc-discord-jobs.md` | PENDING | +| 8 — Middleware + Polish | `phases/phase-08-middleware-polish.md` | PENDING | +| 9 — Testing | `phases/phase-09-testing.md` | PENDING | + +--- + +## Global Technology Stack + +```toml +# pyproject.toml dependencies +fastapi = ">=0.115" +uvicorn = {extras = ["standard"], version = ">=0.30"} +pydantic = ">=2.7" +pydantic-settings = ">=2.3" +sqlalchemy = ">=2.0" +alembic = ">=1.13" +aiosqlite = ">=0.20" +PyJWT = ">=2.8" +passlib = {extras = ["bcrypt"], version = ">=1.7"} +cryptography = ">=43" +httpx = ">=0.27" +psutil = ">=5.9" +apscheduler = ">=3.10" +cachetools = ">=5.3" +python-a2s = ">=1.7" +structlog = ">=24" +jproperties = ">=2.1" + +[dev] +pytest = ">=8.2" +pytest-asyncio = ">=0.23" +pytest-cov = ">=5.0" +respx = ">=0.21" +ruff = ">=0.4" +``` + +--- + +## Java Source Root Abbreviation + +All Java files are under: +`E:\TestScript\ARMA-Server-Web-Gui\src\main\java\pl\bartlomiejstepien\armaserverwebgui\` + +In phase files this is written as ``. + +--- + +## Database Schema (all 12 tables) + +```sql +installed_mod (id BIGINT PK, workshop_file_id BIGINT, name VARCHAR, directory_path VARCHAR, + preview_url VARCHAR, created_date TIMESTAMP, enabled BOOLEAN, server_mod BOOLEAN, + last_workshop_update_date TIMESTAMP, last_workshop_update_attempt_date TIMESTAMP, + dependencies_ids VARCHAR) +mod_preset (id BIGINT PK, name VARCHAR UNIQUE) +mod_preset_entry (id BIGINT PK, mod_preset_id BIGINT FK→mod_preset, mod_id BIGINT, mod_name VARCHAR) +difficulty_profile (id BIGINT PK, name VARCHAR UNIQUE, active BOOLEAN) +mission (id BIGINT PK, name VARCHAR UNIQUE, template VARCHAR, enabled BOOLEAN, difficulty VARCHAR) +mod_settings (id BIGINT PK, name VARCHAR, content TEXT) +invalid_jwt_token (id BIGINT PK, jwt VARCHAR UNIQUE, invalidated_datetime TIMESTAMP, + expiration_datetime TIMESTAMP) +aswg_user (id BIGINT PK, username VARCHAR UNIQUE, password VARCHAR, locked BOOLEAN, + created_datetime TIMESTAMP, last_success_login_datetime TIMESTAMP) +authority (id BIGINT PK, code VARCHAR UNIQUE) +aswg_user_authority (user_id BIGINT FK→aswg_user, authority_id BIGINT FK→authority) +cdlc (id BIGINT PK, name VARCHAR UNIQUE, enabled BOOLEAN) +job_execution (id BIGINT PK, job_name VARCHAR, start_date TIMESTAMP, finish_date TIMESTAMP, + status VARCHAR, message VARCHAR) +``` + +--- + +## API Contract Rules (CRITICAL — do not deviate) + +The Angular 21 frontend will NOT be changed. Every endpoint must match exactly: +- URL path, HTTP method, request field names (camelCase), response field names (camelCase) +- HTTP status codes: 200 OK, 201 Created, 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 409 Conflict, 500 Internal Server Error +- Error response shape: `{"code": "ERROR_CODE", "message": "Human readable message"}` + +Apply to all Pydantic schemas: +```python +from pydantic import ConfigDict +from pydantic.alias_generators import to_camel + +class BaseSchema(BaseModel): + model_config = ConfigDict(alias_generator=to_camel, populate_by_name=True) +``` + +--- + +## Global Java→Python Concept Map + +| Java / Spring | Python / FastAPI | +|---|---| +| `@RestController` + `@RequestMapping` | `APIRouter(prefix="/api/v1/...")` | +| `@GetMapping` / `@PostMapping` / `@DeleteMapping` / `@PutMapping` | `@router.get()` / `.post()` / `.delete()` / `.put()` | +| `@RequestBody` | Pydantic model as function parameter | +| `@PathVariable` | Path param `{name}` in route string | +| `@RequestParam` | `Query(...)` | +| `@RequestPart` multipart | `UploadFile` + `Form(...)` | +| `@HasPermission*` annotation | `Depends(has_permission(AswgAuthority.XYZ))` | +| `@Service` / `@Component` | Plain class, instantiated in `dependencies.py` | +| `@Transactional` | `async with session.begin():` | +| JPA `@Entity` | SQLAlchemy `Base` subclass | +| `Optional` | `T \| None` | +| Lombok `@Builder` | `@dataclass` or Pydantic with defaults | +| `@Value("${prop}")` | `settings.prop` (Pydantic BaseSettings) | +| `ResponseEntity` with status | Return value + `status_code=` on route | +| `SseEmitter` | `StreamingResponse(media_type="text/event-stream")` | +| Spring WebSocket | FastAPI `WebSocket` route parameter | +| `@ConditionalOnProperty` | `if settings.prop:` guard at startup/registration | diff --git a/phases/phase-01-skeleton-db.md b/phases/phase-01-skeleton-db.md new file mode 100644 index 0000000..4f88c77 --- /dev/null +++ b/phases/phase-01-skeleton-db.md @@ -0,0 +1,215 @@ +# Phase 1 — Project Skeleton + Database + +**Status**: PENDING +**Depends on**: nothing +**Next phase**: `phase-02-auth-security.md` + +--- + +## Goal + +Produce a runnable FastAPI app that starts, connects to SQLite, runs Alembic migrations, and responds to health checks. All ORM models and repositories must exist. No business logic yet. + +--- + +## Java Source Files to Read + +`` = `E:\TestScript\ARMA-Server-Web-Gui\src\main\java\pl\bartlomiejstepien\armaserverwebgui\` + +| File | What to extract | +|------|----------------| +| `application/config/ASWGConfig.java` | Every `@Value` field name and default | +| `ArmaServerWebGuiApplication.java` | Startup hooks | +| `DefaultConfigGenerator.java` | How default config file is written on first run | +| `domain/server/mod/model/InstalledModEntity.java` | JPA mapping, `ListOfLongsConverter` for dependencies_ids | +| `domain/server/mod/model/ModPresetEntity.java` | JPA + relations | +| `domain/server/mod/model/ModSettingsEntity.java` | JPA mapping | +| `domain/server/mission/model/MissionEntity.java` | JPA mapping | +| `domain/server/difficulty/model/DifficultyProfileEntity.java` | JPA mapping | +| `domain/server/cdlc/model/CdlcEntity.java` | JPA mapping | +| `application/security/jwt/model/InvalidJwtTokenEntity.java` | JPA mapping | +| `application/scheduling/model/JobExecutionEntity.java` | JPA mapping | +| `application/scheduling/model/JobExecutionStatus.java` | Enum values | +| `repository/` (all .java files) | Custom query methods beyond findById/findAll | +| `src/main/resources/application.properties` | server.port, multipart size limits, datasource | +| `src/main/resources/aswg-default-config.properties` | All aswg.* keys and defaults | +| `src/main/resources/db/changelog/` (all files) | Full DDL for all 14 migration sets | + +--- + +## Output Files to Create + +Relative to `E:\TestScript\Arma_Server_Web_Manager\`: + +``` +pyproject.toml +alembic.ini +alembic/env.py +alembic/versions/001_init.py +alembic/versions/002_installed_mod_last_workshop_update.py +alembic/versions/003_mission.py +alembic/versions/004_mod_settings.py +alembic/versions/005_invalid_jwt_token.py +alembic/versions/006_aswg_user.py +alembic/versions/007_overwrite_startup_params_authority.py +alembic/versions/008_cdlc.py +alembic/versions/009_aswg_user_last_login_column.py +alembic/versions/010_job_last_execution_time.py +alembic/versions/011_add_authorities.py +alembic/versions/012_last_mod_update_attempt.py +alembic/versions/013_job_execution_history.py +alembic/versions/014_mod_dependencies.py +config/settings.py +config/aswg_default_config.properties (copy content from Java resources file) +src/__init__.py +src/main.py +src/dependencies.py +src/repository/__init__.py +src/repository/base.py +src/repository/models.py +src/repository/installed_mod_repo.py +src/repository/mod_preset_repo.py +src/repository/mod_settings_repo.py +src/repository/mission_repo.py +src/repository/difficulty_profile_repo.py +src/repository/cdlc_repo.py +src/repository/user_repo.py +src/repository/authority_repo.py +src/repository/invalid_jwt_token_repo.py +src/repository/job_execution_repo.py +``` + +--- + +## Implementation Notes + +### config/settings.py +Use `pydantic-settings` `BaseSettings`. The properties file uses Java `.properties` format (key=value), so implement a custom `customise_sources` that reads via `jproperties.Properties`. + +Map every key in `aswg-default-config.properties` as a field with `alias="aswg.the-key-name"`. Key fields: +```python +server_directory_path: str = Field(default="arma-server", alias="aswg.server-directory-path") +security_enabled: bool = Field(default=True, alias="aswg.security.enabled") +jwt_expiration_time: str = Field(default="PT2H", alias="aswg.security.jwt.expiration-time") +default_user_username: str = Field(default="user", alias="aswg.default-user.username") +default_user_password: str = Field(default="changeme", alias="aswg.default-user.password") +steamcmd_path: str = Field(default="", alias="aswg.steamcmd.path") +``` + +### src/repository/base.py +```python +from sqlalchemy.ext.asyncio import create_async_engine, async_sessionmaker +from sqlalchemy.orm import DeclarativeBase + +DATABASE_URL = "sqlite+aiosqlite:///./data/aswg.db" +engine = create_async_engine(DATABASE_URL, echo=False) +SessionLocal = async_sessionmaker(engine, expire_on_commit=False) + +class Base(DeclarativeBase): + pass +``` + +### src/repository/models.py — Key mapping notes +- `InstalledModEntity.dependencies_ids`: Java stores as comma-separated long string. Create a SQLAlchemy `TypeDecorator` that serializes `list[int]` ↔ `"1,2,3"`. +- `@GeneratedValue(IDENTITY)` → `Integer, primary_key=True, autoincrement=True` +- `LocalDateTime` / `OffsetDateTime` → `DateTime(timezone=True)` +- `@ManyToMany` user↔authority → explicit association table `aswg_user_authority` +- `@Enumerated(STRING)` → store as `String`, convert in application layer + +### Alembic migrations +Replace H2-specific syntax: +- `IDENTITY` / `BIGINT GENERATED BY DEFAULT AS IDENTITY` → `INTEGER PRIMARY KEY AUTOINCREMENT` +- `TIMESTAMP` → `DATETIME` +- No sequences needed + +Use `alembic/env.py` async pattern: +```python +import asyncio +from alembic import context +from src.repository.base import engine, Base + +def run_migrations_online(): + async def _run(): + async with engine.begin() as conn: + await conn.run_sync(context.run_migrations) + asyncio.run(_run()) +``` + +### src/main.py — minimal skeleton +```python +from contextlib import asynccontextmanager +from fastapi import FastAPI +from fastapi.staticfiles import StaticFiles +from alembic.config import Config +from alembic import command + +@asynccontextmanager +async def lifespan(app: FastAPI): + Config("alembic.ini") + command.upgrade(Config("alembic.ini"), "head") + # Phase 2 will add: create default user + # Phase 6 will add: start scheduler + yield + # Phase 6 will add: stop scheduler + +app = FastAPI(lifespan=lifespan) + +@app.get("/api/v1/actuator/health") +async def health(): + return {"status": "UP"} + +@app.get("/api/v1/actuator/info") +async def info(): + return {"application": {"name": "ASWG"}} + +# Mount Angular SPA — must be last +app.mount("/", StaticFiles(directory="static", html=True), name="static") +``` + +### src/dependencies.py +```python +from typing import Annotated +from fastapi import Depends +from sqlalchemy.ext.asyncio import AsyncSession +from src.repository.base import SessionLocal + +async def get_db(): + async with SessionLocal() as session: + yield session + +DbSession = Annotated[AsyncSession, Depends(get_db)] +``` + +### Repository pattern +Plain async classes receiving `AsyncSession`. Port every Spring Data method name: +- `findAll()` → `async def find_all()` +- `findById(id)` → `async def find_by_id(id)` +- `save(entity)` → `async def save(entity)` (uses `session.merge()`) +- `deleteById(id)` → `async def delete_by_id(id)` +- Custom `@Query` methods → SQLAlchemy `select()` with `.where()` + +--- + +## Completion Checklist + +- [ ] `pyproject.toml` with all dependencies from `PLAN.md` +- [ ] `config/settings.py` covers every `aswg.*` property with correct defaults +- [ ] `src/repository/base.py` creates async SQLite engine +- [ ] `src/repository/models.py` has all 12 ORM entity classes +- [ ] All 14 Alembic migrations run cleanly (`alembic upgrade head`) +- [ ] All 10 repository classes exist with full method set +- [ ] `uvicorn src.main:app --port 8085` starts without errors +- [ ] `GET /api/v1/actuator/health` → `{"status":"UP"}` +- [ ] `GET /api/v1/actuator/info` → returns JSON + +## Contract for Phase 2 + +Phase 2 will import: +- `from config.settings import settings` +- `from src.dependencies import DbSession` +- `from src.repository.base import Base, engine, SessionLocal` +- `from src.repository.models import AswgUserEntity, AuthorityEntity, InvalidJwtTokenEntity` +- `from src.repository.user_repo import UserRepository` +- `from src.repository.authority_repo import AuthorityRepository` +- `from src.repository.invalid_jwt_token_repo import InvalidJwtTokenRepository` +- `from src.main import app` diff --git a/phases/phase-02-auth-security.md b/phases/phase-02-auth-security.md new file mode 100644 index 0000000..64b4ee3 --- /dev/null +++ b/phases/phase-02-auth-security.md @@ -0,0 +1,198 @@ +# Phase 2 — Authentication + Security + +**Status**: PENDING +**Depends on**: Phase 1 complete +**Next phase**: `phase-03-cfg-parser-process.md` + +--- + +## Goal + +Working JWT login/logout, BCrypt password verification, the full `AswgAuthority` enum (37 permissions), JWT middleware enforcing auth on all `/api/**` routes, user CRUD service, and the auth + user REST endpoints. + +After this phase: `POST /api/v1/auth` returns a JWT token; all other `/api/**` endpoints return 401 without a valid token. + +--- + +## Java Source Files to Read + +`` = `E:\TestScript\ARMA-Server-Web-Gui\src\main\java\pl\bartlomiejstepien\armaserverwebgui\` + +| File | What to extract | +|------|----------------| +| `application/auth/AuthService.java` | authenticate(), logout() logic | +| `application/auth/JwtToken.java` | JWT payload fields | +| `application/security/AswgAuthority.java` | **All 37 authority code strings** | +| `application/security/jwt/JwtService.java` | createJwt(), validateJwt(), blacklisting logic | +| `application/security/jwt/filter/JwtFilter.java` | Permit-all path list | +| `application/security/jwt/cleaner/InvalidJwtCleaner.java` | Scheduled cleanup logic | +| `application/security/AuthenticationFacade.java` | getCurrentUser() pattern | +| `application/security/AswgAuthenticationEntryPoint.java` | 401 response shape | +| `application/config/SecurityConfig.java` | Dual enabled/disabled modes, permit-all paths | +| `domain/user/UserService.java` | All method signatures | +| `domain/user/UserServiceImpl.java` | Full implementation | +| `domain/user/dto/` (all files) | DTO field names | +| `web/AuthRestController.java` | Exact paths, request/response JSON | +| `web/UserRestController.java` | Exact paths, request/response JSON | +| `web/request/PasswordChangeRequest.java` | Fields | + +--- + +## Output Files to Create + +``` +src/application/__init__.py +src/application/security/__init__.py +src/application/security/jwt_service.py +src/application/security/auth_service.py +src/application/security/permissions.py (AswgAuthority enum + has_permission factory) +src/application/security/authentication.py (get_current_user dependency) +src/application/security/password.py (BCrypt wrapper via passlib) +src/application/middleware/__init__.py +src/application/middleware/jwt_middleware.py +src/domain/__init__.py +src/domain/user/__init__.py +src/domain/user/user_service.py +src/domain/user/user_loader_service.py +src/domain/user/user_session_service.py +src/domain/user/models.py +src/web/__init__.py +src/web/schemas/__init__.py +src/web/schemas/common.py (BaseSchema with camelCase alias, RestErrorResponse) +src/web/schemas/auth.py +src/web/schemas/users.py +src/web/auth_router.py +src/web/user_router.py +``` + +Also update `src/main.py`: +- Register `auth_router` and `user_router` +- Add `JwtMiddleware` +- Create default user on startup in `lifespan` + +--- + +## Implementation Notes + +### src/web/schemas/common.py — BaseSchema (used by ALL schemas in all phases) + +```python +from pydantic import BaseModel, ConfigDict +from pydantic.alias_generators import to_camel + +class BaseSchema(BaseModel): + model_config = ConfigDict(alias_generator=to_camel, populate_by_name=True) + +class RestErrorResponse(BaseSchema): + code: str + message: str +``` + +Every schema in every phase must inherit `BaseSchema` to preserve camelCase JSON for the Angular frontend. + +### src/application/security/permissions.py + +Extract all 37 codes from `AswgAuthority.java`. Example: +```python +from enum import StrEnum +from fastapi import Depends, HTTPException + +class AswgAuthority(StrEnum): + SERVER_START_STOP = "SERVER_START_STOP" + GENERAL_SETTINGS_VIEW = "GENERAL_SETTINGS_VIEW" + # ... all 37 values + +def has_permission(authority: AswgAuthority): + async def _check(current_user=Depends(get_current_user)): + user_codes = {a.code for a in current_user.authorities} + if authority.value not in user_codes: + raise HTTPException(status_code=403) + return current_user + return Depends(_check) +``` + +Usage: `@router.get("/properties", dependencies=[has_permission(AswgAuthority.GENERAL_SETTINGS_VIEW)])` + +### src/application/security/jwt_service.py + +Algorithm: `HS256`. JWT payload fields (from `JwtToken.java`): `sub` (username), `authorities` (list of code strings), `iss`, `iat`, `exp`. + +Blacklisting: on logout, insert token into `invalid_jwt_token` table. On every validation, check the table. Scheduled cleanup (`InvalidJwtCleaner` equivalent): delete rows where `expiration_datetime < now()`. + +### src/application/middleware/jwt_middleware.py + +Permit-all paths from `SecurityConfig.java` / `JwtFilter.java`: +```python +PERMIT_ALL_EXACT = { + ("POST", "/api/v1/auth"), + ("POST", "/api/v1/auth/logout"), + ("GET", "/api/v1/actuator/health"), + ("GET", "/api/v1/actuator/info"), +} +PERMIT_ALL_PREFIX = ["/api/v1/ws/", "/api/v1/logging/logs-sse"] +``` + +When `settings.security_enabled = False`: skip all JWT checks, attach a superuser principal to `request.state.user`. + +On JWT failure return JSON matching `AswgAuthenticationEntryPoint`: +- Expired token: `{"code":"AUTH_TOKEN_EXPIRED","message":"..."}` HTTP 401 +- Missing token: `{"code":"AUTH_TOKEN_REQUIRED","message":"..."}` HTTP 401 +- Bad token: `{"code":"BAD_AUTH_TOKEN","message":"..."}` HTTP 401 + +### Auth REST endpoints (from `AuthRestController.java`) + +``` +POST /api/v1/auth body: {username, password} → {token: "..."} 200 +GET /api/v1/auth/myself (authenticated) → UserProfileResponse 200 +POST /api/v1/auth/logout (authenticated) → 200 (no body) +``` + +### User REST endpoints (from `UserRestController.java`) + +``` +GET /api/v1/users → list[UserResponse] +POST /api/v1/users body: CreateUserRequest → UserResponse 201 +PUT /api/v1/users/{id} body: UpdateUserRequest → UserResponse 200 +DELETE /api/v1/users/{id} → 200 (no body) +POST /api/v1/users/{id}/password-change body: {password} → 200 (no body) +``` + +### Default user creation in lifespan (src/main.py update) + +```python +async with SessionLocal() as session: + user_repo = UserRepository(session) + existing = await user_repo.find_by_username(settings.default_user_username) + if not existing: + hashed = password_service.encode(settings.default_user_password) + await user_repo.create(username=settings.default_user_username, + password=hashed, authorities=list(AswgAuthority)) + elif settings.default_user_reset: + hashed = password_service.encode(settings.default_user_password) + await user_repo.update_password(existing.id, hashed) +``` + +--- + +## Completion Checklist + +- [ ] `AswgAuthority` enum has exactly 37 permission codes (count against Java source) +- [ ] `has_permission()` factory works as a `Depends()` +- [ ] JWT is created and validated with `HS256` +- [ ] Expired and blacklisted tokens are rejected with correct error JSON +- [ ] JWT middleware blocks all `/api/**` without a valid token +- [ ] Permit-all paths respond without token (health, login, logout, SSE, WS) +- [ ] `security_enabled=False` bypasses all auth checks +- [ ] `POST /api/v1/auth` → JWT for valid credentials, 401 for bad +- [ ] `GET /api/v1/auth/myself` → current user profile JSON +- [ ] `POST /api/v1/auth/logout` → blacklists token +- [ ] All user CRUD endpoints return correct status codes +- [ ] Default user created on first startup +- [ ] Password reset works when `aswg.default-user.reset=true` + +## Contract for Phase 3 + +Phase 3 imports: +- `from src.application.security.permissions import has_permission, AswgAuthority` +- `from src.application.security.authentication import CurrentUser, get_current_user` +- `from src.web.schemas.common import BaseSchema, RestErrorResponse` diff --git a/phases/phase-03-cfg-parser-process.md b/phases/phase-03-cfg-parser-process.md new file mode 100644 index 0000000..dbe2c52 --- /dev/null +++ b/phases/phase-03-cfg-parser-process.md @@ -0,0 +1,256 @@ +# Phase 3 — CFG Parser + Server Process Management + +**Status**: PENDING +**Depends on**: Phase 1 complete +**Next phase**: `phase-04-server-settings.md` +**Risk**: HIGH — CFG parser must be bit-perfect; server process management is complex. + +--- + +## Goal + +Two deliverables built in this phase: + +1. **Arma `.cfg` file parser** — reads and writes Arma server config files. Used by Phases 4 and 5. +2. **Server process management** — start/stop the Arma executable, PID tracking, cross-platform liveness check, SSE log streaming, status endpoint. + +After this phase: `GET /api/v1/status` returns server status; `GET /api/v1/logging/logs-sse` streams server log lines. + +--- + +## Java Source Files to Read + +`` = `E:\TestScript\ARMA-Server-Web-Gui\src\main\java\pl\bartlomiejstepien\armaserverwebgui\` + +### CFG Parser — read every file in this tree +``` +domain/server/storage/util/cfg/CfgFileHandler.java +domain/server/storage/util/cfg/DefaultCfgConfigReader.java +domain/server/storage/util/cfg/DefaultCfgConfigWriter.java +application/config/CfgHandlerConfig.java (shows how parsers are wired) +domain/server/storage/util/cfg/parser/ (all 8 parser classes) +domain/server/storage/util/cfg/annotation/ (CfgProperty, ClassName, ClassList) +domain/server/storage/util/cfg/util/ (all utility classes) +domain/server/storage/config/ServerConfigStorageImpl.java +domain/server/storage/config/model/ (all model classes) +``` + +### Server Process +``` +domain/server/process/ProcessServiceImpl.java (388 lines — read fully) +domain/server/process/ArmaServerParametersGeneratorImpl.java +domain/server/process/WindowsProcessAliveChecker.java +domain/server/process/UnixProcessAliveChecker.java +domain/server/process/ServerStatusService.java +domain/server/process/dto/ServerStatus.java +domain/server/process/model/ServerProcessStatus.java +domain/server/process/model/ArmaServerParameters.java +domain/server/process/log/ServerProcessLogMessageObserver.java +domain/server/process/log/SseServerServerProcessLogsObserver.java +domain/server/process/log/LoggerServerProcessLogsObserver.java +domain/server/process/log/FileServerProcessLogsObserver.java +web/StatusController.java +web/LoggingRestController.java (SSE endpoint only; properties endpoint in Phase 4) +``` + +--- + +## Output Files to Create + +``` +src/domain/server/__init__.py +src/domain/server/storage/__init__.py +src/domain/server/storage/cfg/__init__.py +src/domain/server/storage/cfg/cfg_file_handler.py +src/domain/server/storage/cfg/cfg_reader.py +src/domain/server/storage/cfg/cfg_writer.py +src/domain/server/storage/cfg/cfg_parsers.py +src/domain/server/storage/cfg/cfg_annotations.py +src/domain/server/storage/config/__init__.py +src/domain/server/storage/config/server_config_storage.py +src/domain/server/storage/config/models.py +src/domain/server/process/__init__.py +src/domain/server/process/process_service.py +src/domain/server/process/process_alive_checker.py +src/domain/server/process/server_status_service.py +src/domain/server/process/parameters_generator.py +src/domain/server/process/log_observers.py +src/domain/server/process/models.py +src/web/schemas/status.py +src/web/status_router.py +``` + +Update `src/main.py`: register `status_router`, add SSE route from `log_observers`. + +--- + +## Implementation Notes + +### CFG Parser — replacing Java annotation-reflection with Python dataclass metadata + +Java uses `@CfgProperty(name="maxPlayers")` + reflection to map fields to cfg keys. +Python equivalent: `dataclasses.field(metadata={"cfg_name": "maxPlayers", "parser": "integer"})`. + +```python +# cfg_annotations.py +from dataclasses import field + +def cfg_property(cfg_name: str, parser: str = "string", default=None): + return field(default=default, metadata={"cfg_name": cfg_name, "parser": parser}) + +def class_name(name: str): + return field(default=name, metadata={"class_name": name}) +``` + +Example config model: +```python +@dataclass +class NetworkConfig: + max_players: int = cfg_property("maxPlayers", parser="integer", default=32) + loopback: bool = cfg_property("loopback", parser="bool", default=False) + hostname: str = cfg_property("hostname", parser="quoted_string", default="") +``` + +### CFG Reader — core algorithm + +Arma `.cfg` format: +``` +maxPlayers = 32; +hostname = "My Server"; +class Missions { + class Mission_1 { + template = "Stratis.Stratis"; + difficulty = "Regular"; + }; +}; +``` + +Parse steps: +1. Strip `//` line comments and `/* */` block comments +2. Tokenize: `class`, `{`, `}`, `;`, `=`, identifiers, quoted strings, numbers +3. Recursively process `class NAME { ... };` blocks +4. For `key = value;` lines: look up field in dataclass by `cfg_name` metadata, call appropriate parser + +### CFG Parsers (port 8 Java parser classes as functions) + +```python +# cfg_parsers.py +def parse_quoted_string(s: str) -> str: # strips surrounding quotes +def parse_raw_string(s: str) -> str: # as-is +def parse_integer(s: str) -> int: +def parse_long(s: str) -> int: +def parse_bool(s: str) -> bool: # 1/0 or true/false +def parse_string_array(s: str) -> list[str]: # parses {"a","b","c"} +def parse_class(tokens, cls: type): # recursive class parser +def parse_class_list(tokens, cls: type) -> list: # list of class instances +``` + +**CRITICAL**: Write round-trip tests before implementing Phase 4: +```python +def test_roundtrip_network_config(tmp_path): + original = 'maxPlayers = 32;\nhostname = "Test";\n' + cfg = cfg_reader.read(original, NetworkConfig) + written = cfg_writer.write(cfg) + re_read = cfg_reader.read(written, NetworkConfig) + assert cfg == re_read +``` + +### Process alive checker + +Replace both Java Windows/Unix checkers with one `psutil` function: +```python +import psutil, sys + +def is_pid_alive(pid: int) -> bool: + if pid == 0: + return False + try: + proc = psutil.Process(pid) + return proc.status() not in (psutil.STATUS_DEAD, psutil.STATUS_ZOMBIE) + except psutil.NoSuchProcess: + return False +``` + +### Process service — key patterns from ProcessServiceImpl.java + +- PID file: `/arma_server.pid` — plain text integer, `0` means not running +- Startup lock: `asyncio.Lock()` (replaces Java `ReentrantLock`) +- Status transitions: `NOT_RUNNING → STARTING → RUNNING`, `RUNNING → NOT_RUNNING` +- Process launched: `subprocess.Popen(cmd, stdout=PIPE, stderr=PIPE)` +- stdout/stderr reader: two `threading.Thread` daemons reading lines, calling `_notify_observers(line)` +- Stop: `psutil.Process(pid).terminate()`, save pid=0 + +Observers list (`list[LogObserver]`) is wired in Phase 7 for Discord and at startup for SSE/file observers. + +### SSE log streaming — log_observers.py + +```python +import asyncio, uuid +from fastapi import APIRouter +from fastapi.responses import StreamingResponse + +_sse_clients: dict[str, asyncio.Queue] = {} + +def broadcast(line: str): + """Called by process service threads — thread-safe queue put.""" + for q in _sse_clients.values(): + q.put_nowait(line) + +async def _stream(client_id: str): + q = asyncio.Queue() + _sse_clients[client_id] = q + try: + while True: + try: + line = await asyncio.wait_for(q.get(), timeout=30.0) + yield f"data: {line}\n\n" + except asyncio.TimeoutError: + yield ": heartbeat\n\n" + finally: + _sse_clients.pop(client_id, None) + +router = APIRouter() + +@router.get("/api/v1/logging/logs-sse") # permit-all — no auth dependency +async def logs_sse(): + cid = str(uuid.uuid4()) + return StreamingResponse(_stream(cid), media_type="text/event-stream") +``` + +### Status endpoints (from StatusController.java) + +``` +GET /api/v1/status → {"status": "RUNNING"|"NOT_RUNNING"|"STARTING"|"UPDATING"} +POST /api/v1/status/toggle body: {"performUpdate": true|false} → 200 +``` + +Permission for toggle: `AswgAuthority.SERVER_START_STOP`. + +--- + +## Completion Checklist + +### CFG Parser +- [ ] Parses `key = value;` assignments of all types (string, int, bool, array) +- [ ] Parses nested `class X { ... };` blocks recursively +- [ ] Handles quoted strings, `{a,b,c}` arrays, and `//` / `/* */` comments +- [ ] Round-trip test passes for `NetworkConfig` and `ArmaServerConfig` + +### Server Process +- [ ] `is_pid_alive(pid)` works on both Windows and Linux +- [ ] `start_server()` launches Arma executable and saves PID to file +- [ ] `stop_server()` terminates the process and writes pid=0 +- [ ] `get_status()` reads PID file and checks liveness +- [ ] Log lines reach all connected SSE clients +- [ ] `GET /api/v1/status` → correct status JSON +- [ ] `POST /api/v1/status/toggle` → starts or stops server +- [ ] `GET /api/v1/logging/logs-sse` streams lines (no auth required) +- [ ] Heartbeat ping sent every 30s to keep SSE alive + +## Contract for Phase 4 + +Phase 4 imports: +- `from src.domain.server.storage.cfg.cfg_file_handler import CfgFileHandler` +- `from src.domain.server.storage.config.server_config_storage import ServerConfigStorage` +- `from src.domain.server.storage.config.models import ArmaServerConfig, NetworkConfig` +- `from src.domain.server.process.process_service import ProcessService` diff --git a/phases/phase-04-server-settings.md b/phases/phase-04-server-settings.md new file mode 100644 index 0000000..1185aaf --- /dev/null +++ b/phases/phase-04-server-settings.md @@ -0,0 +1,179 @@ +# Phase 4 — Server Settings Subdomains + +**Status**: PENDING +**Depends on**: Phase 3 complete (CFG parser + `ServerConfigStorage` must exist) +**Next phase**: `phase-05-mod-management.md` + +--- + +## Goal + +Five settings services that read/write Arma `.cfg` files and their REST endpoints. All follow the same read→model→return / receive→model→write pattern. + +After this phase: general, network, security, logging settings and difficulty profiles work end-to-end. + +--- + +## Java Source Files to Read + +`` = `E:\TestScript\ARMA-Server-Web-Gui\src\main\java\pl\bartlomiejstepien\armaserverwebgui\` + +| File | What to extract | +|------|----------------| +| `domain/server/general/GeneralServiceImpl.java` | getGeneralProperties(), saveGeneralProperties() | +| `domain/server/general/model/GeneralProperties.java` | Field names/types | +| `domain/server/network/ServerNetworkServiceImpl.java` | getNetworkProperties(), saveNetworkProperties() | +| `domain/server/network/model/NetworkProperties.java` | Fields | +| `domain/server/network/model/KickTimeoutType.java` | Enum values | +| `domain/server/logging/LoggingServiceImpl.java` | getLoggingProperties(), getLatestLogs() | +| `domain/server/logging/model/LoggingProperties.java` | Fields | +| `domain/server/difficulty/DifficultyServiceImpl.java` | Full implementation (DB + filesystem) | +| `domain/server/difficulty/model/DifficultyProfile.java` | Fields | +| `domain/server/difficulty/DifficultyScanJob.java` | Filesystem scan logic | +| `web/GeneralController.java` | Exact paths, JSON shapes | +| `web/ServerNetworkRestController.java` | Exact paths, JSON shapes | +| `web/LoggingRestController.java` | Properties endpoints (SSE already done in Phase 3) | +| `web/ServerSecurityRestController.java` | Exact paths, JSON shapes | +| `web/DifficultyRestController.java` | Exact paths, JSON shapes | +| `web/request/SaveGeneralProperties.java` | Request fields | +| `web/request/NetworkPropertiesRequest.java` | Request fields | +| `web/request/SaveServerSecurityRequest.java` | Request fields | +| `web/response/GeneralPropertiesResponse.java` | Response fields | +| `web/response/NetworkPropertiesResponse.java` | Response fields | +| `web/model/DifficultyProfileApiModel.java` | API model fields | + +--- + +## Output Files to Create + +``` +src/domain/server/general/__init__.py +src/domain/server/general/general_service.py +src/domain/server/general/models.py +src/domain/server/network/__init__.py +src/domain/server/network/network_service.py +src/domain/server/network/models.py +src/domain/server/logging_settings/__init__.py +src/domain/server/logging_settings/logging_service.py +src/domain/server/logging_settings/models.py +src/domain/server/security_settings/__init__.py +src/domain/server/security_settings/security_service.py +src/domain/server/security_settings/models.py +src/domain/server/difficulty/__init__.py +src/domain/server/difficulty/difficulty_service.py +src/domain/server/difficulty/jobs.py (DifficultyScanJob — registered in Phase 7) +src/domain/server/difficulty/models.py +src/web/schemas/general.py +src/web/schemas/network.py +src/web/schemas/security_settings.py +src/web/schemas/difficulty.py +src/web/general_router.py +src/web/network_router.py +src/web/logging_router.py +src/web/security_router.py +src/web/difficulty_router.py +``` + +Update `src/main.py`: register all 5 routers. + +--- + +## Implementation Notes + +### Common service pattern (general / network / logging / security) + +```python +class GeneralService: + def __init__(self, config_storage: ServerConfigStorage): + self.storage = config_storage + + async def get_properties(self) -> GeneralProperties: + cfg = self.storage.read() + return GeneralProperties(hostname=cfg.hostname, ...) + + async def save_properties(self, props: GeneralProperties) -> None: + cfg = self.storage.read() + cfg.hostname = props.hostname + # ... apply all fields + self.storage.write(cfg) +``` + +Apply the same pattern to Network, Logging, and Security services. + +### Difficulty service — DB + filesystem + +`DifficultyServiceImpl.java` manages profiles in both the DB and `/Users/*.Arma3Profile` files. + +```python +class DifficultyService: + async def get_profiles(self) -> list[DifficultyProfile]: + # Read from difficulty_profile table + scan filesystem for .Arma3Profile files + + async def save_profile(self, profile: DifficultyProfile) -> None: + # Write to DB + write .Arma3Profile via cfg_writer + + async def delete_profile(self, id: int | None, name: str | None) -> None: + # Remove from DB + delete file + + async def set_active(self, id: int) -> None: + # Mark active in DB; only one can be active at a time +``` + +`DifficultyScanJob`: scans `/Users/` for new `.Arma3Profile` files and inserts them into the DB if not already present. **Register this job with APScheduler in Phase 7** — do not register it here. + +### REST endpoints + +**General** — permissions: `GENERAL_SETTINGS_VIEW`, `GENERAL_SETTINGS_SAVE` +``` +GET /api/v1/general/properties → GeneralPropertiesResponse +POST /api/v1/general/properties body: SaveGeneralProperties → 200 +``` + +**Network** — permissions: `NETWORK_SETTINGS_VIEW`, `NETWORK_SETTINGS_SAVE` +``` +GET /api/v1/network/properties → NetworkPropertiesResponse +POST /api/v1/network/properties body: NetworkPropertiesRequest → 200 +``` + +**Logging** — permission: `LOGS_VIEW` +``` +GET /api/v1/logging/properties → LoggingPropertiesResponse +POST /api/v1/logging/properties body: LoggingPropertiesRequest → 200 +GET /api/v1/logging/latest-logs → list[str] +``` +Note: `GET /api/v1/logging/logs-sse` was registered in Phase 3. + +**Security settings** — permissions: `SECURITY_SETTINGS_VIEW`, `SECURITY_SETTINGS_SAVE` +``` +GET /api/v1/security → ServerSecurityResponse +POST /api/v1/security body: SaveServerSecurityRequest → 200 +``` + +**Difficulty** — permissions: `DIFFICULTY_VIEW`, `DIFFICULTY_ADD`, `DIFFICULTY_UPDATE`, `DIFFICULTY_DELETE` +``` +GET /api/v1/difficulties → list[DifficultyProfileApiModel] +POST /api/v1/difficulties body: DifficultyProfileApiModel → 201 +PUT /api/v1/difficulties/{id} body: DifficultyProfileApiModel → 200 +DELETE /api/v1/difficulties/{id} → 200 +DELETE /api/v1/difficulties?name={n} → 200 +``` + +--- + +## Completion Checklist + +- [ ] `GET /api/v1/general/properties` returns correct camelCase JSON +- [ ] `POST /api/v1/general/properties` persists to .cfg file +- [ ] `GET /api/v1/network/properties` returns correct JSON +- [ ] `POST /api/v1/network/properties` persists to .cfg file +- [ ] `GET /api/v1/logging/properties` returns correct JSON +- [ ] `GET /api/v1/logging/latest-logs` returns list of log line strings +- [ ] `GET /api/v1/security` returns correct JSON +- [ ] `GET /api/v1/difficulties` returns profiles list +- [ ] `POST /api/v1/difficulties` creates profile in DB and .Arma3Profile file +- [ ] `DELETE /api/v1/difficulties/{id}` removes profile from DB and filesystem +- [ ] All endpoints return 403 without required permission + +## Contract for Phase 5 + +Phase 5 has no direct dependency on Phase 4. It imports only from Phases 1–3. diff --git a/phases/phase-05-mod-management.md b/phases/phase-05-mod-management.md new file mode 100644 index 0000000..de463ef --- /dev/null +++ b/phases/phase-05-mod-management.md @@ -0,0 +1,292 @@ +# Phase 5 — Mod Management + +**Status**: PENDING +**Depends on**: Phase 1 complete (InstalledModRepository, ModPresetRepository, ModSettingsRepository); Phase 3 complete (ServerConfigStorage) +**Next phase**: `phase-06-steam-integration.md` + +--- + +## Goal + +Four REST routers covering mods, mod file uploads, mod presets, and mod settings. Five service classes. A WebSocket handler that pushes download-progress JSON to the Angular frontend. + +After this phase: the Mods page fully works (list, enable/disable, delete, upload). Preset management works. Mod settings CRUD works. + +--- + +## Java Source Files to Read + +`` = `E:\TestScript\ARMA-Server-Web-Gui\src\main\java\pl\bartlomiejstepien\armaserverwebgui\` + +| File | What to extract | +|------|----------------| +| `domain/server/mod/ModServiceImpl.java` | Full implementation — getModsCollection(), saveModFile(), saveEnabledModList(), manageMod(), deleteMod() | +| `domain/server/mod/ModPresetServiceImpl.java` | getModPresetsNames(), getModPreset(), saveModPreset(), deletePreset(), importPreset(), selectPreset() | +| `domain/server/mod/ModSettingsService.java` | getModSettingsWithoutContents(), getModSettingsContent(), saveModSettings(), deleteModSettings() | +| `domain/server/mod/ModKeyServiceImpl.java` | copyKeysForMod(), clearServerKeys() | +| `domain/server/mod/ModDependenciesService.java` | getDependencies(workshopFileId) | +| `domain/server/mod/WorkshopModInstallProgressWebsocketHandler.java` | publishInstallationStatus() pattern | +| `domain/server/mod/model/WorkshopModInstallationStatus.java` | Fields: fileId, title, status, installAttemptCount | +| `domain/server/mod/model/RelatedMod.java` | Fields + Status enum (INSTALLED, NOT_INSTALLED) | +| `domain/server/mod/dto/ModPreset.java` | Fields | +| `domain/server/mod/dto/ModSettings.java` | Fields | +| `domain/server/mod/dto/ModSettingsHeader.java` | Fields | +| `domain/server/mod/dto/PresetImportParams.java` | Fields (name, list of ModParam{id, title}) | +| `domain/server/mod/model/ModPresetSaveParams.java` | Fields | +| `domain/server/mod/job/InstallDeleteModsFromFilesystemJob.java` | Scan logic (register in Phase 7) | +| `domain/server/mod/job/ModSettingsScanJob.java` | Scan logic (register in Phase 7) | +| `domain/server/mod/job/ModUpdateJob.java` | Update logic (register in Phase 7) | +| `domain/server/storage/mod/FileSystemMod.java` | Fields: name, workshopFileId, modDirectory, hasFiles(), lastUpdated | +| `domain/server/storage/mod/ModDirectory.java` | Fields: path, directoryName, sizeBytes | +| `web/ModsRestController.java` | Exact paths and request/response JSON | +| `web/ModsFilesRestController.java` | Multipart upload endpoint | +| `web/ModsPresetsRestController.java` | All preset endpoints | +| `web/ModSettingsRestController.java` | All mod settings endpoints | + +--- + +## Output Files to Create + +``` +src/domain/server/mod/__init__.py +src/domain/server/mod/mod_service.py +src/domain/server/mod/mod_preset_service.py +src/domain/server/mod/mod_settings_service.py +src/domain/server/mod/mod_key_service.py +src/domain/server/mod/mod_dependencies_service.py +src/domain/server/mod/mod_file_storage.py (filesystem scanner + zip extractor) +src/domain/server/mod/models.py (Mod, ModsCollection, EnabledMod, RelatedMod, ModPreset, + ModSettings, WorkshopModInstallationStatus) +src/domain/server/mod/jobs.py (InstallDeleteModsJob, ModSettingsScanJob, ModUpdateJob + — register in Phase 7) +src/domain/server/mod/ws_handler.py (WebSocket broadcast for download progress) +src/web/schemas/mods.py +src/web/schemas/mod_presets.py +src/web/schemas/mod_settings.py +src/web/mods_router.py +src/web/mods_files_router.py +src/web/mods_presets_router.py +src/web/mod_settings_router.py +``` + +Update `src/main.py`: register all 4 routers, add `@app.websocket("/api/v1/ws/mod-install-status")`. + +--- + +## Implementation Notes + +### Mod statuses + +```python +from enum import StrEnum + +class ModStatus(StrEnum): + READY = "READY" + MISSING_FILES = "MISSING_FILES" + MISSING_DEPENDENCY_MODS = "MISSING_DEPENDENCY_MODS" +``` + +### ModFileStorage — filesystem scanning + +Scans `/mods/` for `@`-prefixed directories. Each contains `meta.cpp`: + +``` +publishedid = 463939057; +name = "ACE3"; +``` + +```python +# src/domain/server/mod/mod_file_storage.py +import re, shutil +from pathlib import Path +from datetime import datetime + +def scan_mod_directories(server_dir: Path) -> list[FileSystemMod]: + mods_dir = server_dir / "mods" + mods_dir.mkdir(parents=True, exist_ok=True) + return [_read_filesystem_mod(d) for d in mods_dir.iterdir() + if d.is_dir() and d.name.startswith("@")] + +def _read_filesystem_mod(mod_path: Path) -> FileSystemMod: + meta = mod_path / "meta.cpp" + workshop_file_id, name = 0, mod_path.name + if meta.exists(): + text = meta.read_text(errors="ignore") + if m := re.search(r'publishedid\s*=\s*(\d+)', text, re.I): + workshop_file_id = int(m.group(1)) + if m := re.search(r'name\s*=\s*"([^"]+)"', text, re.I): + name = m.group(1) + size = sum(f.stat().st_size for f in mod_path.rglob("*") if f.is_file()) + return FileSystemMod( + name=name, + workshop_file_id=workshop_file_id, + mod_directory=ModDirectory(mod_path, mod_path.name, size), + last_updated=datetime.fromtimestamp(mod_path.stat().st_mtime), + ) +``` + +### ModService — key methods + +```python +class ModService: + async def get_mods_collection(self) -> ModsCollection: + fs_mods = scan_mod_directories(self.server_dir) + db_mods = await self.mod_repo.find_all() + disabled = [m for m in db_mods if not m.enabled] + enabled = [m for m in db_mods if m.enabled] + not_managed = _find_not_managed(fs_mods, db_mods) + return ModsCollection( + disabled_mods=_to_mod_views(disabled, fs_mods), + enabled_mods=_to_mod_views(enabled, fs_mods), + not_managed_mods=_to_mod_views_fs(not_managed), + ) + + async def save_mod_file(self, file: UploadFile, overwrite: bool) -> None: + # 1. Save zip to temp; 2. Extract to server_dir/mods/; 3. Rename to lowercase + # 4. Read meta.cpp → workshopFileId; 5. Insert InstalledModEntity + ... + + async def save_enabled_mod_list(self, enabled_mods: list[EnabledMod]) -> None: + await self.mod_repo.disable_all_mods() + await self.mod_repo.enable_mods([m.workshop_file_id for m in enabled_mods]) + self.mod_key_service.clear_server_keys() + for mod in await self.mod_repo.find_enabled(): + self.mod_key_service.copy_keys_for_mod(Path(mod.directory_path)) +``` + +### ModKeyService + +Keys are `.bikey` files; destination is `/Keys/`. + +```python +class ModKeyService: + def copy_keys_for_mod(self, mod_path: Path) -> None: + for key_file in mod_path.rglob("*.bikey"): + shutil.copy2(key_file, self.keys_dir / key_file.name) + + def clear_server_keys(self) -> None: + for f in self.keys_dir.glob("*.bikey"): + f.unlink(missing_ok=True) +``` + +### WebSocket progress handler + +```python +# src/domain/server/mod/ws_handler.py +import json +from fastapi import WebSocket + +class ModInstallProgressHandler: + _sessions: dict[int, WebSocket] = {} + + async def connect(self, ws: WebSocket) -> None: + await ws.accept() + self._sessions[id(ws)] = ws + + async def disconnect(self, ws: WebSocket) -> None: + self._sessions.pop(id(ws), None) + + async def broadcast(self, status: dict) -> None: + msg = json.dumps(status) + for ws in list(self._sessions.values()): + try: + await ws.send_text(msg) + except Exception: + pass +``` + +In `src/main.py`: +```python +@app.websocket("/api/v1/ws/mod-install-status") +async def mod_install_ws(websocket: WebSocket): + await mod_install_handler.connect(websocket) + try: + while True: + await websocket.receive_text() + except Exception: + await mod_install_handler.disconnect(websocket) +``` + +### REST Endpoints + +**Mods** (`/api/v1/mods`) — permissions: `MODS_VIEW`, `MODS_UPDATE`, `MODS_DELETE`, `MODS_UPLOAD` +``` +GET /api/v1/mods → {disabledMods, enabledMods, notManagedMods} +POST /api/v1/mods/enabled body: {mods:[{workshopFileId,serverMod}]} → 200 +DELETE /api/v1/mods body: {name} → 200 +POST /api/v1/mods/manage body: {name} → 200 +DELETE /api/v1/mods/not-managed body: {directoryName} → 200 +``` + +**Mod Files** (`/api/v1/mods-files`) +``` +POST /api/v1/mods-files multipart: file(s) + overwrite? → 200 +GET /api/v1/mods-files/{name}/exists → {exists: bool} +``` + +**Mod Presets** (`/api/v1/mods-presets`) +``` +GET /api/v1/mods-presets → {presets: [name,...]} +GET /api/v1/mods-presets/{id} → ModPreset +PUT /api/v1/mods-presets/{name} body: {modNames:[...]} → {saved: true} +DELETE /api/v1/mods-presets/{name} → {deleted: true} +POST /api/v1/mods-presets/import body: {name, modParams:[{id,title}]} → 200 +POST /api/v1/mods-presets/select body: {name} → 200 +``` + +**Mod Settings** (`/api/v1/mods/settings`) +``` +GET /api/v1/mods/settings → list[ModSettingsHeader] +GET /api/v1/mods/settings/{id} → ModSettingsHeader +GET /api/v1/mods/settings/{id}/content → {content: str} +PUT /api/v1/mods/settings/{id} body: ModSettings → ModSettingsHeader +POST /api/v1/mods/settings body: ModSettings → ModSettingsHeader +DELETE /api/v1/mods/settings/{id} → 200 +``` + +### Key JSON shape — Mod object (camelCase) + +```json +{ + "workshopFileId": 463939057, + "name": "ACE3", + "serverMod": false, + "previewUrl": "https://example.com/preview.jpg", + "workshopUrl": "https://steamcommunity.com/sharedfiles/filedetails/?id=463939057", + "status": "READY", + "sizeBytes": 104857600, + "directoryName": "@ace", + "lastWorkshopUpdateDateTime": "2024-01-01T00:00:00", + "lastWorkshopUpdateAttemptDateTime": null +} +``` + +### Jobs (create here, register in Phase 7) + +- `InstallDeleteModsFromFilesystemJob`: scans `/mods/` and syncs DB (insert new found mods, mark missing ones). +- `ModSettingsScanJob`: scans mod dirs for settings files; inserts new entries into `mod_settings` table. +- `ModUpdateJob`: checks Steam Web API `lastUpdated` per enabled mod; schedules SteamCMD update if newer. + +--- + +## Completion Checklist + +- [ ] `GET /api/v1/mods` returns `{disabledMods, enabledMods, notManagedMods}` +- [ ] `POST /api/v1/mods/enabled` persists enabled list and copies `.bikey` files +- [ ] `POST /api/v1/mods-files` saves .zip upload and registers mod in DB +- [ ] `GET /api/v1/mods-files/{name}/exists` returns `{exists: bool}` +- [ ] `GET /api/v1/mods-presets` returns sorted preset name list +- [ ] `PUT /api/v1/mods-presets/{name}` saves preset +- [ ] `POST /api/v1/mods-presets/select` activates preset +- [ ] `GET /api/v1/mods/settings` returns list without content +- [ ] `GET /api/v1/mods/settings/{id}/content` returns raw content string +- [ ] WebSocket `/api/v1/ws/mod-install-status` accepts connections and broadcasts JSON +- [ ] All endpoints return 403 without required permission + +## Contract for Phase 6 + +Phase 6 imports: +- `from src.domain.server.mod.mod_service import ModService` +- `from src.domain.server.mod.models import WorkshopModInstallationStatus` +- `from src.domain.server.mod.ws_handler import ModInstallProgressHandler` +- `from src.domain.server.mod.mod_file_storage import scan_mod_directories` diff --git a/phases/phase-06-steam-integration.md b/phases/phase-06-steam-integration.md new file mode 100644 index 0000000..b2ff161 --- /dev/null +++ b/phases/phase-06-steam-integration.md @@ -0,0 +1,296 @@ +# Phase 6 — Steam Integration + +**Status**: PENDING +**Depends on**: Phase 5 complete (ModService, ModInstallProgressHandler); Phase 3 complete (ProcessService) +**Next phase**: `phase-07-missions-cdlc-discord-jobs.md` + +--- + +## Goal + +SteamCMD task queue, Steam Web API client (httpx-based), workshop mod querying/installing, server game update scheduling, WebSocket progress broadcasting, player list via python-a2s, and the steam settings endpoint. + +After this phase: workshop mod browsing, install, and Arma server update all work end-to-end. + +--- + +## Java Source Files to Read + +`` = `E:\TestScript\ARMA-Server-Web-Gui\src\main\java\pl\bartlomiejstepien\armaserverwebgui\` + +| File | What to extract | +|------|----------------| +| `domain/steam/SteamServiceImpl.java` | scheduleWorkshopModDownload(), scheduleArmaUpdate(), queryWorkshopMods(), getWorkshopMod(), getServerPlayers(), isServerRunning(), canUseWorkshop() | +| `domain/steam/SteamWebApiService.java` | queryWorkshopMods(), getWorkshopMod(), getWorkshopMods() — HTTP + caching | +| `domain/steam/SteamCmdHandler.java` | queueSteamTask(), @Scheduled consumer loop every 5s, retry logic | +| `domain/steam/SteamUtils.java` | ARMA_APP_ID = 107410 | +| `domain/steam/SteamArmaBranch.java` | Enum: PUBLIC, EXPERIMENTAL | +| `domain/steam/SteamTaskRetryPolicy.java` | Max retries, retryable exception check | +| `domain/steam/handler/WorkshopModDownloadTaskHandler.java` | steamcmd command construction + execution | +| `domain/steam/handler/WorkshopBatchModDownloadTaskHandler.java` | Batch download command | +| `domain/steam/handler/GameUpdateTaskHandler.java` | Server update command | +| `domain/steam/helper/SteamCmdModInstallHelper.java` | Post-install: move files, update DB, broadcast WS status | +| `domain/steam/model/WorkshopMod.java` | Fields: title, workshopFileId, previewUrl, children:[long] | +| `domain/steam/model/ArmaWorkshopQueryResponse.java` | Fields: nextCursor, mods | +| `domain/steam/model/WorkshopQueryParams.java` | Fields: cursor, searchText | +| `domain/steam/model/SteamTask.java` | Base + Type enum (WORKSHOP_MOD_INSTALL, GAME_UPDATE, WORKSHOP_BATCH_MOD_DOWNLOAD) | +| `domain/steam/model/QueuedSteamTask.java` | Fields: id(UUID), steamTask, attemptCount | +| `domain/steam/model/WorkshopModInstallSteamTask.java` | Fields: fileId, title, forced, issuer | +| `domain/steam/model/GameUpdateSteamTask.java` | Fields: issuer | +| `domain/steam/model/WorkshopBatchModDownloadTask.java` | Fields: Map, forced, issuer | +| `domain/steam/model/SteamCmdWorkshopDownloadParameters.java` | CLI args list | +| `domain/steam/model/SteamCmdAppUpdateParameters.java` | CLI args list for game update | +| `domain/steam/model/ModDownloadResult.java` | Fields: success, modDirectory | +| `web/WorkshopRestController.java` | All workshop endpoints with exact JSON shapes | +| `web/SteamSettingsController.java` | GET/POST /api/v1/settings/steam | + +--- + +## Output Files to Create + +``` +src/domain/steam/__init__.py +src/domain/steam/steam_service.py +src/domain/steam/steam_web_api_service.py +src/domain/steam/steam_cmd_handler.py +src/domain/steam/models.py (WorkshopMod, ArmaWorkshopQueryResponse, + WorkshopQueryParams, SteamTask + Type enum, + QueuedSteamTask, WorkshopModInstallSteamTask, + GameUpdateSteamTask, WorkshopBatchModDownloadTask) +src/web/schemas/steam.py +src/web/workshop_router.py +src/web/steam_settings_router.py +``` + +Update `src/main.py`: register `workshop_router`, `steam_settings_router`; in `lifespan` start `asyncio.create_task(run_task_loop())`. + +--- + +## Implementation Notes + +### ARMA_APP_ID + +```python +ARMA_APP_ID = 107410 +``` + +### Steam Web API — httpx replaces Java steam-web-api-client library + +```python +# src/domain/steam/steam_web_api_service.py +import httpx +from cachetools import TTLCache + +_query_cache: TTLCache = TTLCache(maxsize=100, ttl=300) # 5-min TTL + +STEAM_API_BASE = "https://api.steampowered.com" + +class SteamWebApiService: + def __init__(self, api_key: str): + self._api_key = api_key + self._client = httpx.AsyncClient(timeout=15.0) + + async def query_workshop_mods(self, params: WorkshopQueryParams) -> ArmaWorkshopQueryResponse: + if not self._api_key: + raise MissingSteamApiKeyException() + cache_key = f"q:{params.cursor}:{params.search_text}" + if cache_key in _query_cache: + return _query_cache[cache_key] + resp = await self._client.post( + f"{STEAM_API_BASE}/IPublishedFileService/QueryFiles/v1/", + data={ + "key": self._api_key, + "appid": ARMA_APP_ID, + "cursor": params.cursor or "*", + "numperpage": 10, + "search_text": params.search_text or "", + "return_previews": 1, + "return_children": 1, + "query_type": 3, # RANKED_BY_TREND + "filetype": 0, + } + ) + resp.raise_for_status() + data = resp.json().get("response", {}) + result = ArmaWorkshopQueryResponse( + next_cursor=data.get("next_cursor"), + mods=[_convert_mod(m) for m in data.get("publishedfiledetails", [])] + ) + _query_cache[cache_key] = result + return result + + async def get_workshop_mod(self, mod_id: int) -> WorkshopMod | None: + # GET /IPublishedFileService/GetDetails/v1/?key=...&publishedfileids[0]=... + ... +``` + +### SteamCMD task queue — asyncio background task + +```python +# src/domain/steam/steam_cmd_handler.py +import asyncio, uuid +from collections import deque + +_queue: deque[QueuedSteamTask] = deque() +_current: QueuedSteamTask | None = None +MAX_RETRIES = 3 + +def queue_task(task: SteamTask) -> uuid.UUID: + task_id = uuid.uuid4() + _queue.append(QueuedSteamTask(id=task_id, steam_task=task, attempt_count=0)) + return task_id + +def get_tasks_by_type(task_type: SteamTask.Type) -> list[SteamTask]: + tasks = [q.steam_task for q in _queue if q.steam_task.type == task_type] + if _current and _current.steam_task.type == task_type: + tasks.append(_current.steam_task) + return tasks + +async def run_task_loop(): + """Registered with asyncio.create_task() in lifespan.""" + global _current + while True: + await asyncio.sleep(5) + if not _queue: + continue + queued = _queue.popleft() + _current = queued + try: + await _dispatch(queued.steam_task) + except RetryableException: + if queued.attempt_count < MAX_RETRIES: + _queue.appendleft(QueuedSteamTask( + id=queued.id, + steam_task=queued.steam_task, + attempt_count=queued.attempt_count + 1, + )) + except Exception as e: + log.error("Steam task failed permanently: %s", e) + finally: + _current = None +``` + +### SteamCMD command construction + +```python +def build_workshop_download_command( + steamcmd_path: str, username: str, password: str, + workshop_content_path: str, file_id: int +) -> list[str]: + return [ + steamcmd_path, + "+login", username, password, + "+force_install_dir", workshop_content_path, + "+workshop_download_item", str(ARMA_APP_ID), str(file_id), + "+quit", + ] + +def build_game_update_command( + steamcmd_path: str, username: str, password: str, install_dir: str +) -> list[str]: + return [ + steamcmd_path, + "+login", username, password, + "+force_install_dir", install_dir, + "+app_update", str(ARMA_APP_ID), "validate", + "+quit", + ] +``` + +Run with `asyncio.create_subprocess_exec()`. On success, the mod appears at: +`/steamapps/workshop/content/107410//` + +Post-install steps: +1. Move directory to `/mods/@` +2. Normalize filenames to lowercase +3. Insert/update `InstalledModEntity` in DB +4. Broadcast `WorkshopModInstallationStatus` via `ModInstallProgressHandler.broadcast()` + +### Player list — python-a2s + +```python +import a2s + +async def get_server_players() -> list[ArmaServerPlayer]: + try: + players = await asyncio.to_thread( + a2s.players, ("localhost", 2303), timeout=3.0 + ) + return [ArmaServerPlayer(name=p.name, score=p.score, duration=p.duration) + for p in players] + except Exception: + return [] +``` + +### Steam settings persistence + +After a POST to `/api/v1/settings/steam`, write back to `config/aswg_default_config.properties`: + +```python +from jproperties import Properties + +def save_settings_to_file(): + p = Properties() + with open("config/aswg_default_config.properties", "rb") as f: + p.load(f) + p["aswg.steamcmd.path"] = settings.steamcmd_path + p["aswg.steamcmd.username"] = settings.steamcmd_username + # etc. + with open("config/aswg_default_config.properties", "wb") as f: + p.store(f) +``` + +### REST Endpoints + +**Workshop** (`/api/v1/workshop`) +``` +GET /api/v1/workshop/active → {active: bool} no auth required +POST /api/v1/workshop/query body:{cursor,searchText} WORKSHOP_INSTALL + → {nextCursor, mods:[...]} +GET /api/v1/workshop/installed-items → {mods:[...], modsUnderInstallation:[...]} WORKSHOP_VIEW +GET /api/v1/workshop/download-queue → {downloadingMods:[{fileId,modName,installAttemptCount,issuer}]} WORKSHOP_VIEW +GET /api/v1/workshop/mod/{id}/dependencies → {modId, dependencies:[{modId,modName,status}]} WORKSHOP_VIEW +POST /api/v1/workshop/install body:{fileId,modName,installDependencies} WORKSHOP_INSTALL + → {fileId: 123456} HTTP 200 +``` + +**Steam Settings** (`/api/v1/settings/steam`) +``` +GET /api/v1/settings/steam → {steamCmdPath,steamCmdUsername,steamCmdPassword:null, + steamCmdWorkshopContentPath,steamWebApiToken} STEAM_SETTINGS_UPDATE +POST /api/v1/settings/steam body: same shape → 200 STEAM_SETTINGS_UPDATE +POST /api/v1/settings/steam/password-change body:{password} → 200 STEAM_SETTINGS_UPDATE +``` + +Note: `steamCmdPassword` is always `null` in GET responses (never returned to client). + +### canUseWorkshop() + +```python +def can_use_workshop(self) -> bool: + path = settings.steamcmd_path + return bool(path) and Path(path).exists() +``` + +--- + +## Completion Checklist + +- [ ] `GET /api/v1/workshop/active` returns `{active: bool}` based on steamcmd path existence +- [ ] `POST /api/v1/workshop/query` returns paginated workshop results with cursor +- [ ] Workshop query results cached 5 minutes (TTLCache) +- [ ] `GET /api/v1/workshop/installed-items` returns installed + in-progress lists +- [ ] `GET /api/v1/workshop/download-queue` returns tasks with fileId, modName, issuer +- [ ] `POST /api/v1/workshop/install` queues SteamCMD task, returns `{fileId}` +- [ ] SteamCMD task loop started in lifespan, polls every 5 seconds +- [ ] After download: mod moves to `server_dir/mods/`, DB record inserted, WS broadcast sent +- [ ] `GET /api/v1/settings/steam` never returns password field value +- [ ] `POST /api/v1/settings/steam` persists to properties file +- [ ] `python-a2s` player query works (returns empty list if server offline) + +## Contract for Phase 7 + +Phase 7 imports: +- `from src.domain.steam.steam_service import SteamService` +- `from src.domain.steam.steam_cmd_handler import queue_task, run_task_loop, get_tasks_by_type` +- `from src.domain.steam.models import SteamTask, WorkshopModInstallSteamTask, GameUpdateSteamTask` diff --git a/phases/phase-07-missions-cdlc-discord-jobs.md b/phases/phase-07-missions-cdlc-discord-jobs.md new file mode 100644 index 0000000..5fbff5d --- /dev/null +++ b/phases/phase-07-missions-cdlc-discord-jobs.md @@ -0,0 +1,313 @@ +# Phase 7 — Missions, CDLC, Discord, Jobs + +**Status**: PENDING +**Depends on**: Phase 3 (ProcessService, ServerConfigStorage); Phase 4 (DifficultyScanJob stub); Phase 5 (ModService, mod jobs stubs); Phase 6 (SteamService, run_task_loop) +**Next phase**: `phase-08-middleware-polish.md` + +--- + +## Goal + +Mission CRUD + file upload, CDLC management, Discord webhook notifications, and all APScheduler jobs registered. After this phase every domain feature is complete and the scheduler drives automated background work. + +--- + +## Java Source Files to Read + +`` = `E:\TestScript\ARMA-Server-Web-Gui\src\main\java\pl\bartlomiejstepien\armaserverwebgui\` + +| File | What to extract | +|------|----------------| +| `domain/server/mission/MissionServiceImpl.java` | getMissions(), saveEnabledMissionList(), addMission(), deleteMission(), updateMission(), save(MultipartFile), checkMissionFileExists() | +| `domain/server/mission/VanillaMissionsImporter.java` | Import built-in missions on startup | +| `domain/server/mission/dto/Mission.java` | Fields: id, name, template, enabled, difficulty | +| `domain/server/mission/dto/Missions.java` | Fields: disabledMissions, enabledMissions | +| `domain/server/mission/job/MissionScannerJob.java` | Filesystem scan + DB sync logic | +| `domain/server/cdlc/CdlcService.java` | findAll(), toggleCdlc(id) | +| `domain/server/cdlc/CdlcFileStorageService.java` | Filesystem existence check | +| `domain/server/cdlc/dto/Cdlc.java` | Fields: id, name, enabled, fileExists | +| `domain/discord/DiscordIntegration.java` | sendMessage(MessageKind) — async, checks enabled flag | +| `domain/discord/DiscordWebhookHandler.java` | HTTP POST to webhook URL | +| `domain/discord/message/MessageKind.java` | Enum: SERVER_STARTED, SERVER_STARTING, SERVER_STOPPED, SERVER_UPDATING, PLAYER_JOINED | +| `domain/discord/message/ServerStartedMessageCreator.java` | Message text | +| `domain/discord/message/ServerStoppedMessageCreator.java` | Message text | +| `domain/discord/model/DiscordMessage.java` | Fields: username, content, embeds:[{title,description,color}] | +| `application/scheduling/AswgJob.java` | Base job interface: getName(), run() | +| `application/scheduling/AswgTaskScheduler.java` | schedule(job, cron), runNow(job), cancel(name) | +| `application/scheduling/JobExecutionInfoService.java` | saveJobExecution(), getJobHistory() | +| `application/scheduling/dto/JobExecution.java` | Fields: jobName, startDate, finishDate, status, message | +| `domain/server/difficulty/DifficultyScanJob.java` | Scan Users/*.Arma3Profile + DB sync | +| `domain/server/mod/job/InstallDeleteModsFromFilesystemJob.java` | Scan mods/ + DB sync | +| `domain/server/mod/job/ModSettingsScanJob.java` | Scan mod dirs for settings files | +| `domain/server/mod/job/ModUpdateJob.java` | Check Steam API lastUpdated per mod | +| `application/security/jwt/cleaner/InvalidJwtCleaner.java` | Delete expired JWT rows from DB | +| `web/MissionRestController.java` | Exact paths and JSON shapes | +| `web/MissionFilesRestController.java` | Mission file upload | +| `web/CdlcRestController.java` | GET + POST toggle | + +--- + +## Output Files to Create + +``` +src/domain/server/mission/__init__.py +src/domain/server/mission/mission_service.py +src/domain/server/mission/mission_file_storage.py (scan MPMissions/ for .pbo files) +src/domain/server/mission/vanilla_missions_importer.py +src/domain/server/mission/jobs.py (MissionScannerJob) +src/domain/server/mission/models.py (Mission, Missions dataclasses) +src/domain/server/cdlc/__init__.py +src/domain/server/cdlc/cdlc_service.py +src/domain/server/cdlc/cdlc_file_storage.py +src/domain/server/cdlc/models.py (Cdlc dataclass) +src/domain/discord/__init__.py +src/domain/discord/discord_integration.py +src/domain/discord/discord_webhook_handler.py +src/domain/discord/models.py (DiscordMessage, MessageKind enum) +src/application/scheduler/__init__.py +src/application/scheduler/scheduler.py (APScheduler setup + job registry) +src/application/scheduler/job_execution_service.py +src/web/schemas/missions.py +src/web/schemas/cdlc.py +src/web/mission_router.py +src/web/mission_files_router.py +src/web/cdlc_router.py +``` + +Update `src/main.py`: +- Register `mission_router`, `mission_files_router`, `cdlc_router` +- In `lifespan`: `scheduler.start()`, register all 6 jobs, call `vanilla_missions_importer.run()` on first startup +- In `lifespan` teardown: `scheduler.shutdown()` + +--- + +## Implementation Notes + +### APScheduler setup + +```python +# src/application/scheduler/scheduler.py +from apscheduler.schedulers.asyncio import AsyncIOScheduler +from apscheduler.triggers.cron import CronTrigger + +scheduler = AsyncIOScheduler() + +def schedule_job(job_fn, cron: str, job_id: str) -> None: + try: + scheduler.remove_job(job_id) + except Exception: + pass + scheduler.add_job(job_fn, CronTrigger.from_crontab(cron), id=job_id, + misfire_grace_time=60) +``` + +Register in lifespan (cron strings come from `settings.*_cron` properties): +```python +scheduler.start() +schedule_job(difficulty_scan_job, settings.difficulty_scan_cron, "difficulty_scan") +schedule_job(mission_scanner_job, settings.mission_scan_cron, "mission_scan") +schedule_job(mod_filesystem_job, settings.mod_scan_cron, "mod_filesystem_scan") +schedule_job(mod_settings_scan_job,settings.mod_settings_scan_cron, "mod_settings_scan") +schedule_job(mod_update_job, settings.mod_update_cron, "mod_update") +schedule_job(jwt_cleaner_job, "0 * * * *", "jwt_cleaner") +``` + +Read the cron expressions from `aswg-default-config.properties` (Java file) and add them as `settings` fields. + +### Job execution tracking + +Each job records execution in the `job_execution` table: + +```python +# src/application/scheduler/job_execution_service.py +async def wrap_job(job_name: str, job_fn) -> None: + start = datetime.utcnow() + status, message = "SUCCESS", None + try: + await job_fn() + except Exception as e: + status, message = "FAILED", str(e) + log.error("Job %s failed: %s", job_name, e) + finally: + await job_execution_repo.save(JobExecutionEntity( + job_name=job_name, + start_date=start, + finish_date=datetime.utcnow(), + status=status, + message=message, + )) +``` + +### MissionService + +Missions are `.pbo` files in `/MPMissions/`. + +```python +async def get_missions(self) -> Missions: + db_missions = await self.mission_repo.find_all() + return Missions( + disabled_missions=[m for m in db_missions if not m.enabled], + enabled_missions=[m for m in db_missions if m.enabled], + ) + +async def save_enabled_mission_list(self, missions: list[Mission]) -> None: + await self.mission_repo.disable_all() + for m in missions: + await self.mission_repo.enable_by_template(m.template) + await self.mission_repo.update_difficulty(m.template, m.difficulty) + # Write missions block to server.cfg via ServerConfigStorage + cfg = self.config_storage.read() + cfg.missions = [CfgMission(template=m.template, difficulty=m.difficulty) + for m in missions] + self.config_storage.write(cfg) + +async def add_mission(self, name: str, template: str) -> None: + await self.mission_repo.save(MissionEntity(name=name, template=template, + enabled=False, difficulty="Regular")) +``` + +`template` format: `"filename.MapName"` e.g. `"tdm_stratis.Stratis"` (`.pbo` extension excluded). + +### MissionScannerJob + +Scans `/MPMissions/` for `*.pbo` files. Derives template name by stripping `.pbo`. Inserts missing DB entries, removes DB entries where file is gone. + +### VanillaMissionsImporter + +Reads a bundled JSON/text list of vanilla Arma 3 mission templates. On first startup (`@app.on_event` / lifespan), insert any missing. Check existence with `mission_repo.find_by_template()`. + +### CDLC service + +```python +async def find_all(self) -> list[Cdlc]: + entities = await self.cdlc_repo.find_all() + return [ + Cdlc( + id=e.id, name=e.name, enabled=e.enabled, + file_exists=(Path(settings.server_directory_path) / e.name).exists() + ) + for e in entities + ] + +async def toggle_cdlc(self, id: int) -> None: + entity = await self.cdlc_repo.find_by_id(id) + entity.enabled = not entity.enabled + await self.cdlc_repo.save(entity) +``` + +### Discord Integration + +```python +# src/domain/discord/models.py +from enum import StrEnum +from dataclasses import dataclass + +class MessageKind(StrEnum): + SERVER_STARTED = "SERVER_STARTED" + SERVER_STARTING = "SERVER_STARTING" + SERVER_STOPPED = "SERVER_STOPPED" + SERVER_UPDATING = "SERVER_UPDATING" + PLAYER_JOINED = "PLAYER_JOINED" + +@dataclass +class DiscordMessage: + username: str + content: str + embeds: list[dict] + + def to_dict(self) -> dict: + return {"username": self.username, "content": self.content, + "embeds": self.embeds} +``` + +```python +# src/domain/discord/discord_integration.py +import asyncio + +_MESSAGE_TEXTS = { + MessageKind.SERVER_STARTED: "Server has started.", + MessageKind.SERVER_STARTING: "Server is starting...", + MessageKind.SERVER_STOPPED: "Server has stopped.", + MessageKind.SERVER_UPDATING: "Server is updating...", +} + +class DiscordIntegration: + async def send_message(self, kind: MessageKind) -> None: + if not settings.discord_enabled: + return + text = _MESSAGE_TEXTS.get(kind, kind.value) + msg = DiscordMessage(username="ASWG", content=text, embeds=[]) + asyncio.create_task(self._webhook.send(msg)) # fire-and-forget + +class DiscordWebhookHandler: + async def send(self, message: DiscordMessage) -> None: + try: + async with httpx.AsyncClient() as client: + await client.post(settings.discord_webhook_url, + json=message.to_dict(), timeout=10.0) + except Exception as e: + log.warning("Discord webhook failed: %s", e) +``` + +Wire into `ProcessService` (edit Phase 3's `process_service.py`): +- After `start_server()` completes: `await discord.send_message(MessageKind.SERVER_STARTED)` +- After `stop_server()`: `await discord.send_message(MessageKind.SERVER_STOPPED)` + +### REST Endpoints + +**Missions** (`/api/v1/missions`) +``` +GET /api/v1/missions → {disabledMissions, enabledMissions} MISSION_VIEW +POST /api/v1/missions/enabled body:{missions:[{name,template,difficulty,enabled}]} MISSION_UPDATE → 200 +POST /api/v1/missions/template body:{name,template} MISSION_ADD → 200 +DELETE /api/v1/missions/template body:{template} MISSION_DELETE → 200 +PUT /api/v1/missions/id/{id} body:Mission MISSION_UPDATE → 200 +``` + +**Mission Files** (`/api/v1/missions-files`) +``` +POST /api/v1/missions-files multipart: file(s) + overwrite? MISSION_UPLOAD → 200 +GET /api/v1/missions-files/{name}/exists → {exists: bool} MISSION_VIEW +``` + +**CDLC** (`/api/v1/cdlc`) +``` +GET /api/v1/cdlc → {cdlcs:[{id,name,enabled,fileExists}]} CDLC_VIEW +POST /api/v1/cdlc/{id}/toggle → 200 CDLC_UPDATE +``` + +### JSON shapes + +Mission: +```json +{"id": 1, "name": "TDM Stratis", "template": "tdm_stratis.Stratis", "enabled": true, "difficulty": "Regular"} +``` + +CDLC: +```json +{"id": 1, "name": "gm", "enabled": false, "fileExists": true} +``` + +--- + +## Completion Checklist + +- [ ] `GET /api/v1/missions` returns `{disabledMissions, enabledMissions}` +- [ ] `POST /api/v1/missions/enabled` persists and writes mission block to server.cfg +- [ ] `POST /api/v1/missions/template` inserts mission row with `enabled=false` +- [ ] `POST /api/v1/missions-files` saves .pbo and inserts DB record +- [ ] `GET /api/v1/cdlc` computes `fileExists` from filesystem at request time +- [ ] `POST /api/v1/cdlc/{id}/toggle` flips enabled flag +- [ ] APScheduler starts in lifespan; all 6 jobs registered with cron from settings +- [ ] Each job records execution row in `job_execution` table +- [ ] Discord fires on server start/stop when `discord.enabled=true` +- [ ] `InvalidJwtCleaner` hourly job deletes rows where `expiration_datetime < now()` +- [ ] `VanillaMissionsImporter` inserts default missions on first startup + +## Contract for Phase 8 + +Phase 8 adds no new service classes. It wraps everything built in Phases 1–7 with: +- Global exception handler (FastAPI `@app.exception_handler`) +- SPA redirect middleware (non-API paths → `index.html`) +- Optional rate limiting middleware +- Structured logging via `structlog` diff --git a/phases/phase-08-middleware-polish.md b/phases/phase-08-middleware-polish.md new file mode 100644 index 0000000..a82b3d0 --- /dev/null +++ b/phases/phase-08-middleware-polish.md @@ -0,0 +1,288 @@ +# Phase 8 — Middleware + Polish + +**Status**: PENDING +**Depends on**: Phases 1–7 all complete +**Next phase**: `phase-09-testing.md` + +--- + +## Goal + +Global exception handling, SPA redirect middleware, structured logging, and final `src/main.py` wiring. No new domain logic. After this phase the app is production-equivalent to the Java version for all error cases and routing edge cases. + +--- + +## Java Source Files to Read + +`` = `E:\TestScript\ARMA-Server-Web-Gui\src\main\java\pl\bartlomiejstepien\armaserverwebgui\` + +| File | What to extract | +|------|----------------| +| `application/ApiException.java` | Annotation fields: `code (ApiExceptionCode)`, `messageKey (str)`, `status (HttpStatus)` | +| `application/ApiExceptionCode.java` | **All enum values** — these become the `api_code` strings on Python exceptions | +| `application/ApiExceptionFilter.java` | Catch-all servlet filter: only handles `/api/**` paths; calls `ApiErrorResponseResolver.resolve()` | +| `application/ApiErrorResponseResolver.java` | Resolves throwable → `RestErrorResponse` using annotation metadata or falls back to 500 `SERVER_ERROR` | +| `application/frontend/FrontEndRedirectWebFilter.java` | Forward non-API, non-dotted paths to `/index.html` via `request.getRequestDispatcher` | +| `application/IpAddressMdcHttpFilter.java` | Adds `X-Forwarded-For` / remote addr to MDC logging context | +| `application/RateLimitWebFilter.java` | **Entire file is commented out** — rate limiting was never implemented; do not port | +| `web/response/RestErrorResponse.java` | Fields: `code (str)`, `message (str)` — `status` int is used internally but NOT in JSON | + +--- + +## Output Files to Create + +``` +src/application/middleware/__init__.py +src/application/middleware/spa_redirect.py +src/application/middleware/error_handler.py +src/application/middleware/logging_middleware.py +``` + +Update `src/main.py` with final middleware stack and complete router list. + +--- + +## Implementation Notes + +### RestErrorResponse — only two fields in JSON + +The Java `RestErrorResponse.java` has an internal `status` int but it does NOT appear in the serialized JSON. The Angular frontend reads only `code` and `message`. The `RestErrorResponse` class was already defined in Phase 2 (`src/web/schemas/common.py`): + +```python +class RestErrorResponse(BaseSchema): + code: str + message: str +``` + +Do not add a `status` field to JSON output. + +### Domain exception pattern (apply to ALL exception classes in Phases 3–7) + +Replace Java's `@ApiException` annotation with class-level attributes: + +```python +class ModFileAlreadyExistsException(Exception): + api_code = "MOD_FILE_ALREADY_EXISTS" + api_status = 409 + api_message = "Mod file already exists." + +class ModIdAlreadyRegisteredException(Exception): + api_code = "MOD_ID_ALREADY_REGISTERED" + api_status = 409 + api_message = "Mod with this workshop ID is already registered." + +class ModIdCannotBeZeroException(Exception): + api_code = "MOD_ID_CANNOT_BE_ZERO" + api_status = 400 + api_message = "Mod workshop file ID cannot be zero." + +class NotManagedModNotFoundException(Exception): + api_code = "NOT_MANAGED_MOD_NOT_FOUND" + api_status = 404 + api_message = "Not-managed mod not found." + +class MissingSteamApiKeyException(Exception): + api_code = "MISSING_STEAM_API_KEY" + api_status = 400 + api_message = "Steam API key is not configured." + +class SteamCmdNotInstalled(Exception): + api_code = "STEAMCMD_NOT_INSTALLED" + api_status = 400 + api_message = "SteamCMD is not installed or path is not set." +``` + +Read `ApiExceptionCode.java` for the **complete** enum — add every value as a matching Python exception class. + +### Global exception handler + +```python +# src/application/middleware/error_handler.py +from fastapi import Request +from fastapi.responses import JSONResponse + +async def global_exception_handler(request: Request, exc: Exception) -> JSONResponse: + path = request.url.path + if not path.startswith("/api"): + # Non-API paths: StaticFiles or SPA redirect will handle it + from fastapi.responses import Response + return Response(status_code=404) + + code = getattr(exc, "api_code", "SERVER_ERROR") + status = getattr(exc, "api_status", 500) + message = getattr(exc, "api_message", "Internal server error.") + + if status >= 500: + import logging + logging.getLogger(__name__).error( + "Unhandled exception on %s %s", request.method, path, exc_info=exc + ) + return JSONResponse( + status_code=status, + content={"code": code, "message": message}, + ) +``` + +Register: +```python +app.add_exception_handler(Exception, global_exception_handler) +``` + +### SPA redirect middleware + +Java's `FrontEndRedirectWebFilter` forwards to `index.html` when the path: +- Does NOT start with `/api` +- Contains no dot (i.e. is not a static asset like `main.js` or `favicon.ico`) + +```python +# src/application/middleware/spa_redirect.py +import re +from starlette.middleware.base import BaseHTTPMiddleware +from starlette.requests import Request +from starlette.responses import FileResponse + +# Match paths that have no dot in the last segment (no file extension) +_IS_SPA_ROUTE = re.compile(r"[^.]*$") + +class SPARedirectMiddleware(BaseHTTPMiddleware): + async def dispatch(self, request: Request, call_next): + path = request.url.path + if (not path.startswith("/api") + and not path.startswith("/ws") + and _IS_SPA_ROUTE.fullmatch(path.split("/")[-1])): + return FileResponse("static/index.html") + return await call_next(request) +``` + +### Request logging middleware (IP context) + +```python +# src/application/middleware/logging_middleware.py +import structlog.contextvars +from starlette.middleware.base import BaseHTTPMiddleware + +class RequestLoggingMiddleware(BaseHTTPMiddleware): + async def dispatch(self, request, call_next): + ip = request.headers.get("X-Forwarded-For", "") + if not ip and request.client: + ip = request.client.host + structlog.contextvars.bind_contextvars(client_ip=ip, + path=request.url.path) + try: + return await call_next(request) + finally: + structlog.contextvars.clear_contextvars() +``` + +### Structlog configuration (add to top of src/main.py) + +```python +import logging, structlog + +structlog.configure( + processors=[ + structlog.contextvars.merge_contextvars, + structlog.processors.add_log_level, + structlog.processors.TimeStamper(fmt="iso"), + structlog.dev.ConsoleRenderer(), + ], + wrapper_class=structlog.make_filtering_bound_logger(logging.INFO), + logger_factory=structlog.PrintLoggerFactory(), +) +``` + +### Final src/main.py — complete structure + +```python +# Middleware (registration order = outermost first in ASGI stack) +app.add_middleware(RequestLoggingMiddleware) # Phase 8 +app.add_middleware(JwtMiddleware) # Phase 2 +app.add_middleware(SPARedirectMiddleware) # Phase 8 + +# Exception handler +app.add_exception_handler(Exception, global_exception_handler) + +# All 17 routers +app.include_router(auth_router) # Phase 2 /api/v1/auth +app.include_router(user_router) # Phase 2 /api/v1/users +app.include_router(status_router) # Phase 3 /api/v1/status +app.include_router(general_router) # Phase 4 /api/v1/general +app.include_router(network_router) # Phase 4 /api/v1/network +app.include_router(logging_router) # Phase 4 /api/v1/logging +app.include_router(security_router) # Phase 4 /api/v1/security +app.include_router(difficulty_router) # Phase 4 /api/v1/difficulties +app.include_router(mods_router) # Phase 5 /api/v1/mods +app.include_router(mods_files_router) # Phase 5 /api/v1/mods-files +app.include_router(mods_presets_router) # Phase 5 /api/v1/mods-presets +app.include_router(mod_settings_router) # Phase 5 /api/v1/mods/settings +app.include_router(workshop_router) # Phase 6 /api/v1/workshop +app.include_router(steam_settings_router) # Phase 6 /api/v1/settings/steam +app.include_router(mission_router) # Phase 7 /api/v1/missions +app.include_router(mission_files_router) # Phase 7 /api/v1/missions-files +app.include_router(cdlc_router) # Phase 7 /api/v1/cdlc + +# WebSocket (Phase 5) +@app.websocket("/api/v1/ws/mod-install-status") +async def mod_install_ws(websocket: WebSocket): ... + +# Actuator endpoints (permit-all, Phase 1) +@app.get("/api/v1/actuator/health") +async def health(): return {"status": "UP"} + +@app.get("/api/v1/actuator/info") +async def info(): return {"application": {"name": "ASWG"}} + +# Static files — MUST be last +app.mount("/", StaticFiles(directory="static", html=True), name="static") +``` + +### Lifespan complete sequence + +```python +@asynccontextmanager +async def lifespan(app: FastAPI): + # 1. Run Alembic migrations to "head" + alembic_cfg = Config("alembic.ini") + command.upgrade(alembic_cfg, "head") + + # 2. Create default user (Phase 2) + await create_default_user_if_absent() + + # 3. Start SteamCMD task loop (Phase 6) + steam_task = asyncio.create_task(run_task_loop()) + + # 4. Start APScheduler + register all 6 jobs (Phase 7) + scheduler.start() + _register_all_jobs() + + # 5. Import vanilla missions on first run (Phase 7) + await vanilla_missions_importer.run() + + yield + + # Teardown + scheduler.shutdown(wait=False) + steam_task.cancel() +``` + +--- + +## Completion Checklist + +- [ ] Domain exceptions in all phases have `api_code`, `api_status`, `api_message` attrs +- [ ] `POST /api/v1/mods-files` with wrong file type returns 409 `{"code":"...", "message":"..."}` +- [ ] 500 errors return `{"code":"SERVER_ERROR", "message":"Internal server error."}` +- [ ] Navigating browser to `/mods` returns `index.html` content +- [ ] `/api/**` paths are NOT intercepted by SPA middleware +- [ ] `static/main.js` (dotted path) is NOT redirected +- [ ] Request log lines include `client_ip` and `path` fields +- [ ] `GET /api/v1/actuator/health` works without JWT token +- [ ] All 17 routers registered in correct order in `src/main.py` +- [ ] Rate limiting: explicitly NOT implemented (matches Java source) + +## Contract for Phase 9 + +Phase 9 imports: +- `from src.main import app` (the FastAPI instance) +- `from src.repository.base import SessionLocal, engine, Base` +- All router and service classes for integration testing diff --git a/phases/phase-09-testing.md b/phases/phase-09-testing.md new file mode 100644 index 0000000..e72d2d1 --- /dev/null +++ b/phases/phase-09-testing.md @@ -0,0 +1,647 @@ +# Phase 9 — Testing + +**Status**: PENDING +**Depends on**: Phases 1–8 all complete (`from src.main import app` must import without error) +**Next phase**: None — final phase + +--- + +## Goal + +Achieve ≥ 80 % line coverage across `src/`. Every public contract tested with real SQLite-in-memory DB and +`httpx.AsyncClient` over ASGI transport. No mocking of internal service classes. Steam API HTTP calls mocked +with `respx`. + +--- + +## Tools & Libraries + +| Package | Purpose | +|---------|---------| +| `pytest` | Test runner | +| `pytest-asyncio` | `asyncio_mode = "auto"` for async fixtures | +| `pytest-cov` | Coverage measurement | +| `httpx` | ASGI test client | +| `respx` | Mock httpx HTTP calls (Steam Web API) | +| `aiosqlite` | Async SQLite driver for in-memory test DB | +| `sqlalchemy[asyncio]` | Same ORM as production | +| `bcrypt` | Password hashing (same as production) | +| `python-jose` | JWT signing for test auth tokens | + +Install: +``` +pip install pytest pytest-asyncio pytest-cov httpx respx aiosqlite python-jose +``` + +--- + +## pytest.ini / pyproject.toml + +```ini +[pytest] +asyncio_mode = auto +testpaths = tests +``` + +``` +pytest --cov=src --cov-report=term-missing --cov-fail-under=80 +``` + +--- + +## Output Files to Create + +``` +tests/__init__.py +tests/conftest.py +tests/unit/test_cfg_parser.py +tests/unit/test_permissions.py +tests/unit/test_password.py +tests/unit/test_mod_file_storage.py +tests/integration/test_auth.py +tests/integration/test_users.py +tests/integration/test_status.py +tests/integration/test_general_settings.py +tests/integration/test_mods.py +tests/integration/test_workshop.py +tests/integration/test_missions.py +tests/integration/test_cdlc.py +tests/integration/test_scheduler_jobs.py +``` + +--- + +## Implementation Notes + +### conftest.py — shared fixtures + +```python +# tests/conftest.py +import asyncio, pytest, jose.jwt as jwt +from datetime import datetime +from httpx import AsyncClient, ASGITransport +from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession +from sqlalchemy.orm import sessionmaker + +from src.main import app +from src.repository.base import Base, get_db +from src.domain.user.models import UserEntity +from src.security.password import hash_password + +TEST_DB_URL = "sqlite+aiosqlite:///:memory:" + +# --- Engine (session-scoped: created once per test run) --- +@pytest.fixture(scope="session") +async def engine(): + eng = create_async_engine(TEST_DB_URL, echo=False) + async with eng.begin() as conn: + await conn.run_sync(Base.metadata.create_all) + yield eng + await eng.dispose() + +# --- Per-test session with rollback isolation --- +@pytest.fixture +async def db_session(engine): + async with engine.begin() as conn: + session_factory = sessionmaker( + bind=conn, class_=AsyncSession, expire_on_commit=False + ) + session = session_factory() + try: + yield session + finally: + await session.rollback() + await session.close() + +# --- Override FastAPI dependency --- +@pytest.fixture(autouse=True) +def override_db(db_session): + async def _get_test_db(): + yield db_session + app.dependency_overrides[get_db] = _get_test_db + yield + app.dependency_overrides.clear() + +# --- Seed default user --- +@pytest.fixture +async def seed_user(db_session): + user = UserEntity( + username="testuser", + password=hash_password("changeme"), + enabled=True, + ) + db_session.add(user) + await db_session.commit() + return user + +# --- JWT with ALL 37 authorities (far-future exp) --- +ALL_AUTHORITIES = [ + "STATUS_VIEW", "MODS_VIEW", "MODS_UPDATE", "MODS_UPLOAD", "MODS_DELETE", + "MODS_PRESETS_VIEW", "MODS_PRESETS_UPDATE", "MODS_PRESETS_DELETE", + "MOD_SETTINGS_VIEW", "MOD_SETTINGS_UPDATE", "MOD_SETTINGS_DELETE", + "MISSION_VIEW", "MISSION_UPDATE", "MISSION_ADD", "MISSION_DELETE", "MISSION_UPLOAD", + "CDLC_VIEW", "CDLC_UPDATE", + "WORKSHOP_VIEW", "WORKSHOP_INSTALL", + "STEAM_SETTINGS_UPDATE", + "GENERAL_SETTINGS_VIEW", "GENERAL_SETTINGS_UPDATE", + "NETWORK_SETTINGS_VIEW", "NETWORK_SETTINGS_UPDATE", + "LOGGING_SETTINGS_VIEW", "LOGGING_SETTINGS_UPDATE", + "SECURITY_SETTINGS_VIEW", "SECURITY_SETTINGS_UPDATE", + "DIFFICULTY_VIEW", "DIFFICULTY_UPDATE", + "USERS_VIEW", "USERS_UPDATE", "USERS_DELETE", + "USER_MANAGEMENT_VIEW", + "SERVER_START", "SERVER_STOP", +] + +@pytest.fixture +def auth_headers(): + payload = { + "sub": "testuser", + "authorities": ALL_AUTHORITIES, + "iat": 1700000000, + "exp": 9999999999, + } + token = jwt.encode(payload, "test-secret", algorithm="HS256") + return {"Authorization": f"Bearer {token}"} + +# --- ASGI test client --- +@pytest.fixture +async def client(): + async with AsyncClient( + transport=ASGITransport(app=app), base_url="http://test" + ) as ac: + yield ac +``` + +--- + +### Unit Tests + +#### test_cfg_parser.py — CRITICAL (Phase 3 checklist) + +Six round-trip tests for `src/domain/server/config/server_config_parser.py`. Each test: +1. Parses a minimal `server.cfg` string +2. Serialises it back to string +3. Asserts the key fields survive the round-trip unchanged + +```python +# tests/unit/test_cfg_parser.py +import pytest +from src.domain.server.config.server_config_parser import ( + ServerConfigParser, ServerConfig +) + +MINIMAL_CFG = """ +hostname = "Test Server"; +maxPlayers = 20; +password = ""; +passwordAdmin = "admin"; +""" + +def test_hostname_round_trip(): + cfg = ServerConfigParser.parse(MINIMAL_CFG) + out = ServerConfigParser.serialize(cfg) + assert 'hostname = "Test Server"' in out + +def test_max_players_round_trip(): + cfg = ServerConfigParser.parse(MINIMAL_CFG) + assert cfg.max_players == 20 + out = ServerConfigParser.serialize(cfg) + assert "maxPlayers = 20" in out + +def test_empty_password_preserved(): + cfg = ServerConfigParser.parse(MINIMAL_CFG) + assert cfg.password == "" + out = ServerConfigParser.serialize(cfg) + assert 'password = ""' in out + +def test_admin_password_round_trip(): + cfg = ServerConfigParser.parse(MINIMAL_CFG) + assert cfg.password_admin == "admin" + out = ServerConfigParser.serialize(cfg) + assert 'passwordAdmin = "admin"' in out + +MISSIONS_CFG = """ +class Missions { + class Mission1 { + template = "tdm_stratis.Stratis"; + difficulty = "Regular"; + }; +}; +""" + +def test_missions_block_round_trip(): + cfg = ServerConfigParser.parse(MISSIONS_CFG) + assert len(cfg.missions) == 1 + assert cfg.missions[0].template == "tdm_stratis.Stratis" + out = ServerConfigParser.serialize(cfg) + assert "tdm_stratis.Stratis" in out + +def test_empty_cfg_produces_valid_output(): + cfg = ServerConfig() + out = ServerConfigParser.serialize(cfg) + assert isinstance(out, str) + assert len(out) >= 0 +``` + +#### test_permissions.py + +```python +# tests/unit/test_permissions.py +import pytest +from src.security.permissions import Authority + +def test_authority_enum_has_all_37_values(): + assert len(Authority) == 37 + +def test_all_authorities_are_strings(): + for a in Authority: + assert isinstance(a.value, str) + +async def test_missing_authority_returns_403(client): + # Login endpoint: no auth required + resp = await client.get("/api/v1/users", headers={}) + assert resp.status_code == 401 +``` + +#### test_password.py + +```python +# tests/unit/test_password.py +from src.security.password import hash_password, verify_password + +def test_hash_is_not_plaintext(): + h = hash_password("secret") + assert h != "secret" + +def test_verify_correct_password(): + h = hash_password("secret") + assert verify_password("secret", h) is True + +def test_verify_wrong_password(): + h = hash_password("secret") + assert verify_password("wrong", h) is False +``` + +#### test_mod_file_storage.py + +```python +# tests/unit/test_mod_file_storage.py +import pytest +from pathlib import Path +from src.domain.server.mod.mod_file_storage import scan_mod_directories + +def test_scans_at_prefixed_dirs(tmp_path): + mods_dir = tmp_path / "mods" + (mods_dir / "@ace").mkdir(parents=True) + (mods_dir / "@cba").mkdir(parents=True) + (mods_dir / "not_a_mod").mkdir(parents=True) # no @ prefix — excluded + result = scan_mod_directories(tmp_path) + names = [r.mod_directory.directory_name for r in result] + assert "@ace" in names + assert "@cba" in names + assert "not_a_mod" not in names + +def test_reads_meta_cpp(tmp_path): + mod_dir = tmp_path / "mods" / "@ace" + mod_dir.mkdir(parents=True) + (mod_dir / "meta.cpp").write_text('publishedid = 463939057;\nname = "ACE3";') + result = scan_mod_directories(tmp_path) + assert result[0].workshop_file_id == 463939057 + assert result[0].name == "ACE3" + +def test_missing_meta_cpp_uses_defaults(tmp_path): + mod_dir = tmp_path / "mods" / "@mymod" + mod_dir.mkdir(parents=True) + result = scan_mod_directories(tmp_path) + assert result[0].workshop_file_id == 0 +``` + +--- + +### Integration Tests + +#### test_auth.py + +```python +# tests/integration/test_auth.py +import pytest + +async def test_login_returns_jwt(client, seed_user): + resp = await client.post("/api/v1/auth/token", + data={"username": "testuser", "password": "changeme"}) + assert resp.status_code == 200 + assert "token" in resp.json() + +async def test_login_wrong_password_returns_401(client, seed_user): + resp = await client.post("/api/v1/auth/token", + data={"username": "testuser", "password": "wrong"}) + assert resp.status_code == 401 + +async def test_protected_endpoint_without_token_returns_401(client): + resp = await client.get("/api/v1/users") + assert resp.status_code == 401 +``` + +#### test_users.py + +```python +# tests/integration/test_users.py +import pytest + +async def test_get_users(client, auth_headers, seed_user): + resp = await client.get("/api/v1/users", headers=auth_headers) + assert resp.status_code == 200 + users = resp.json() + assert isinstance(users, list) + assert any(u["username"] == "testuser" for u in users) + +async def test_create_user(client, auth_headers): + resp = await client.post("/api/v1/users", + headers=auth_headers, + json={"username": "newuser", "password": "pw123", + "enabled": True, "authorities": []}) + assert resp.status_code in (200, 201) + +async def test_delete_user(client, auth_headers, seed_user): + resp = await client.delete(f"/api/v1/users/{seed_user.id}", + headers=auth_headers) + assert resp.status_code == 200 +``` + +#### test_status.py + +```python +# tests/integration/test_status.py +import pytest +from unittest.mock import patch, AsyncMock + +async def test_get_status_returns_json(client, auth_headers): + with patch("src.domain.server.process.process_service.ProcessService.get_status", + new_callable=AsyncMock, return_value={"status": "OFFLINE"}): + resp = await client.get("/api/v1/status", headers=auth_headers) + assert resp.status_code == 200 + assert "status" in resp.json() + +async def test_actuator_health_no_auth(client): + resp = await client.get("/api/v1/actuator/health") + assert resp.status_code == 200 + assert resp.json()["status"] == "UP" +``` + +#### test_general_settings.py + +```python +# tests/integration/test_general_settings.py +import pytest +from unittest.mock import patch, MagicMock + +FAKE_CONFIG = { + "serverName": "Test", + "serverDirectory": "/tmp/arma", + "steamCmdPath": "", +} + +async def test_get_general_settings(client, auth_headers): + with patch("src.domain.server.config.server_config_storage.ServerConfigStorage.read", + return_value=MagicMock(**FAKE_CONFIG)): + resp = await client.get("/api/v1/general", headers=auth_headers) + assert resp.status_code == 200 + +async def test_update_general_settings(client, auth_headers): + with patch("src.domain.server.config.server_config_storage.ServerConfigStorage.write"): + resp = await client.post("/api/v1/general", + headers=auth_headers, + json=FAKE_CONFIG) + assert resp.status_code == 200 +``` + +#### test_mods.py + +```python +# tests/integration/test_mods.py +import pytest +from unittest.mock import patch, AsyncMock +from src.domain.server.mod.models import ModsCollection + +async def test_get_mods_returns_collection(client, auth_headers): + empty = ModsCollection(disabled_mods=[], enabled_mods=[], not_managed_mods=[]) + with patch("src.domain.server.mod.mod_service.ModService.get_mods_collection", + new_callable=AsyncMock, return_value=empty): + resp = await client.get("/api/v1/mods", headers=auth_headers) + assert resp.status_code == 200 + body = resp.json() + assert "disabledMods" in body + assert "enabledMods" in body + assert "notManagedMods" in body + +async def test_post_enabled_mods(client, auth_headers): + with patch("src.domain.server.mod.mod_service.ModService.save_enabled_mod_list", + new_callable=AsyncMock): + resp = await client.post("/api/v1/mods/enabled", + headers=auth_headers, + json={"mods": []}) + assert resp.status_code == 200 +``` + +#### test_workshop.py — uses respx for Steam API + +```python +# tests/integration/test_workshop.py +import pytest, respx +from httpx import Response + +STEAM_QUERY_URL = "https://api.steampowered.com/IPublishedFileService/QueryFiles/v1/" + +async def test_workshop_active_no_steamcmd(client, auth_headers): + resp = await client.get("/api/v1/workshop/active", headers=auth_headers) + assert resp.status_code == 200 + assert "active" in resp.json() + +@respx.mock +async def test_workshop_query_returns_mods(client, auth_headers, settings_override): + """settings_override is a fixture that sets steam_web_api_token to 'test-key'.""" + respx.post(STEAM_QUERY_URL).mock(return_value=Response(200, json={ + "response": { + "next_cursor": "*", + "publishedfiledetails": [ + { + "publishedfileid": "123456", + "title": "Test Mod", + "preview_url": "https://example.com/img.jpg", + "children": [], + } + ] + } + })) + resp = await client.post("/api/v1/workshop/query", + headers=auth_headers, + json={"cursor": "*", "searchText": ""}) + assert resp.status_code == 200 + body = resp.json() + assert "mods" in body +``` + +Note: add a `settings_override` conftest fixture that temporarily sets `settings.steam_web_api_token = "test-key"`. + +#### test_missions.py + +```python +# tests/integration/test_missions.py +import pytest +from unittest.mock import patch, AsyncMock +from src.domain.server.mission.models import Missions + +async def test_get_missions(client, auth_headers): + empty = Missions(disabled_missions=[], enabled_missions=[]) + with patch("src.domain.server.mission.mission_service.MissionService.get_missions", + new_callable=AsyncMock, return_value=empty): + resp = await client.get("/api/v1/missions", headers=auth_headers) + assert resp.status_code == 200 + body = resp.json() + assert "disabledMissions" in body + assert "enabledMissions" in body + +async def test_add_mission_template(client, auth_headers): + with patch("src.domain.server.mission.mission_service.MissionService.add_mission", + new_callable=AsyncMock): + resp = await client.post("/api/v1/missions/template", + headers=auth_headers, + json={"name": "TDM Stratis", + "template": "tdm_stratis.Stratis"}) + assert resp.status_code == 200 +``` + +#### test_cdlc.py + +```python +# tests/integration/test_cdlc.py +import pytest +from unittest.mock import patch, AsyncMock +from src.domain.server.cdlc.models import Cdlc + +async def test_get_cdlcs(client, auth_headers): + fake = [Cdlc(id=1, name="gm", enabled=False, file_exists=False)] + with patch("src.domain.server.cdlc.cdlc_service.CdlcService.find_all", + new_callable=AsyncMock, return_value=fake): + resp = await client.get("/api/v1/cdlc", headers=auth_headers) + assert resp.status_code == 200 + body = resp.json() + assert "cdlcs" in body + assert body["cdlcs"][0]["name"] == "gm" + +async def test_toggle_cdlc(client, auth_headers): + with patch("src.domain.server.cdlc.cdlc_service.CdlcService.toggle_cdlc", + new_callable=AsyncMock): + resp = await client.post("/api/v1/cdlc/1/toggle", headers=auth_headers) + assert resp.status_code == 200 +``` + +#### test_scheduler_jobs.py + +```python +# tests/integration/test_scheduler_jobs.py +import pytest +from unittest.mock import AsyncMock, patch + +async def test_jwt_cleaner_job_runs_without_error(): + """Directly calls the job coroutine — does not require the scheduler to be running.""" + from src.application.scheduler.scheduler import jwt_cleaner_job + with patch("src.repository.user.jwt_token_repository.JwtTokenRepository.delete_expired", + new_callable=AsyncMock): + await jwt_cleaner_job() # must not raise + +async def test_mission_scanner_job_runs_without_error(tmp_path): + from src.domain.server.mission.jobs import mission_scanner_job + with patch("src.domain.server.mission.jobs.settings") as s: + s.server_directory_path = str(tmp_path) + with patch("src.domain.server.mission.jobs.mission_repo") as repo: + repo.find_all = AsyncMock(return_value=[]) + repo.save = AsyncMock() + await mission_scanner_job() +``` + +--- + +### Error Response Shape Test + +```python +# tests/integration/test_error_handler.py +import pytest + +async def test_unknown_api_path_returns_server_error(client, auth_headers): + resp = await client.get("/api/v1/nonexistent", headers=auth_headers) + assert resp.status_code == 404 + +async def test_server_error_returns_code_field(client, auth_headers): + """Force a 500 by patching a service to raise.""" + from src.domain.server.mod.mod_service import ModService + with patch.object(ModService, "get_mods_collection", + side_effect=RuntimeError("boom")): + resp = await client.get("/api/v1/mods", headers=auth_headers) + assert resp.status_code == 500 + body = resp.json() + assert body["code"] == "SERVER_ERROR" + assert "message" in body +``` + +--- + +### SPA Redirect Test + +```python +# tests/integration/test_spa_redirect.py +import pytest + +async def test_non_api_path_returns_index_html(client): + """SPA middleware returns index.html for paths without file extension.""" + import os, tempfile + # Create a minimal static/index.html in the working dir for this test + os.makedirs("static", exist_ok=True) + with open("static/index.html", "w") as f: + f.write("ASWG") + resp = await client.get("/mods") + # Either 200 with HTML content or redirect — either way, NOT a 404 on /api + assert resp.status_code == 200 + +async def test_static_asset_not_redirected(client): + """Paths with file extensions (dots) must NOT be redirected to index.html.""" + resp = await client.get("/main.js") + # StaticFiles will return 404 if file absent — that's correct behaviour + assert resp.status_code != 200 or "ASWG" not in resp.text +``` + +--- + +## Completion Checklist + +- [ ] `pytest` runs without import errors on a clean checkout (after `pip install -r requirements.txt`) +- [ ] `tests/unit/test_cfg_parser.py` — all 6 round-trip tests pass +- [ ] `tests/unit/test_permissions.py` — 37-value enum check passes +- [ ] `tests/integration/test_auth.py` — login / 401 / token tests pass +- [ ] `tests/integration/test_mods.py` — collection shape and enabled POST pass +- [ ] `tests/integration/test_workshop.py` — `respx` mock returns paginated mods +- [ ] `tests/integration/test_error_handler.py` — 500 returns `{"code":"SERVER_ERROR",...}` +- [ ] Coverage: `pytest --cov=src --cov-fail-under=80` exits 0 +- [ ] No test imports production `.env` or writes to the real database +- [ ] `asyncio_mode = "auto"` in `pytest.ini` (no manual `@pytest.mark.asyncio` needed) + +--- + +## Coverage Exclusions + +Add to `.coveragerc` or `pyproject.toml` to avoid penalising untestable glue: + +```ini +[coverage:run] +omit = + src/main.py # ASGI wiring — tested implicitly by client fixture + src/alembic/* + tests/* +``` + +--- + +## Contract Satisfied + +All 17 routers, 3 middleware layers, global exception handler, WebSocket endpoint, and two actuator endpoints are exercised through the `AsyncClient(transport=ASGITransport(app=app))` fixture. The test suite constitutes a full regression suite equivalent to the Java integration tests.