Skip to main content

Backend Server

IDPS Backend Service will be set up by ASTRI to:

  • manage subscriptions of routers with Lightweight Adaptive IDPS installed
  • receive network flow meta from Lightweight Adaptive IDPS
  • generate new ids/firewall rules for Lightweight Adaptive IDPS

We will provide detailed services and ports list with the deliverable.

Architecture Diagram​

Backend Diagram

Hardware requirement​

  • 16 cores CPU
  • 32GB RAM
  • 1TB storage
  • Able to visit internet
  • With public IP for connections from router

Support​

First Tier support: Customer Support, directly facing the broadband subscribers

Second Tier support: Maintenance of server and router hardware/infrastructure, including OS update and networking

HKT: Provide Server Hardware

ASTRI: Deploy the Service

Maintenance​

ACW's infra team don't need to manage the services on server. They may need to restart the server if the services are not available.

Security measures enforced and maintained by HKT.

Updates​

ASTRI will carry out update on ASTRI's services

HKT will update the base OS

Service Reactivation​

Example Standard Operating Procedure (SOP):

  1. Monitoring liveliness of backend service (ASTRI will provide documentation, HKT can try with the Operation guide first)
  2. In case of outage, inform ASTRI and reboot machine. The services will start automatically upon reboot.
  3. Contact ASTRI for support for further issues

Design​

See Rules Generation

RESTful API Endpoints​

Diagram​

Health Check​

  • GET /health - Service health status

Event Logging​

  • POST /log/events - Accepts Suricata IDS events as JSON arrays or application/x-ndjson. Each record is stored with MAC scoping.
  • POST /log/cfm - Ingests Network Flow Meta (CFM) as JSON arrays or application/x-ndjson. Each record is stored with MAC scoping.

Rule Management​

  • GET /rules/last_updated - Returns IDS + firewall updated_at timestamps for the requesting router, each time the router calls the API will create an activation record
  • GET /rules/ids - Retrieves the latest IDS rules JSON for the router.
  • GET /rules/firewall - Retrieves the latest firewall rules JSON for the router.

All endpoints require X-ID header containing the router's MAC address for device identification.

Background automation​

  • Rule preprocess scheduler
    • Downloads the Suricata Emerging Threats archive (RULE_PREPROCESS_ARCHIVE_URL) and version stamp, extracts .rules files into RULE_DATA_DIR/rules, converts them to JSON and assigns weights.
    • Controlled by cron expression RULE_PREPROCESS_CRON_SCHEDULE (default weekly).
  • Adaptive rule generation service (usecases.RuleGenerationUseCase)
    • Keeps a priority queue of registered routers, triggers rule generation with compromised IP list, and writes IDS/firewall payloads to the router_rules table. Concurrency, sampling size, filenames, queue depth, and command timeout come from the RULE_GENERATION_* env vars.
    • Uses RULE_DATA_DIR for workspace directories (rulefile_json/, idps_rules/, rules/rules/compromised-ips.txt, etc.).
  • Data retention scheduler (jobs/data_retention_scheduler.go)
    • Prunes CFM rows and network events older than RETENTION_DAYS on the cron defined by RETENTION_CRON_SCHEDULE. Toggle with DATA_RETENTION_ENABLED.

CLI entry points​

Utility entry point for operational jobs that do not need the HTTP server.

Router access report​

Generate CSV reports of router access activity. See Device Activation

# Daily report for a window (end date optional):
docker exec -i idps-api-server ./cli routeraccessreport --type daily --start-date 2025-11-15 --end-date 2025-11-26 > daily.csv

# Monthly summary (unique access days per router within the calendar month):
docker exec -i idps-api-server ./cli routeraccessreport --type monthly --month 2025-11 > monthly.csv

Rule generation​

Trigger the adaptive rule-generation pipeline for a specific router MAC address.

docker exec -i idps-api-server ./cli rulegen --mac 00:11:22:33:44:55

Rule preprocess​

Run the Suricata rule preprocess job once without waiting for the scheduler.

docker exec -i idps-api-server ./cli rulepreprocess

Configuration reference​

Core service and database​

  • DB_HOST, DB_PORT, DB_NAME, DB_USER, DB_PASSWORD, DB_SSLMODE – PostgreSQL connection settings; migrations run automatically on boot.
  • DB_LOG_LEVEL – silent, error, warning, or info for GORM logging (warning default).

Rule generation​

  • RULE_GENERATION_SAMPLE_COUNT – number of Suricata rules to sample per router (default 550).
  • RULE_GENERATION_MAX_CONCURRENT – worker count for the generation queue (1 default).
  • RULE_GENERATION_QUEUE_CAPACITY – buffered queue depth (defaults to 64 or MaxConcurrent * 4).
  • RULE_GENERATION_COMMAND_TIMEOUT_SECONDS – wall-clock timeout for rule generation (default 300).
  • RULE_GENERATION_INTERVAL_SECONDS – how often each registered router should be revisited (default 24h).

Rule preprocess pipeline​

  • RULE_PREPROCESS_ARCHIVE_URL – download URL for the Suricata rules zip (https://rules.emergingthreats.net/open/suricata-7.0.11/emerging.rules.zip).
  • RULE_PREPROCESS_VERSION_URL – version stamp endpoint (https://rules.emergingthreats.net/version.txt).
  • RULE_PREPROCESS_CRON_SCHEDULE – cron spec executed by the scheduler (default 0 0 * * 0).

Data retention​

  • DATA_RETENTION_ENABLED – true to enable scheduled pruning.
  • RETENTION_DAYS – amount of telemetry to keep (default 30).
  • RETENTION_CRON_SCHEDULE – cron spec for the pruning job (default 0 0 * * 0).

Docker-specific helpers​

  • HOST_RULE_DATA_DIR – host path mounted into /opt/rules-data inside the API container (Compose-only).
  • POSTGRES_INIT_SQL_PATH – SQL file bind-mounted into the Postgres container for boot-time seed data (Compose-only).
  • POSTGRES_DATA_PATH – host path mounted into /var/lib/postgresql/data inside the Postgres container (Compose-only).
  • CADDY_DATA_PATH – host path mounted into /data inside the Caddy container (Compose-only).
  • CADDY_CONFIG_PATH – host path mounted into /etc/caddy inside the Caddy container (Compose-only).