Air-Gapped On-Premise Deployment

Target: Dell VXRail (VMware vSphere) at a Thai hospital/nursing home with no internet access. Region: Thailand (Asia/Bangkok, locale=th, currency=THB)

---

Table of Contents

1. Hardware Requirements
2. What to Buy
3. Software Prerequisites
4. Pre-Build Phase (internet-connected machine)
5. Transfer to Air-Gapped Environment
6. Install on VXRail / Air-Gapped Server
7. Run Seed Data
8. Post-Install Verification
9. Backup & Recovery
10. Scaling & HA
11. Troubleshooting

---

1. Hardware Requirements

Minimum (Small Hospital / Nursing Home, up to 200 beds)

Component	Spec	Purpose
CPU	16 cores (8C/16T)	15 microservices + MongoDB + NATS + PostgreSQL
RAM	64 GB	MongoDB (16GB), microservices (32GB), OS + overhead (16GB)
Storage	500 GB SSD (NVMe preferred)	OS + Docker images (~20GB), MongoDB data, PostgreSQL, IPFS files
Network	1 Gbps NIC	Internal LAN access

Recommended (General Hospital, 200-500 beds)

Component	Spec	Purpose
CPU	32 cores (16C/32T) or 2x 16C	Headroom for concurrent users, report generation
RAM	128 GB	MongoDB caching, more concurrent API requests
Storage	1 TB NVMe SSD + 2 TB HDD	SSD for databases, HDD for IPFS/file storage
Network	2x 1 Gbps NIC (bonded) or 10 Gbps	Redundancy + throughput
Backup	External NAS or USB drive	Nightly database dumps

Large (Multi-department Hospital, 500+ beds)

Component	Spec	Purpose
CPU	64 cores (2x 32C)	Run MongoDB replica set, multiple service replicas
RAM	256 GB	Full in-memory MongoDB working set
Storage	2 TB NVMe SSD (RAID 10) + 4 TB HDD	High IOPS for database, bulk storage for files
Network	10 Gbps	Medical imaging, high concurrency
Backup	Dedicated NAS with RAID	Continuous replication

Dell VXRail Sizing

VXRail Model	Fits	Notes
VxRail E660 (entry)	Small hospital	2x Intel Xeon Silver, 64-128GB, adequate for <200 beds
VxRail V670 (standard)	Recommended	2x Intel Xeon Gold, 128-256GB, good for 200-500 beds
VxRail P670 (performance)	Large hospital	2x Intel Xeon Platinum, 256-512GB, GPU-ready for AI

VM Allocation on VXRail:

VM	vCPU	RAM	Disk	OS
medos-app	12	48 GB	200 GB SSD	Ubuntu 22.04 LTS
medos-db	8	32 GB	500 GB SSD	Ubuntu 22.04 LTS
medos-files	4	8 GB	1 TB HDD	Ubuntu 22.04 LTS

Or single VM (simplest for small deployments):

VM	vCPU	RAM	Disk	OS
medos-all-in-one	16	64 GB	500 GB SSD	Ubuntu 22.04 LTS

---

2. What to Buy

Option A: Dell VXRail (VMware Hyperconverged)

If you’re buying a VXRail cluster:

Item	Qty	Est. Price (USD)	Notes
Dell VxRail E660 (2x Xeon Silver 4316, 128GB, 4x 960GB SSD)	1	$25,000-35,000	Entry node, sufficient for single hospital
Dell VxRail V670 (2x Xeon Gold 6338, 256GB, 6x 1.92TB SSD)	1	$45,000-65,000	Recommended for 200+ beds
VMware vSAN + vSphere license	1	Included with VXRail	Per-node licensing
10GbE switch (Dell S5248F-ON)	1	$3,000-5,000	If not using existing network
UPS (APC Smart-UPS 3000VA)	1	$1,500-2,500	Protect against power loss
External backup drive (8TB USB)	1	$150-300	Nightly backup target

Total VXRail: ~$30,000-70,000 depending on size.

Option B: Standalone Server (Non-VMware)

If you don’t need VXRail/VMware:

Item	Qty	Est. Price (USD)	Notes
Dell PowerEdge R760 (2x Xeon Gold 5416S, 128GB, 4x 960GB SSD)	1	$8,000-12,000	Direct Docker on bare metal
Ubuntu 22.04 LTS	1	Free	No VMware license needed
UPS (APC 1500VA)	1	$500-800
External backup drive	1	$150-300

Total Standalone: ~$9,000-13,000

Option C: Budget (Repurposed Hardware / Small Clinic)

Item	Spec	Est. Price
Any x86_64 server/workstation	8+ cores, 32GB+ RAM, 256GB+ SSD	$0-3,000
Ubuntu 22.04 LTS	Free	$0
Docker Engine	Free	$0

Total Budget: $0-3,000 (uses existing hardware)

---

3. Software Prerequisites

Install these on the air-gapped server (or pre-install before air-gapping):

Software	Version	Install Method
Ubuntu Server	22.04 LTS	ISO image
Docker Engine	24.x+	.deb packages (offline install)
Docker Compose	v2.x (plugin)	Included with Docker Engine
OpenSSL	3.x	Included with Ubuntu

No Node.js, Yarn, pnpm, or Java needed on the server — everything runs inside Docker containers.

---

4. Pre-Build Phase (Internet-Connected Machine)

Do all of this on a machine WITH internet (your laptop, a build server, etc.). The output is a single .tar.gz file you’ll transfer to the air-gapped server.

4a. Clone the repo

git clone <medos-ultra-repo-url> medOS-ultra
cd medOS-ultra

4b. Build all Docker images

cd infrastructure

# Build the all-in-one on-premise stack
docker compose -f docker-compose-onpremise.yml build

# This builds:
#   medos-web          (frontend)
#   medos-api          (all 13 NestJS microservices — same image, different SCOPE)
#   medos-eform        (e-form service)
#   reporting          (Spring Boot PDF service)
#   reverse-proxy      (nginx)
#   nats               (message broker)

4c. Pull all external images

# Pull images that aren't built locally
docker pull mongo:5
docker pull postgres:14.5
docker pull ipfs/go-ipfs:master
docker pull dpage/pgadmin4:6.10

4d. Save all images to a tar archive

# List all images used by the compose file
IMAGES=(
  "medos-web:latest"
  "medos-api:latest"
  "medos-eform:latest"
  "mongo:5"
  "postgres:14.5"
  "ipfs/go-ipfs:master"
  "dpage/pgadmin4:6.10"
)

# Add the locally-built images (check with docker images)
docker images --format "{{.Repository}}:{{.Tag}}" | grep -E "^(medos|ever|nats)" >> /tmp/image_list.txt

# Save ALL images to one tar file
echo "Saving Docker images..."
docker save \
  medos-web:latest \
  medos-api:latest \
  medos-eform:latest \
  mongo:5 \
  postgres:14.5 \
  "ipfs/go-ipfs:master" \
  "dpage/pgadmin4:6.10" \
  -o medos-images.tar

# Compress (saves ~40-60% space)
echo "Compressing..."
gzip medos-images.tar
ls -lh medos-images.tar.gz
# Expected size: ~3-5 GB

4e. Package everything for transfer

cd /path/to/medOS-ultra

# Create the transfer package
mkdir -p /tmp/medos-airgap-package

# Copy Docker images
cp infrastructure/medos-images.tar.gz /tmp/medos-airgap-package/

# Copy config files (no source code needed — it's all in the Docker images)
cp -r infrastructure/docker-compose-onpremise.yml /tmp/medos-airgap-package/
cp -r infrastructure/.env.onpremise /tmp/medos-airgap-package/
cp -r infrastructure/scripts/ /tmp/medos-airgap-package/scripts/
cp -r infrastructure/market-packs/ /tmp/medos-airgap-package/market-packs/
cp -r infrastructure/deploy.sh /tmp/medos-airgap-package/
cp -r docker/reverse-proxy/templates/onpremise.conf.template /tmp/medos-airgap-package/
cp -r docker/reverse-proxy/nginx.conf /tmp/medos-airgap-package/
cp -r docker/nats/nats-server.conf /tmp/medos-airgap-package/

# Generate SSL cert now (so it's ready)
bash infrastructure/scripts/generate-ssl-cert.sh medos.local
cp -r docker/reverse-proxy/certbot/ssl/live/onpremise /tmp/medos-airgap-package/ssl-cert/

# Create the final archive
cd /tmp
tar czf medos-airgap-package.tar.gz medos-airgap-package/
ls -lh medos-airgap-package.tar.gz
# Expected size: ~3-6 GB total

4f. What’s in the package

medos-airgap-package/
├── medos-images.tar.gz              ← All Docker images (~3-5 GB)
├── docker-compose-onpremise.yml     ← Compose file
├── .env.onpremise                   ← Environment template
├── onpremise.conf.template          ← Nginx config
├── nginx.conf                       ← Nginx base config
├── nats-server.conf                 ← NATS config
├── ssl-cert/                        ← Pre-generated SSL certificate
│   ├── fullchain.pem
│   └── privkey.pem
├── scripts/
│   ├── setup-onpremise.sh
│   ├── generate-ssl-cert.sh
│   └── mongo_setup.sh
├── market-packs/                    ← Seed data for all regions
│   ├── medos-japan/
│   ├── medos-philippines/
│   └── medos-thailand/
└── deploy.sh                        ← Deployment CLI

---

5. Transfer to Air-Gapped Environment

Choose your transfer method:

Method	Speed	Best For
USB 3.0 drive	~100 MB/s	Most common, ~60s for 6GB
USB-C/Thunderbolt SSD	~500 MB/s	Fastest portable
DVD/Blu-ray	Slow	Compliance requirements (write-once media)
Sneakernet LAN	Varies	Transfer to a staging machine first

# On USB drive
cp /tmp/medos-airgap-package.tar.gz /Volumes/USB_DRIVE/

---

6. Install on VXRail / Air-Gapped Server

6a. Prepare the VM

On VXRail vCenter, create a VM:

• OS: Ubuntu 22.04 LTS (install from ISO)
• CPU: 16 vCPU (minimum)
• RAM: 64 GB (minimum)
• Disk: 500 GB thin-provisioned (SSD datastore)
• Network: Assign static IP on hospital LAN

After Ubuntu install, install Docker (from .deb packages if fully air-gapped, or pre-install before disconnecting):

# If Docker was pre-installed or you have .deb packages:
sudo apt-get install -y docker-ce docker-ce-cli containerd.io docker-compose-plugin

# Add your user to docker group
sudo usermod -aG docker $USER
newgrp docker

6b. Copy and extract the package

# Mount USB / copy from transfer media
mkdir -p /opt/medos
cp /media/usb/medos-airgap-package.tar.gz /opt/medos/
cd /opt/medos
tar xzf medos-airgap-package.tar.gz
cd medos-airgap-package

6c. Load Docker images

echo "Loading Docker images (this takes 5-10 minutes)..."
gunzip -c medos-images.tar.gz | docker load

# Verify all images loaded
docker images
# Should show: medos-web, medos-api, medos-eform, mongo, postgres, ipfs, etc.

6d. Configure environment

# Thailand deployment
REGION="thailand"
HOSTNAME="medos.local"  # or use server IP like 192.168.1.100

# Create env file
cp .env.onpremise .env.onpremise.thailand

# Edit the env file
nano .env.onpremise.thailand
# Change:
#   BASE_URL=https://medos.local  (or https://192.168.1.100)
#   TZ=Asia/Bangkok
#   JWT_SECRET=<random string>
#   SOCKET_IO_TOKEN=<random string>

Generate secrets without internet:

# Generate random secrets
JWT_SECRET=$(head -c 32 /dev/urandom | xxd -p)
SOCKET_TOKEN=$(head -c 16 /dev/urandom | xxd -p)
echo "JWT_SECRET=$JWT_SECRET"
echo "SOCKET_IO_TOKEN=$SOCKET_TOKEN"
# Paste these into your .env file

6e. Set up SSL certificate

# Option 1: Use the pre-generated cert from the package
mkdir -p /opt/medos/ssl
cp -r ssl-cert/* /opt/medos/ssl/

# Option 2: Generate a new one on the server
openssl req -x509 -nodes -days 3650 \
  -newkey rsa:2048 \
  -keyout /opt/medos/ssl/privkey.pem \
  -out /opt/medos/ssl/fullchain.pem \
  -subj "/C=TH/ST=Bangkok/O=Hospital/CN=$HOSTNAME" \
  -addext "subjectAltName=DNS:$HOSTNAME,IP:$(hostname -I | awk '{print $1}')"

6f. Set up the nginx config

mkdir -p /opt/medos/nginx
cp onpremise.conf.template /opt/medos/nginx/default.conf.template
cp nginx.conf /opt/medos/nginx/nginx.conf

6g. Set up the compose file

The compose file needs paths adjusted for the air-gapped layout. Create a simplified version:

# Update volume paths in compose file to use /opt/medos paths
# The compose file references ../docker/ paths — we need to adjust for flat layout

# Create required directories
mkdir -p /opt/medos/nats
cp nats-server.conf /opt/medos/nats/

# Add hostname to /etc/hosts
echo "127.0.0.1  $HOSTNAME" | sudo tee -a /etc/hosts

6h. Start the system

cd /opt/medos/medos-airgap-package

# Start all services
docker compose \
  --env-file .env.onpremise.thailand \
  -f docker-compose-onpremise.yml \
  up -d

# Watch startup (takes 60-120 seconds)
docker compose -f docker-compose-onpremise.yml logs -f --tail=50
# Wait until you see "Gateway started" and "MongoDB replica set initialized"

# Check all services are running
docker compose -f docker-compose-onpremise.yml ps
# All should show "Up" or "Up (healthy)"

6i. Configure client machines

On each workstation that will access medOS:

# Add to C:\Windows\System32\drivers\etc\hosts (Windows)
# or /etc/hosts (Mac/Linux)
192.168.1.100  medos.local

Open browser: https://medos.local (accept self-signed certificate warning).

---

7. Run Seed Data

For air-gapped environments, seed data must be loaded directly into the databases.

Load SQL seeds into Supabase/PostgreSQL

If using Supabase (hosted or self-hosted):

# Connect to Supabase and run seed SQL files
for sql in market-packs/medos-$REGION/seed-*.sql; do
  echo "Applying: $sql"
  psql "$SUPABASE_DB_URL" -f "$sql"
done

If running PostgreSQL locally (e-form database):

# E-form seeds
docker exec -i $(docker ps -qf "name=eform-postgres") \
  psql -U eform-user -d eform < market-packs/medos-$REGION/seed-hospital-facility.sql

Load blood donor data

The TypeScript seed script requires Node.js. In air-gapped mode, run it inside a container:

# Run seed script inside the web container
docker exec -it $(docker ps -qf "name=web") \
  sh -c "cd /app && npx tsx /seed/seed-blood-donors.ts"

Or pre-run on the internet-connected build machine and export as SQL.

---

8. Post-Install Verification

# Check all containers are healthy
docker compose -f docker-compose-onpremise.yml ps

# Test API gateway
curl -k https://medos.local/api/health

# Test frontend
curl -k https://medos.local/ | head -20

# Check MongoDB replica set
docker exec -it $(docker ps -qf "name=mongo") mongosh --eval "rs.status()"

# Check disk usage
df -h
docker system df

Expected resource usage (idle):

Resource	Usage
CPU	5-15% (idle)
RAM	~20-30 GB
Disk	~15-25 GB (images + empty databases)

Under load (100 concurrent users):

Resource	Usage
CPU	30-60%
RAM	~40-50 GB
Disk I/O	Moderate (MongoDB queries)

---

9. Backup & Recovery

Daily Backup Script

Create /opt/medos/backup.sh:

#!/bin/bash
BACKUP_DIR="/opt/medos/backups/$(date +%Y%m%d)"
mkdir -p "$BACKUP_DIR"

echo "Backing up MongoDB..."
docker exec $(docker ps -qf "name=mongo") \
  mongodump --archive --gzip > "$BACKUP_DIR/mongo.archive.gz"

echo "Backing up PostgreSQL..."
docker exec $(docker ps -qf "name=eform-postgres") \
  pg_dump -U eform-user eform | gzip > "$BACKUP_DIR/postgres-eform.sql.gz"

echo "Backing up config..."
cp /opt/medos/medos-airgap-package/.env.onpremise.* "$BACKUP_DIR/"

echo "Backup complete: $BACKUP_DIR"
ls -lh "$BACKUP_DIR"

# Cleanup backups older than 30 days
find /opt/medos/backups -maxdepth 1 -type d -mtime +30 -exec rm -rf {} \;

Add to cron:

sudo crontab -e
# Add: 0 2 * * * /opt/medos/backup.sh >> /var/log/medos-backup.log 2>&1

Restore from Backup

# Restore MongoDB
cat /opt/medos/backups/20260413/mongo.archive.gz | \
  docker exec -i $(docker ps -qf "name=mongo") \
  mongorestore --archive --gzip --drop

# Restore PostgreSQL
gunzip -c /opt/medos/backups/20260413/postgres-eform.sql.gz | \
  docker exec -i $(docker ps -qf "name=eform-postgres") \
  psql -U eform-user -d eform

---

10. Scaling & HA

Single Server (Default)

Good for up to ~200-300 concurrent users. All services on one machine.

Two-Server Split

Server	Services	Spec
App Server	All microservices + nginx + NATS	16 vCPU, 48 GB RAM, 200 GB SSD
DB Server	MongoDB + PostgreSQL + IPFS	8 vCPU, 32 GB RAM, 500 GB SSD

Change MONGO_URI in compose to point to DB server IP.

Three-Server HA

Server	Services	Spec
App Server 1	Microservices (primary) + nginx	16 vCPU, 48 GB
App Server 2	Microservices (replica) + nginx	16 vCPU, 48 GB
DB Server	MongoDB (replica set) + PostgreSQL + IPFS	16 vCPU, 64 GB

Use a load balancer (HAProxy or hardware) in front of the two app servers.

---

11. Troubleshooting

Services not starting

# Check logs for a specific service
docker compose -f docker-compose-onpremise.yml logs api-gateway
docker compose -f docker-compose-onpremise.yml logs mongo

# Restart a specific service
docker compose -f docker-compose-onpremise.yml restart api-clinical

MongoDB replica set not initializing

# Manually initialize
docker exec -it $(docker ps -qf "name=mongo") mongosh --eval "rs.initiate()"

Out of memory

# Check which container uses most memory
docker stats --no-stream --format "table {{.Name}}\t{{.MemUsage}}\t{{.CPUPerc}}"

# Reduce memory: scale down non-critical services
docker compose -f docker-compose-onpremise.yml stop pgadmin

SSL certificate issues

# Regenerate certificate
openssl req -x509 -nodes -days 3650 -newkey rsa:2048 \
  -keyout /opt/medos/ssl/privkey.pem \
  -out /opt/medos/ssl/fullchain.pem \
  -subj "/CN=medos.local"

# Restart nginx
docker compose -f docker-compose-onpremise.yml restart reverse-proxy

Can’t connect from other machines on LAN

# Check firewall
sudo ufw status
sudo ufw allow 80/tcp
sudo ufw allow 443/tcp

# Verify server IP
ip addr show

# On client machine, add to hosts file:
# 192.168.1.100  medos.local

Updates in air-gapped environment

To update medOS on an air-gapped server:

1. Build new Docker images on internet-connected machine
2. Save images to .tar.gz (same as step 4d)
3. Transfer via USB
4. Load images: gunzip -c new-images.tar.gz | docker load
5. Restart: docker compose -f docker-compose-onpremise.yml up -d

Data volumes are preserved across updates.

---

Quick Reference Card

Start:    docker compose --env-file .env.onpremise.thailand -f docker-compose-onpremise.yml up -d
Stop:     docker compose -f docker-compose-onpremise.yml down
Restart:  docker compose -f docker-compose-onpremise.yml restart
Logs:     docker compose -f docker-compose-onpremise.yml logs -f
Status:   docker compose -f docker-compose-onpremise.yml ps
Backup:   /opt/medos/backup.sh
URL:      https://medos.local

---

Consensus Layer on Air-Gapped Sites (Tier D)

The optional consensus/anchoring node (consensus0, --profile consensus — see infrastructure/consensus/README.md) works air-gapped, with one honest caveat: a disconnected node cannot participate in cross-site QBFT consensus. It is not part of the 4-validator HA set until it can gossip.

Air-gapped mode of operation:

1. Include the consensus node image in the airgap bundle (CONSENSUS_NODE_IMAGE, default hyperledger/besu:26.5.0) when packaging with package-airgap.sh.
2. Run the node standalone (single-node genesis, its own key as the sole validator) for local anchoring and audit: CONSENSUS_ENABLED=true docker compose ... --profile consensus up -d
3. Batch-reconcile: export anchors periodically (same sneakernet channel as image updates / WORM exports) and replay them to the consortium chain when a link is available.

No firewall ask at all — this is tier D of the connectivity ladder in infrastructure/consensus/README.md.