valheim-map/README.md

# Nerdhalla Valheim World Map

> **Seed:** `yzZ5fr2tGa` | **World:** 24000 × 24000 | **Biomes:** 9 | **POIs:** 11,309

An interactive web map of the Nerdhalla Valheim world, built from a `valheim-map.world` export. Includes biome/height map tiles, a searchable POI database, and a roadmap for fog-of-war integration.

---

## Quick Start

```bash
# Serve the map viewer
cd ~/projects/valheim-map/output && python3 -m http.server 8765
# Open: http://72.60.69.120:8765/index.html
```

**Live:** http://72.60.69.120:8765/index.html (port 8765, ufw allowed)

---

## Project Structure

```
~/projects/valheim-map/
├── src/                    # Source scripts
│   ├── valheim_db.py              # Phase 1: Build POI SQLite database from locations.json
│   ├── decode_valheim_tiles.py    # Phase 2: Decode .bin.gz tiles → PNG images
│   ├── build_viewer.py            # Phase 2: Generate Leaflet.js web viewer
│   ├── parse_valheim_db.py        # Phase 3: Attempted .db parser (WIP)
│   └── parse_valheim_zpackage.py  # Phase 3: ZPackage format exploration (WIP)
├── data/                   # Data files (small)
│   ├── valheim_poi.db             # SQLite POI database (55 MB)
│   └── Nerdhalla.fwl              # World metadata file (499 bytes)
├── output/                 # Generated map assets
│   ├── index.html                 # Leaflet.js web viewer (5.4 MB)
│   ├── world_composite.png        # Full world map, biome+height (3.7 MB)
│   ├── world_biome.png            # Full world biome classification (374 KB)
│   ├── world_height.png           # Full world heightmap (2.4 MB)
│   ├── XX-YY_*.png                # Individual tile images (16 tiles × 3 layers)
│   └── explored.png              # (future) Explored/unexplored overlay
├── notes/                  # Reference notes
│   └── tile-format-readme.txt    # Original valheim-map.world tile format spec
└── README.md               # This file
```

**Large files not in repo** (stored in /tmp/):
- `/tmp/valheim_locations.json` — 52 MB raw location export
- `/tmp/Nerdhalla.db` — 85 MB world save from Nerdcade AMP container
- `/tmp/Nerdhalla.json` — 1.5 GB JSON conversion (valheim-save-tools)
- `/tmp/valheim_tiles/tiles/*.bin.gz` — 16 tile files, ~100 MB total

---

## Roadmap

### ✅ Phase 1 — POI Database (Complete)

**Script:** `src/valheim_db.py`

Parsed `locations.json` (52 MB) into a queryable SQLite database with spatial support.

**Tables:**
- `locations` — 11,309 entries (prefab name, category, coordinates, dungeon/loot flags)
- `important_contents` — 24,407 entries (veggisirs, surtling cores, treasure, spawners)
- `random_contents` — 457,617 entries (dungeon interior loot)
- `dungeon_components` — 49,338 entries (room layouts)

**Spatial queries work:**
- "Highest density of tar pits?" → Zone (1500, 3500) has 11 in 1000u radius
- "Most Surtling Cores?" → Crypt3 @ (2039, -1162) has 20 cores
- "Best base location for tar farming?" → (1210, -4159) — 5 tar pits in 200m, Plains biome

### ✅ Phase 2 — Web Map Viewer (Complete)

**Scripts:** `src/decode_valheim_tiles.py`, `src/build_viewer.py`

Decoded 16 tile files (4×4 grid, 1024×1024 samples each) into PNG images and built a Leaflet.js viewer.

**Features:**
- **3 map layers:** Composite (biome+height shading), Biome (9-color classification), Height (grayscale elevation)
- **11,309 POI markers** — color-coded by category, clickable with loot popups
- **Category legend** — toggle groups on/off (boss, dungeon, vegvisir, vendor, resource, etc.)
- **Search** — by prefab name or category
- **Coordinate display** — Valheim world coords in popups

**Layer toggle buttons** (bottom-left): Composite | Biome | Height

### ⏳ Phase 3 — Fog-of-War Overlay (Blocked)

**Goal:** Show explored vs unexplored areas on the map.

**Blocked by:** The explored map data is stored per-player in `.fch` character files on each player's local machine, NOT in the world `.db` file. The `.db` contains 1.8M ZDOs (world objects, terrain, structures) but no explored bitmap.

**What we have:**
- `Nerdhalla.db` (85 MB) — pulled from Nerdcade AMP container, parsed via valheim-save-tools
- `Nerdhalla.fwl` (499 bytes) — world settings (seed, presets, difficulty)
- ZPackage parser WIP in `src/parse_valheim_zpackage.py`

**What's needed:** Player `.fch` files from:
- Windows: `%USERPROFILE%\AppData\LocalLow\IronGate\Valheim\characters\`
- Linux: `~/.config/unity3d/IronGate/Valheim/characters/`

### 🔮 Phase 4 — Cartography Table Integration (Planned)

**Goal:** A shared map of truth for the server.

**Approach options (in order of preference):**

1. **ServerSideMap mod (BepInEx)** — Gold standard. Strips map data from client files, hosts on server in real-time. As you explore, fog clears for everyone. Pin sharing with dedup. Requires BepInEx on AMP Valheim instance + all players install the mod.

2. **Periodic merge script** — Collect `.fch` files from players, merge explored bitmaps (OR operation), deduplicate pins (same name + within 15m), write back clean files. Run on cron.

3. **Cartography table automation** — Lightweight BepInEx plugin that auto-triggers "Record Discoveries" near the table. Still manual but removes forgetfulness.

**Pin cleaning strategy:**
- Dedup by name + proximity (15m radius)
- "Last writer wins" for conflicting pins
- Ghost pin prevention: track pin origin, only remove if no active player has it

---

## Technical Details

### Tile Format

From the valheim-map.world export Readme:

```
struct MapSample {
    uint16_t biome;       // 2 bytes
    float height;         // 4 bytes
    float forestFactor;   // 4 bytes
}  // 10 bytes per sample
```

- 1024 × 1024 samples per tile = 10,485,760 bytes uncompressed
- 4 × 4 tiles = 16 tiles, 24000 × 24000 Valheim units
- Gzip compressed individually

**Biome enum:** None=0, Meadows=1, Swamp=2, Mountain=4, BlackForest=8, Plains=16, AshLands=32, DeepNorth=64, Ocean=256, Mistlands=512

### Coordinate Mapping

```
Valheim coords: (-12000, -12000) to (+12000, +12000)
Image coords:   (0, 0) to (4096, 4096)

pixel_x = (valheim_x + 12000) / 24000 * 4096
pixel_y = (valheim_z + 12000) / 24000 * 4096
```

### Zone System

From the [Valheim Wiki](https://valheim.fandom.com/wiki/Zones):
- Zones are 64m × 64m
- Generated: 9×9 around each player (288m radius)
- Loaded: 5×5 (160m radius)
- Active: 3×3 (96m radius)
- Current: the zone(s) containing the player

### Database Schema

```sql
-- Core locations table
CREATE TABLE locations (
    id INTEGER PRIMARY KEY,
    prefab_name TEXT NOT NULL,
    category TEXT NOT NULL,
    pos_x REAL, pos_y REAL, pos_z REAL,
    has_dungeon INTEGER DEFAULT 0,
    has_important INTEGER DEFAULT 0,
    has_random INTEGER DEFAULT 0
);

-- Important contents (veggisirs, cores, treasure)
CREATE TABLE important_contents (
    id INTEGER PRIMARY KEY,
    location_id INTEGER NOT NULL,
    friendly_name TEXT,
    count INTEGER,
    icon TEXT,
    FOREIGN KEY (location_id) REFERENCES locations(id)
);

-- Random contents (dungeon interior loot)
CREATE TABLE random_contents (
    id INTEGER PRIMARY KEY,
    location_id INTEGER NOT NULL,
    type TEXT, parent TEXT, name TEXT, interior INTEGER,
    FOREIGN KEY (location_id) REFERENCES locations(id)
);

-- Dungeon components (room layouts)
CREATE TABLE dungeon_components (
    id INTEGER PRIMARY KEY,
    location_id INTEGER NOT NULL,
    delta_x REAL, delta_y REAL, rotation_y REAL,
    parent_type TEXT, parent_name TEXT,
    FOREIGN KEY (location_id) REFERENCES locations(id)
);
```

### .db File Format (ZPackage)

The world `.db` file uses Unity's ZPackage binary serialization format:

```
int32:   world_version
double:  net_time
int64:   my_id
uint32:  next_uid
int32:   num_zdos
ZDO[]:   zone data objects (1,800,945 in Nerdhalla)
Zones:   zone management data
RandomEvent: random event state
SHA512:  64-byte hash
```

Each ZDO contains: uid, prefab hash, owner, data type, revision, key-value pairs (floats, vectors, strings, byte arrays), position, rotation.

**Current world version:** 37 (valheim-save-tools max: 34 — parses with warning)

---

## Usage Examples

### Query the POI database

```bash
sqlite3 ~/projects/valheim-map/data/valheim_poi.db

# Find all tar pits
SELECT pos_x, pos_z FROM locations WHERE prefab_name LIKE '%TarPit%';

# Find boss altars
SELECT prefab_name, pos_x, pos_z FROM locations WHERE category='boss';

# Find dungeons with most surtling cores
SELECT l.prefab_name, l.pos_x, l.pos_z, SUM(ic.count) as total
FROM locations l JOIN important_contents ic ON l.id = ic.location_id
WHERE ic.friendly_name LIKE '%SurtlingCore%'
GROUP BY l.id ORDER BY total DESC LIMIT 10;

# Find vegvisirs
SELECT ic.friendly_name, l.pos_x, l.pos_z
FROM locations l JOIN important_contents ic ON l.id = ic.location_id
WHERE ic.friendly_name LIKE '%Vegvisir%';
```

### Rebuild from scratch

```bash
# 1. Build POI database
python3 src/valheim_db.py

# 2. Decode tiles to PNG
python3 src/decode_valheim_tiles.py

# 3. Build web viewer
python3 src/build_viewer.py

# 4. Serve
cd output && python3 -m http.server 8765
```

---

## Data Sources

| Source | File | Size | Origin |
|--------|------|------|--------|
| valheim-map.world export | `locations.json` | 52 MB | ROG (E:\Downloads\MapData_yzZ5fr2tGa\) |
| valheim-map.world export | `tiles/*.bin.gz` | ~100 MB | ROG |
| AMP Valheim container | `Nerdhalla.db` | 85 MB | Nerdcade (`docker cp AMP_Nerdhalla01:/AMP/Valheim/896660/Saves/worlds_local/`) |
| AMP Valheim container | `Nerdhalla.fwl` | 499 B | Nerdcade |

---

## Future Work

- [ ] **Fog-of-war overlay** — get `.fch` files from players, extract explored bitmap
- [ ] **ServerSideMap mod** — install BepInEx on AMP Valheim, deploy shared map
- [ ] **Pin cleaning/syncing** — dedup, merge, ghost pin prevention
- [ ] **Auto-update cron** — periodically pull `.db` from Nerdcade, regenerate map
- [ ] **Forgejo repo** — host this project in a self-hosted git instance
- [ ] **FoundryVTT export** — convert POI data for tabletop use
- [ ] **Mobile-friendly** — responsive layout for phone viewing