Build a Wildfire Prevention System with a Compact VLM

• Sentinel-2 satellite images
• LFM2.5-VL-450M, a compact Vision-Language Model running directly on the satellite, so inference happens in orbit and only a lightweight JSON payload is downlinked to Earth.

​

Steps

• Problem framing
• System design
• Data collection and labeling
• Evaluation
• Fine-tuning

​

1. Problem framing

• RGB (B4-B3-B2): natural color. Useful for reading urban texture, terrain shape from shadows, and water bodies.
• SWIR (B12-B8-B4): shortwave infrared. Highlights vegetation moisture stress and dryness, the primary fuel indicator.

​

Example

1. A Sentinel-2 satellite flies over Attica (Greece) on 2024-08-01 and takes these 2 pictures.

   RGB	SWIR
   Attica, Greece. 2024-08-01	Attica, Greece. 2024-08-01

2. This image pair is passed to the Vision-Language Model, which has holistic scene understanding, not just pixel-level statistics, and the model extracts the following risk profile.
   {
     "risk_level": "high",
     "dry_vegetation_present": true,
     "urban_interface": true,
     "steep_terrain": true,
     "water_body_present": false,
     "image_quality_limited": false
   }
   
3. This payload is downlinked to ground control on Earth. As the image tile has high risk, the system sends an alert to local fire services. These can then take precautionary measures like ground patrol deployment or controlled burns to reduce available fuel.

​

2. System design

​

Design rationale

​

Proof of Concept (PoC)

• SimSat: a local Docker service that simulates a satellite orbit and serves real Sentinel-2 imagery from the AWS Element84 STAC catalog.
• predict.py: a lightweight Python watch loop that polls SimSat for the current position, fetches the images, and drives the inference pipeline.
• LFM2.5-VL-450M: the local model running via llama-server, playing the role of the on-board VLM.

​

Quickstart

1. Clone the SimSat repository:
   git clone https://github.com/DPhi-Space/SimSat.git
   cd SimSat
   
2. Start SimSat (keep it running in a separate terminal):
   docker compose up
   
3. Open the SimSat dashboard at http://localhost:8000, click Start, and verify the satellite position is moving.
4. Install Python dependencies:
   uv sync
   
5. Start the watch loop:
   # Watch all 22 locations
   uv run scripts/predict.py --backend local --model LiquidAI/LFM2.5-VL-450M-GGUF --quant Q8_0
   
   # Watch a single location
   uv run scripts/predict.py --backend local --model LiquidAI/LFM2.5-VL-450M-GGUF --quant Q8_0 --location attica_greece
   
6. Optionally, backfill historical predictions to seed the database before the live loop:
   # All locations, last 7 days
   uv run scripts/backfill.py --backend local --model LiquidAI/LFM2.5-VL-450M-GGUF --quant Q8_0 --days 7
   
   # Single location, last 90 days (builds a seasonal dataset)
   uv run scripts/backfill.py --backend local --model LiquidAI/LFM2.5-VL-450M-GGUF --quant Q8_0 --days 90 --location attica_greece
   
7. Once the database has predictions, launch the app:
   uv run streamlit run app/app.py
   

​

3. Data collection and labeling pipeline

export ANTHROPIC_API_KEY=sk-...

# All 22 locations, push final dataset to Hugging Face
uv run scripts/generate_samples.py \
  --start-date 2024-01-01 --end-date 2025-12-31 \
  --n-temporal-tiles 12 --n-spatial-tiles 4 \
  --test-ratio 0.2 --concurrency 3 \
  --hf-dataset your-username/wildfire-risk

# Only 1 region, Attica, Greece
uv run scripts/generate_samples.py \
    --start-date 2024-01-01 --end-date 2024-12-31 \
    --n-temporal-tiles 12 --n-spatial-tiles 4 \
    --test-ratio 0.2 --concurrency 3 \
    --location attica_greece

1. Creates a timestamped run directory under data/ (e.g., data/20260416_143052/).
2. Samples --n-temporal-tiles timestamps evenly spaced within [--start-date, --end-date] using bin-center placement, so timestamps are always in the interior of the window.
3. Builds a centered square grid of --n-spatial-tiles tiles around each location center, spaced --size-km apart.
4. Fetches the RGB and SWIR images in parallel from SimSat for each (spatial tile, timestamp) pair.
5. Saves rgb.png and swir.png to the tile subfolder.
6. Sends both images to claude-opus-4-6 for risk annotation, with automatic retry on rate-limit errors.
7. Saves the structured JSON output as annotation.json.
8. Assigns the tile to train/ or test/ based on a temporal cutoff. This prevents near-duplicate images (Sentinel-2 revisits every 5 days) from appearing on both sides of the split.

data/20260416_143052/
  train/
    attica_greece/
      s00_t00/
        rgb.png
        swir.png
        annotation.json
      s00_t01/
      s01_t00/
      ...
  test/
    attica_greece/
      s00_t09/
      ...

uv run scripts/check_samples.py                  # most recent run
uv run scripts/check_samples.py 20260416_143052  # specific run

uv run scripts/generate_samples.py \
  --start-date 2024-01-01 --end-date 2025-12-31 \
  --n-temporal-tiles 12 \
  --n-spatial-tiles 4 \
  --test-ratio 0.2 \
  --concurrency 4 \
  --hf-dataset your-hf-username/wildfire-prevention

​

4. Evaluation

# Evaluate Claude Opus 4.6 (sanity check)
uv run scripts/evaluate.py \
    --hf-dataset Paulescu/wildfire-prevention \
    --backend anthropic \
    --split test

# Evaluate LFM2.5-VL-450M-GGUF at Q8_0 quantization
uv run scripts/evaluate.py \
  --hf-dataset Paulescu/wildfire-prevention \
  --backend local \
  --model LiquidAI/LFM2.5-VL-450M-GGUF \
  --quant Q8_0 \
  --split test

• report.md: human-readable accuracy table
• results.json: per-sample records with the model’s actual predictions, ground truth, and per-field match results
• meta.json: run metadata (model, dataset, backend, split)

uv run streamlit run app/eval_compare.py

​

Results

• claude-opus-4-6 scores 0.99 overall, near-perfect across all fields. The result is not 100% due to non-determinism in token sampling.
• The base LFM2.5-VL-450M scores 0.38 overall: it produces valid JSON reliably but struggles with field accuracy, especially risk_level (0.08) and urban_interface (0.25). This is expected for a zero-shot compact model on a specialized task. Fine-tuning addresses this gap.

​

5. Fine-tuning

​

Step 1. Install leap-finetune

git clone https://github.com/LiquidAI/leap-finetune.git
cd leap-finetune && uv sync && cd ..

cd leap-finetune
uv run huggingface-cli login   # needed to pull the model and dataset
uv run python -m modal setup   # needed to launch the training job
cd ..

​

Step 2. Prepare the dataset

uv run scripts/prepare_wildfire.py --dataset Paulescu/wildfire-prevention --modal

​

Step 3. Prepare the configuration file

project_name: "wildfire-prevention"
model_name: "lfm2.5-VL-450M"
training_type: "vlm_sft"

dataset:
  ...

training_config:
  ...

peft_config:
  extends: "DEFAULT_VLM_LORA"
  use_peft: false

benchmarks:
  ...

modal:
  app_name: "wildfire-prevention"
  gpu: "H100:1"
  timeout: 7200
  output_volume: "wildfire-prevention"
  output_dir: "/outputs"
  detach: false

• Full fine-tuning, not LoRA (use_peft: false): we update both the multimodal projector and the full language model backbone. Satellite imagery is severely underrepresented in standard VLM pretraining data, so the projector needs to genuinely re-learn how to map multispectral patches into meaningful tokens. At 450M parameters, full fine-tuning fits on a single H100 without the memory pressure that motivates LoRA on larger models.
• Modal section: the modal block tells leap-finetune to run the training job on Modal’s serverless GPU platform rather than locally. It specifies the GPU type (H100:1), a timeout, and the Modal volume where the prepared dataset lives and where checkpoints are written.

​

Step 4. Kick off the fine-tuning

cd leap-finetune && uv run leap-finetune ../configs/wildfire_finetune_modal.yaml

​

Step 5. Retrieve the checkpoint

uv run modal volume ls wildfire-prevention /outputs/
uv run modal volume get wildfire-prevention /outputs/<run-name> ./outputs

​

Step 6. Quantize the model to GGUF

uv run scripts/quantize.py \
    --checkpoint ./outputs/<run-name>/<checkpoint> \
    --output ./outputs/lfm2.5-vl-wildfire-Q8_0.gguf

• --output sets the backbone path: the language model weights, quantized to Q8_0 by default.
• The mmproj (mmproj-lfm2.5-vl-wildfire-Q8_0.gguf) is written automatically to the same directory, with mmproj- prepended. It contains the vision tower and multimodal projector weights (always F16).

​

Step 7. (Optional) Push the GGUF pair to HuggingFace

uv run scripts/push_gguf_to_hf.py \
    --backbone ./outputs/lfm2.5-vl-wildfire-Q8_0.gguf \
    --mmproj ./outputs/mmproj-lfm2.5-vl-wildfire-Q8_0.gguf \
    --repo <your-hf-username>/wildfire-risk-detector

​

Step 8. Evaluate the fine-tuned model

# From local artifacts
uv run scripts/evaluate.py \
    --hf-dataset Paulescu/wildfire-prevention \
    --backend local \
    --model ./outputs/lfm2.5-vl-wildfire-Q8_0.gguf \
    --mmproj ./outputs/mmproj-lfm2.5-vl-wildfire-Q8_0.gguf \
    --split test

# From HF
uv run scripts/evaluate.py \
    --hf-dataset Paulescu/wildfire-prevention \
    --backend local \
    --model Paulescu/wildfire-risk-detector \
    --quant Q8_0 \
    --split test

​

Results

​

Need help?