# Spatiotemporal Deep Learning for Wheat Rust Detection Using SAR–Optical Satellite Data

Detecting wheat rust from radar and optical satellite time series, with an evaluation built to catch shortcuts and leakage.

- Status: in-progress
- Started: 2026-08
- Stack: Python, PyTorch, Hydra, MLflow, Optuna, XGBoost, scikit-learn, pandas, PyArrow, rasterio, GeoPandas, xarray, Google Earth Engine, GitHub Actions, uv
- Page: https://www.architbhuta.space/projects/wheat-rust

## Overview

Wheat rust is hard to monitor at scale. The only public ground truth is district and village-level text in ICAR-IIWBR surveillance newsletters, not pixel labels. Labels are scarce, and a model can look accurate by learning a shortcut such as the date or the location. The project asks whether a model built for this data can match a re-implemented state-of-the-art benchmark using fewer labels, and it sets up the evaluation so the answer can be checked.

## How it works

A staged batch pipeline, run by eight numbered scripts in `scripts/`:

1. Extract and normalize rust severity from newsletters, then geocode survey units.
2. Acquire Sentinel-1/2 scenes (Copernicus STAC, with a Google Earth Engine path for the pilot) and ERA5-Land weather.
3. Preprocess SAR and optical imagery, build parcels and a per-parcel feature store.
4. Build spatial and temporal splits, then train and evaluate.

A hard boundary separates fold-agnostic steps from fold-dependent ones. Wheat masks, phenology fits, neighbour references and normalization statistics are computed per fold from training data only, and `tests/test_leakage.py` checks this. The PACT model is a two-stage cross-modal transformer with continuous-time attention, a gated-attention multiple-instance pooling layer and an ordinal (CORN) severity head. Observations are never gap-filled; an availability mask is used instead.

*[Image: Pipeline with fold boundary]*

```text filename="project.config"
inputs: Sentinel-1, Sentinel-2, ERA5-Land weather
labels: village-level, from ICAR-IIWBR surveillance newsletters
encoder: two-stage cross-modal transformer
time: thermal time (GDD) positional encoding
pooling: gated-attention multiple-instance learning
head: ordinal (CORN) severity
tracking: Hydra, MLflow, Optuna
```

## Where it could break

| Theme | Risk | Check |
| --- | --- | --- |
| leakage | Train/test leakage | Wheat masks, phenology fits, neighbour references and normalization statistics are computed per fold from training data only, and tests/test_leakage.py audits each fold. |
| shortcut learning | A model looks accurate by learning the date or the location | Four shortcut probes (weather, geography, date, metadata) and a shortcut-audit verdict. A date-only shortcut scored 0.79 and 0.83 balanced accuracy, against 0.49 and 0.47 for the re-implemented benchmark (B6) under spatial and temporal holdout. |
| data integrity | Missing satellite observations filled with invented values | Observations are never gap-filled. The model gets an availability mask instead. |
| evaluation | Conclusions drawn from too few samples | Paired bootstrap and McNemar tests, with intervals reported (they are wide with 24 bags). The benchmark is locked with a git tag and a hashed manifest. |
| reliability | A risk map shown without its uncertainty | The risk-map export (GeoPackage, GeoTIFF) refuses to write without uncertainty and a reliability mask. |

## Key features

- Irregular-sampling-native encoder with thermal-time (GDD) positional encoding.
- Multiple-instance learning over parcels, with per-parcel attention used as a risk map.
- Baselines B1-B6 and four shortcut probes (weather, geography, date, metadata).
- Ablations A1-A12 switched by config files.
- Pre-registered protocol, with amendments recorded as dated entries.
- Shortcut-audit verdict, paired bootstrap and McNemar tests, and a risk-map export (GeoPackage, GeoTIFF) that refuses to write without uncertainty and a reliability mask.

*[Image: Benchmark vs shortcut baselines]*

## Highlights

- Built a 20,000-line Python package (`src/pact`, 19 subpackages) covering ground truth, SAR/optical preprocessing, phenology, splits, models, training and inference.
- Wrote 92 test files with 1,046 test functions, including leakage audits for each fold, run in GitHub Actions with ruff, mypy and pytest.
- Implemented a two-stage PACT transformer with gated MIL pooling and a CORN ordinal head, plus a re-implemented benchmark (B6) and four shortcut baselines.
- Locked the benchmark with a git tag and a hashed manifest. B6 averaged 0.49 and 0.47 balanced accuracy over 5 seeds under spatial and temporal holdout, against 0.79 and 0.83 for a date-only shortcut. With 24 bags the intervals are wide.
- Documented 28 architecture decisions and a frozen experiment protocol, and ran GPU experiments with Optuna search on rented A40 nodes.

## Tech stack

- Language: Python 3.11
- ML: PyTorch, scikit-learn, XGBoost, Optuna
- Geospatial: rasterio, GeoPandas, shapely, pyproj, xarray, rioxarray, pystac-client, earthengine-api, cdsapi
- Data and config: pandas, PyArrow (Parquet), Hydra, OmegaConf, MLflow
- Deployment and compute: no deployed service; training ran on rented E2E A40 GPU nodes
- Tooling: uv, hatchling, pytest, ruff, mypy, pre-commit, GitHub Actions