Spatiotemporal Deep Learning for Wheat Rust Detection Using SAR–Optical Satellite Data

Overview

Wheat rust is hard to monitor at scale. The only public ground truth is district and village-level text in ICAR-IIWBR surveillance newsletters, not pixel labels. Labels are scarce, and a model can look accurate by learning a shortcut such as the date or the location. The project asks whether a model built for this data can match a re-implemented state-of-the-art benchmark using fewer labels, and it sets up the evaluation so the answer can be checked.

How it works

A staged batch pipeline, run by eight numbered scripts in scripts/:

1. Extract and normalize rust severity from newsletters, then geocode survey units.
2. Acquire Sentinel-1/2 scenes (Copernicus STAC, with a Google Earth Engine path for the pilot) and ERA5-Land weather.
3. Preprocess SAR and optical imagery, build parcels and a per-parcel feature store.
4. Build spatial and temporal splits, then train and evaluate.

A hard boundary separates fold-agnostic steps from fold-dependent ones. Wheat masks, phenology fits, neighbour references and normalization statistics are computed per fold from training data only, and tests/test_leakage.py checks this. The PACT model is a two-stage cross-modal transformer with continuous-time attention, a gated-attention multiple-instance pooling layer and an ordinal (CORN) severity head. Observations are never gap-filled; an availability mask is used instead.

inputs: Sentinel-1, Sentinel-2, ERA5-Land weather
labels: village-level, from ICAR-IIWBR surveillance newsletters
encoder: two-stage cross-modal transformer
time: thermal time (GDD) positional encoding
pooling: gated-attention multiple-instance learning
head: ordinal (CORN) severity
tracking: Hydra, MLflow, Optuna

Where it could break

Key features

• Irregular-sampling-native encoder with thermal-time (GDD) positional encoding.
• Multiple-instance learning over parcels, with per-parcel attention used as a risk map.
• Baselines B1-B6 and four shortcut probes (weather, geography, date, metadata).
• Ablations A1-A12 switched by config files.
• Pre-registered protocol, with amendments recorded as dated entries.
• Shortcut-audit verdict, paired bootstrap and McNemar tests, and a risk-map export (GeoPackage, GeoTIFF) that refuses to write without uncertainty and a reliability mask.

Highlights

• Built a 20,000-line Python package (src/pact, 19 subpackages) covering ground truth, SAR/optical preprocessing, phenology, splits, models, training and inference.
• Wrote 92 test files with 1,046 test functions, including leakage audits for each fold, run in GitHub Actions with ruff, mypy and pytest.
• Implemented a two-stage PACT transformer with gated MIL pooling and a CORN ordinal head, plus a re-implemented benchmark (B6) and four shortcut baselines.
• Locked the benchmark with a git tag and a hashed manifest. B6 averaged 0.49 and 0.47 balanced accuracy over 5 seeds under spatial and temporal holdout, against 0.79 and 0.83 for a date-only shortcut. With 24 bags the intervals are wide.
• Documented 28 architecture decisions and a frozen experiment protocol, and ran GPU experiments with Optuna search on rented A40 nodes.

Tech stack

• Language: Python 3.11
• ML: PyTorch, scikit-learn, XGBoost, Optuna
• Geospatial: rasterio, GeoPandas, shapely, pyproj, xarray, rioxarray, pystac-client, earthengine-api, cdsapi
• Data and config: pandas, PyArrow (Parquet), Hydra, OmegaConf, MLflow
• Deployment and compute: no deployed service; training ran on rented E2E A40 GPU nodes
• Tooling: uv, hatchling, pytest, ruff, mypy, pre-commit, GitHub Actions