Skip to content
All projects
2025

Multi-Source Sensor ML Pipeline: Pest Population Forecasting

We compared 11 models across count forecasting and catch-day detection on 245 trap-days from 5 traps. A re-analysis found that a key input leaked the answer; without it, no model beats simple baselines.

245

Trap-days

5

Traps

21

Catch days

11

Models compared

Question

Can a few weeks of trap counts plus local weather warn of new insect catches, and which kind of model does it best when catch days are rare (21 of 245 trap-days)?

Problem
Growers check insect traps to decide when to treat crops, and a missed catch costs more than a false alarm. Can a few weeks of trap counts plus local weather warn of new catches when catch days are rare (21 of 245 trap-days)?
My role
Team of 3 (Information Systems and Business Intelligence, Federico II). I designed the model comparison and evaluation and did the re-analysis; Farshad Farahtaj led preprocessing, exploratory analysis and feature engineering; Giovanni Montanile built the Power BI reports.
Result
A negative result. With a leaking input removed, the Random Forest classifier scores AUC 0.49 and F1 0.36 on 49 test trap-days (9 with a catch), against the original 0.92 and 0.667; the regression's mean error rises from 0.355 to 0.496, worse than predicting yesterday's count (0.34).

Honest limits

  • This was the only data available: 5 traps over 49 days (6 July to 23 August 2024), with 21 catch days in two bursts.
  • The original input "recent activity" was 1 on every catch day including the day itself, so it leaked the answer; the original scores cannot be used.
  • Synthetic catch days (SMOTE) made the classifier worse, not better.
Try it live

Runs right here on this page. It loads from Hugging Face, which may set its own cookies, and a sleeping demo takes 30 to 60 seconds to wake up.

Overview

Weather and trap counts from five field traps

Growers check insect traps to decide when to treat crops, and a missed catch costs more than a false alarm. This was the only data available for the university project: 5 traps over 49 days (6 July to 23 August 2024), 245 trap-days with average temperature, humidity and temperature range, and 21 catch days in two bursts.

The data was split by date: the first 196 rows for training and the last 49 for testing. We engineered lag, 3-day rolling-mean, recency and calendar features.

Methodology

Two tasks, 11 models, one time-ordered split

  • Count regression (6 models): a regression task (estimate the day's catch count) with ARIMAX, SARIMAX, Prophet, Random Forest, XGBoost and LightGBM.
  • Catch-day classification (5 models): whether any new insect is caught, with Random Forest, XGBoost, LightGBM, LSTM and GRU. Tree models were tuned with time-ordered cross-validation and class_weight='balanced'. The threshold search found no useful value (validation F1 was 0 at every threshold), so it stayed at the grid's lowest value, 0.10.
  • Model choice: we ran both as tournaments; the winner was picked on the same test window, so even the original scores were optimistic.
Re-analysis

A key input leaked the answer

The original classifier looked strong. When I re-checked the features, the input "recent activity" turned out to be 1 on every catch day, including the catch day itself, so it already contained the answer. It was the strongest feature in the model, and the regression used it too.

I re-ran both notebooks with that input removed, on the same split, the same search grid and the same threshold search, and compared the result with simple baselines.

Results

Without the leak, no model beats simple baselines

With the leaking input removed, the Random Forest classifier scores AUC 0.49 and F1 0.36 on 49 test trap-days (9 with a catch), against the original 0.92 and 0.667. The regression's mean error rises from 0.355 to 0.496 insects per trap-day, worse than simply predicting yesterday's count (0.34). Adding synthetic catch days to training (SMOTE) made the classifier worse, not better.

Catch-day classifier

49 test trap-days, 9 with a catch (higher is better)

ModelAUCF1
Original Random Forest (with the leaking input)Leaked, not a result0.920.667
Random Forest, leaking input removedHonest score0.490.36
Baseline: predict a catch every daySimple baseline0.50.31

An AUC near 0.5 means the model cannot tell catch days from other days. With the leak removed, the Random Forest ranks catch days no better than chance.

Daily count regression

Mean error in insects per trap-day, 50 test trap-days (lower is better)

ModelMean error
Original Random Forest (with the leaking input)Leaked, not a result0.355
Random Forest, leaking input removedHonest score0.496
Baseline: yesterday's count at the same trapSimple baseline0.34
Baseline: predict 0Simple baseline0.52

With the leak removed, the Random Forest's error is worse than simply repeating yesterday's count.

The honest conclusion: on 245 trap-days with 21 catch days, none of these models forecasts catches better than a simple baseline. This is a negative result, reported as one. Scripts and tables for the re-analysis are in the reanalysis folder on GitHub.

Live demo

The corrected dashboard

A Streamlit dashboard on Hugging Face with the data exploration, the corrected results next to the simple baselines, and what was withdrawn and why. The prediction form was removed, because no model beat the baselines.

Loads Pest forecasting dashboard from huggingface.co, which may set its own cookies. A sleeping demo takes 30 to 60 seconds to wake up.

Team

Who did what

Team of 3 (Information Systems and Business Intelligence, Federico II). I designed the model comparison and evaluation and did the re-analysis; Farshad Farahtaj led data preprocessing, exploratory analysis and feature engineering; Giovanni Montanile built the Power BI reports.

Tech stack

Frameworks and infrastructure

PythonScikit-learnXGBoostLightGBMStatsmodelsProphetTensorFlow/KerasStreamlit

Source code on GitHub.