Municipalities lab — machine learning on Italy's data

# 🧪 Municipalities lab

Machine learning on public data, **right in your browser**: pick a region,
group its municipalities by income, age and foreign residents, find the ones
that stand out, discover which indicators move together. The models
(scikit-learn) run on your device: the first run downloads the library
(~60 MB, one-time), then everything is instant and reactive.

:::::page{title="Groups" icon="🗺️"}

## The region's groups of municipalities

Two-digit region code (e.g. `01` Piedmont, `03` Lombardy, `05` Veneto,
`08` Emilia-Romagna, `09` Tuscany, `12` Lazio, `15` Campania, `19` Sicily).
Then move the slider: the clustering recomputes and the map recolors.

::input[reg]{value="08" placeholder="Region code (2 digits)"}

::od-query{into="comuni" sql="SELECT c.comune AS comune, c.geojson AS geojson, r.reddito_medio AS reddito, i.eta_media AS eta, i.perc_stranieri AS stranieri FROM istat_confini_comuni c JOIN mef_redditi r ON r.codice_istat = c.codice_istat JOIN istat_indicatori i ON i.codice_istat = c.codice_istat WHERE c.cod_reg = '{#reg}'" limit="2000"}

::range[k]{min="2" max="8" value="4" legend="Number of groups"}

::ml-cluster{data="comuni" features="reddito,eta,stranieri" k="#k" into="gruppi"}

:::map{path="gruppi" geojson="geojson" fill="cluster" height="26rem"}
**{comune}** — group {cluster} · income {reddito} € · average age {eta}
:::

Similar municipalities end up in the same group: K-means works on average
income, average age and percentage of foreign residents (standardized). The
map colors are the groups.

:::::

:::::page{title="Outliers" icon="🔍"}

## The municipalities that stand out

The Isolation Forest assigns each municipality an anomaly score (0-1):
high = the municipality doesn't resemble the others in the region. Click the
table headers to sort.

::ml-anomaly{data="comuni" features="reddito,eta,stranieri" contamination="0.05" into="anomali"}

:::table{path="anomali" headers="Municipality,Income €,Age,Foreigners %,Anomaly,Outlier" pagesize="10"}
{comune} | {reddito} | {eta} | {stranieri} | {anomalia} | {flag}
:::

:::::

:::::page{title="Correlations" icon="🔗"}

## What moves together

The Pearson correlation between the indicators of the region's municipalities:
`r` close to **1** = they grow together, close to **-1** = opposite, close to
**0** = independent.

::ml-correlate{data="comuni" features="reddito,eta,stranieri" into="correlazioni"}

:::table{path="correlazioni" headers="Indicator A,Indicator B,Correlation r" pagesize="10" search="false"}
{a} | {b} | {r}
:::

::chart-bar{data="correlazioni" x="a" y="r" height="14rem"}

:::::

:::::page{title="Predictions" icon="🎯"}

## How much income do age and foreigners "explain"?

A regression learns the relationship between the demographic indicators and
the average income of the region's municipalities, then **estimates** it for
each one: where the estimate misses the most, something the demographic data
alone can't explain is going on. The **R²** in the status tells you how much
of the income the model explains.

::ml-predict{data="comuni" features="eta,stranieri" target="reddito" model="forest" into="stime"}

Each point is a municipality: the horizontal axis is the **real** income, the
vertical one the model's **estimate**. The closer the points sit to the
diagonal, the better the model explains.

::chart-scatter{data="stime" x="reddito" y="previsione" height="20rem"}

:::table{path="stime" headers="Municipality,Real income €,Model estimate €" pagesize="8"}
{comune} | {reddito} | {previsione}
:::

:::::

---

The models run **on your device** (scikit-learn via Pyodide): the data
downloaded from the open service stays in the browser and none of your data
leaves it. The sources are official: MEF **IRPEF incomes** (CC BY 3.0), ISTAT
**socio-demographic indicators** and **boundaries** (CC BY 4.0), joined on
their own via the municipality's ISTAT code.