# A small network learning as you watch

A small neural network trains in your browser on the time series or the embeddings of the labelled samples, as you choose, and the window's map is repainted after each epoch beside the loss.

[From step 2 of From samples to a land use map](/classification.md).

In the sits book: [Machine learning algorithms for image time series](https://e-sensing.github.io/sitsbook/cl_machinelearning.html) and [Building regional embeddings: a full example](https://e-sensing.github.io/sitsbook/emb_example.html).

Choose the time series or the embeddings, see the untrained network's guess, press Start and watch the map take shape, or change the share of samples, the hidden units or the learning rate and train again.

## The figure

A small network training in your browser on the time series, with the window's map repainted after each epoch. A sketch for viewing only, not a result of sits.

This text follows the time series, the image's default input. The same image with the embeddings is at [/interactive-images/classifier?input=embeddings](/interactive-images/classifier?input=embeddings).

**A small network, learning as you watch.** To show the map taking shape, a small neural network trains here in your browser on the same samples. It is a simple sketch for viewing only; the results in the next steps come from sits. One hidden layer of 64 units, trained on 3,555 samples for 30 epochs. Learning rate 0.003.

Before Start the map shows the untrained network's guess: weights drawn from a seed and one pass over the window, with no training. The reader may lower the share of the samples used, and change the hidden units and the learning rate; a change gives a fresh, untrained network.

After each epoch the network labels every pixel and the map is repainted. A curve shows the loss on the training samples after each epoch. Pause stops after the epoch under way, and Start again trains a new network.

Here the network trains on the time series.

Each sample is a row of 144 values: 6 bands, 24 dates each. The map has 128 × 128 pixels: one in 2 along each side of the 256 × 256 window.

Samples used: 1 % (37 samples), 2 % (71 samples), 10 % (356 samples), 25 % (889 samples) and 100 % (3,555 samples). Hidden units: 16, 32, 64 and 128. Learning rate: 0.001, 0.003, 0.01 and 0.03.

### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Fill the missing months: Where a pixel has no value in a month, sits fills one in on a straight line between the months around it. At either end of the series it repeats the nearest value.
4. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
5. Split the labelled samples: The pipeline splits the labelled samples once, class by class, under a fixed seed: most train a model, the rest are held out to score it. Each input uses the same split. The time series keep their six bands with sits_select().
6. Cut the values for the browser network: For the small network that trains on the page, the pipeline cuts the training samples and the window, on a coarser grid of pixels, to one byte per value, between quantiles of the training samples. No result of sits is in these files.
7. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Splits the drawn samples class by class, under a fixed seed, into training and held-out samples. `classification_split(samples_subset, 0.8, 20261011L)`
11. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
12. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. The training and held-out samples of each input, the time series cut to six bands with sits_select(). `classification_samples(samples_subset, samples_encoded, classify_split_rows)`
15. Writes the list of classes, with their titles and colours, for the website. `site_write_json(samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), "../web/src/data/classes.json")`
16. Names the bands and months of the crops. `window_crop_layers(window_crops, study_window$source, study_window$collection, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))`
17. Reads how sits scales the bands and marks missing values. `collection_band_values(study_window$source, study_window$collection, window_layers$bands)`
18. Reads the stored values of the crops, missing ones as NA. `window_read_raw(window_crops, window_layers$bands, window_layers$dates, band_scale, study_window$size)`
19. Fills the missing months along time with sits::impute_linear(), as sits does before the encoder. `window_fill_series(window_raw, band_scale)`
20. Cuts the training samples and the window to one byte per value, between quantiles of the training samples, for a small network that trains in the browser. `classification_live_data("series", classify_split_samples[["series"]][["train"]], list(window = list(values = window_series, bands = window_layers$bands, size = study_window$size), labels = sits::sits_labels(samples_subset), quantiles = c(0.02, 0.98), stride = 2L))`
21. Writes the training samples, their classes and the window in one byte per value, for the live step of the website. `classification_export_live(classify_live_series, "../web/src/data", "../web/public/data")`

The files:

- `classes.json`: SHA-256 `e479d0f03cfc2258efe34563f02adc3999d25150b0f991643153d439209b2145`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-live.json`: SHA-256 `d66746dee69d830a69c80e63389f8ec492bc98f0c24a14be027254162d876a04`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-live-train.bin`: SHA-256 `053952d7ca076be2daa889c6bc2717f8422453501d158755de7213d5eafada5a`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-live-window.bin`: SHA-256 `1d16187a875fea1085e5f9ffec6ee17d796243bb5f5a3e9e497c6bf2d552a964`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

## What to try

1. **Look at a guess.** Before Start the map is marked Untrained: the network's weights are drawn at random from a seed and nothing has been learned. What does the window look like when every pixel is a guess?
2. **Watch the first epochs.** Press Start and look at the map after one or two epochs. How much of the window has its colours already, and where on the curve does the loss fall fastest?
3. **Use fewer labels.** Set *Samples used* to 1 % and press Start. The line in the box says how many samples are left for all the classes; compare the map they give with the map from all of them.
4. **Change the network.** Change the hidden units or the learning rate, one at a time, and train again. Which change moves the loss curve most?
5. **Change the input.** Choose the other input above the figure. The map goes back to the untrained guess for that input, and Start trains a fresh network on it with the same settings.