# From samples to a land use map

How do labelled samples become a map of land use?

One window of the Brazilian Cerrado, classified from labelled samples of one region. Follow how sits trained a multilayer perceptron on them and how many of the samples held back it labelled correctly. Choose the input, time series or embeddings, and every step follows it.

This text follows the time series, the page's default input. The same page with the embeddings is at [/classification?input=embeddings](/classification?input=embeddings).

Picture: Three beats. 1. The samples train the model. 5 labelled time series samples, one per class (Annual crop, Cerradão, Pasture, Sugarcane and Water), each a curve of NIR08 over 24 dates, go into a model. 2. The cube goes into the trained model. A stack of 4 monthly images of the window stands for it. 3. The model labels every pixel: the map. The land use map of the window, classified from time series.

In the sits book: [Machine learning algorithms for image time series](https://e-sensing.github.io/sitsbook/cl_machinelearning.html), [Map accuracy assessment](https://e-sensing.github.io/sitsbook/val_map.html) and [Building regional embeddings: a full example](https://e-sensing.github.io/sitsbook/emb_example.html).

Before this: [From a cube to labelled samples](/time-series.md).

## 1. What does the classifier learn from?

In the sits book: [Classifying EO embeddings for land use and land cover applications](https://e-sensing.github.io/sitsbook/emb_example.html#classifying-eo-embeddings-for-land-use-and-land-cover-applications).

This page has a small scope: the samples of one region, one window of 256 by 256 pixels and one classifier.

The pipeline kept 4,443 labelled samples of the Cerrado, in 11 classes. [Where a sample's label comes from](/time-series.md) is told in the previous journey.

The classifier receives each sample as a row of numbers with a label. The input you chose decides which numbers.

The samples are split, class by class, at random: 3,555 to train the classifier and 888 held back. The classifier does not see the held-back samples while it learns.

### The figure

The samples of the Cerrado as the classifier receives them, for the input you chose. The hatched part of each bar is held back.

One window of 256 × 256 pixels, one classifier. The samples are split at random, class by class: some train the classifier, the rest are held back and not seen while it learns.

| Class                 | Training | Held back | All |
| --------------------- | -------- | --------- | --- |
| Annual crop           | 400      | 100       | 500 |
| Cerradão              | 400      | 100       | 500 |
| Cerrado               | 400      | 100       | 500 |
| Mangrove              | 160      | 40        | 200 |
| Natural non-vegetated | 26       | 6         | 32  |
| Open cerrado          | 400      | 100       | 500 |
| Pasture               | 400      | 100       | 500 |
| Perennial crop        | 400      | 100       | 500 |
| Silviculture          | 400      | 100       | 500 |
| Sugarcane             | 400      | 100       | 500 |
| Water                 | 169      | 42        | 211 |

One sample: 6 bands × 24 months, 144 numbers in all, then its label.

The bands, one row of each: BLUE, GREEN, RED, NIR08, SWIR16, SWIR22.

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
4. Keep the regional sample IDs: The pipeline retains recorded optical series from the published sample file and their cached encodings. The original optical extraction is undocumented; both forms keep the same IDs.
5. Split the labelled samples: The pipeline splits the labelled samples once, class by class, under a fixed seed: most train a model, the rest are held out to score it. Each input uses the same split. The time series keep their six bands with sits_select().
6. Train a model on each input: sits_train() trains a sits_mlp() model on the training samples of each input, under a fixed seed, and stops early when the validation loss stops falling.
7. Score each model on held-out samples: sits_classify() labels the held-out samples with each model, and sits_accuracy() compares the labels it gives with the labels the samples have.
8. Classify the study window: sits_cube() opens the study window as a cube: the monthly images for the time series model, the embeddings for the other. sits_classify() runs each model on it and gives every pixel a probability for each class.
9. Smooth the probabilities: sits_smooth() smooths the probabilities of each pixel with those of its neighbours, with the defaults of sits. One run classifies, smooths and labels the window.
10. Label each pixel: sits_label_classification() gives each pixel the class of highest probability, once from the smoothed probabilities and once from the raw ones.
11. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Splits the drawn samples class by class, under a fixed seed, into training and held-out samples. `classification_split(samples_subset, 0.8, 20261011L)`
11. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
12. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. Opens the window's crops as a cube with sits_cube(). `encoder_window_cube(window_crops, list(source = "BDC", collection = "LANDSAT-OLI-16D"))`
15. The training and held-out samples of each input, the time series cut to six bands with sits_select(). `classification_samples(samples_subset, samples_encoded, classify_split_rows)`
16. Writes the list of classes, with their titles and colours, for the website. `site_write_json(samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), "../web/src/data/classes.json")`
17. Writes the classes, the samples of each in training and held out, and the window's size, for the website. `classification_export_shared(counts = classification_split_counts(classify_split_rows), labels = sits::sits_labels(samples_subset), size = rep(study_window$size, 2L), path = "../web/src/data/classification.json")`
18. Names the bands and months of the crops. `window_crop_layers(window_crops, study_window$source, study_window$collection, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))`
19. Trains a model with sits_train() and sits_mlp() on the training samples, on the CPU, under a fixed seed. `classification_train(classify_split_samples[["series"]][["train"]], 20261011L)`
20. Binds recorded optical and cached encoded rows by their complete identity. `samples_cohort_record(samples_subset, samples_encoded, list(bands = window_layers[["bands"]], choice_band = "NIR08", classes = samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), source = list( repo = "e-sensing/samples_cerrado", commit = "fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8", file = "labelled_samples_cerrado.parquet")))`
21. Labels the held-out samples with sits_classify() and scores the labels with sits_accuracy(). `classification_accuracy(classify_model_series[["model"]], classify_split_samples[["series"]][["heldout"]])`
22. Classifies the window with sits_classify(), smooths it with sits_smooth() and labels it with sits_label_classification(), on the CPU. `classification_window_run(classify_cube_series, classify_model_series[["model"]], "cache/classification/series", list(memsize = 4L, multicores = 2L))`
23. Checks exact storage in the declared band order. `samples_storage_order(samples_storage_check(labelled_sample_record[["optical"]], samples_grid_metadata(list(crops = window_crops), "cache/src/sits-78d26eb07370874192911854b48b60ebb1d668c8")), labelled_sample_record[["bands"]])`
24. Reads the probabilities and labels sits wrote for the window. `classification_window_read(classify_files_series, rep(study_window$size, 2L), sits::sits_labels(classify_model_series[["model"]]))`
25. Writes axes, counts and descriptors for the common recorded cohort. `site_write_json(samples_cohort_json(labelled_sample_record, labelled_sample_storage, list(source = list(repo = "e-sensing/samples_cerrado", commit = "fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8", file = "labelled_samples_cerrado.parquet"), choice_band = "NIR08", titles = list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated"), settings = list(samples = list(per_class = 500L, period_start = "2017-01-01", train_frac = 0.8), seed = 20261011L, encoder = list(repo = "e-sensing/encoders_cerrado", type = "model", commit = "fbcea998a4173854a6a7d7cbd3bbeae4496de0c5", files = list(lejepa = "ssl_lejepa_tcnn_model_2017_2024.rds"))))), "../web/src/data/labelled-samples.json")`
26. Writes the training record and the held-out accuracy of the model, and how its arrays are stored, for the website. `classification_export_input(input = "series", record = classify_model_series[["record"]], accuracy = classify_accuracy_series, values_per_sample = classification_values_per_sample(classify_split_samples[["series"]][["train"]]), path = "../web/src/data/classification-series.json", layout = classification_export_arrays_layout(classify_arrays_series, rep(study_window$size, 2L)), smoothing = classification_export_smooth_settings(classify_arrays_series[["labels"]]))`

The files:

- `classification.json`: SHA-256 `4ea04e68fdfa0adf801240f138617755152dea821e50d26adc9741c688a4bf0b`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classes.json`: SHA-256 `e479d0f03cfc2258efe34563f02adc3999d25150b0f991643153d439209b2145`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples.json`: SHA-256 `19c17c7964f2917f2ac50dcd32a7c7e37255c0f41b834b2c6e5ac8cfac84862c`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series.json`: SHA-256 `6835a431c2149c463feef6369e066978f2628ea9a6a624a8f5305ec30f7c950e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

## 2. What happens while a network learns?

Before the recorded run, a small network trains here, in your browser. Press Start: in each epoch it makes one pass over the 3,555 training samples, then labels every pixel of the window.

Each colour is one of the 11 classes. After each epoch the map is repainted with the class the network now gives each pixel.

The curve is the loss on the training samples after each epoch: how far the network's guesses are from their labels. It belongs to this sketch alone; the next step shows the run of sits.

### The figure

A small network training in your browser on the input you chose, with the window's map repainted after each epoch. A sketch for viewing only, not a result of sits.

**A small network, learning as you watch.** To show the map taking shape, a small neural network trains here in your browser on the same samples. It is a simple sketch for viewing only; the results in the next steps come from sits.

The figure shows the window's map, empty until the reader presses Start. After each epoch the network labels every pixel and the map is repainted. The classes stand in a legend beside the map, and under both a timeline draws the loss on the training samples after each epoch, with a dot on the epoch shown. Once the run has ended, the timeline can be dragged, or stepped with the arrow keys, to show the map of any epoch. Pause stops after the epoch under way, and Start again trains a new network.

One hidden layer of 64 units, trained on 3,555 samples for 30 epochs.

Each sample is a row of 144 values: 6 bands, 24 dates each.

The map has 128 × 128 pixels: one in 2 along each side of the 256 × 256 window.

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Fill the missing months: Where a pixel has no value in a month, sits fills one in on a straight line between the months around it. At either end of the series it repeats the nearest value.
4. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
5. Split the labelled samples: The pipeline splits the labelled samples once, class by class, under a fixed seed: most train a model, the rest are held out to score it. Each input uses the same split. The time series keep their six bands with sits_select().
6. Cut the values for the browser network: For the small network that trains on the page, the pipeline cuts the training samples and the window, on a coarser grid of pixels, to one byte per value, between quantiles of the training samples. No result of sits is in these files.
7. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Splits the drawn samples class by class, under a fixed seed, into training and held-out samples. `classification_split(samples_subset, 0.8, 20261011L)`
11. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
12. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. The training and held-out samples of each input, the time series cut to six bands with sits_select(). `classification_samples(samples_subset, samples_encoded, classify_split_rows)`
15. Writes the list of classes, with their titles and colours, for the website. `site_write_json(samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), "../web/src/data/classes.json")`
16. Names the bands and months of the crops. `window_crop_layers(window_crops, study_window$source, study_window$collection, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))`
17. Reads how sits scales the bands and marks missing values. `collection_band_values(study_window$source, study_window$collection, window_layers$bands)`
18. Reads the stored values of the crops, missing ones as NA. `window_read_raw(window_crops, window_layers$bands, window_layers$dates, band_scale, study_window$size)`
19. Fills the missing months along time with sits::impute_linear(), as sits does before the encoder. `window_fill_series(window_raw, band_scale)`
20. Cuts the training samples and the window to one byte per value, between quantiles of the training samples, for a small network that trains in the browser. `classification_live_data("series", classify_split_samples[["series"]][["train"]], list(window = list(values = window_series, bands = window_layers$bands, size = study_window$size), labels = sits::sits_labels(samples_subset), quantiles = c(0.02, 0.98), stride = 2L))`
21. Writes the training samples, their classes and the window in one byte per value, for the live step of the website. `classification_export_live(classify_live_series, "../web/src/data", "../web/public/data")`

The files:

- `classes.json`: SHA-256 `e479d0f03cfc2258efe34563f02adc3999d25150b0f991643153d439209b2145`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-live.json`: SHA-256 `d66746dee69d830a69c80e63389f8ec492bc98f0c24a14be027254162d876a04`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-live-train.bin`: SHA-256 `053952d7ca076be2daa889c6bc2717f8422453501d158755de7213d5eafada5a`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-live-window.bin`: SHA-256 `1d16187a875fea1085e5f9ffec6ee17d796243bb5f5a3e9e497c6bf2d552a964`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

Try it: [A small network learning on time series](/interactive-images/classifier.md)

## 3. How does sits train the network?

In the sits book: [Deep learning using multilayer perceptron](https://e-sensing.github.io/sitsbook/cl_machinelearning.html#deep-learning-using-multilayer-perceptron).

The book trains a multilayer perceptron on its labelled samples with `sits_train()`.

The book's code:

```
mlp_model <- sits_train(
    samples = samples_matogrosso_mod13q1, 
    ml_method = sits_mlp(
        optimizer        = torch::optim_adamw, 
        layers           = c(512, 512, 512),
        dropout_rates    = c(0.40, 0.30, 0.20),
        epochs           = 80,
        batch_size       = 64,
        verbose          = FALSE,
        validation_split = 0.2)
)
```

Where the sits book writes it: [cl_machinelearning.qmd, lines 477–487](https://github.com/e-sensing/sitsbook/blob/bc75e250e50bc6fb8222593973b8dfe3b692201f/cl_machinelearning.qmd#L477-L487).

`sits_train()` fits a multilayer perceptron to the 3,555 training samples. It was run once for the input you chose, with a fixed seed.

Each epoch is one pass over the training samples. The loss measures how far the network's guesses are from the labels, and in the recorded run the curve shows it falling as the network learns.

A part of the training samples is kept aside to validate. Training stops early when the loss on that part stops improving.

### The figure

The recorded run of sits_mlp() for the input you chose: its settings and the loss after each epoch.

The classifier is a multilayer perceptron trained with sits_train(). This is the recorded run; the loss is the network's error after each epoch.

Recorded run: 3,555 training samples, stopped after 64 of 150 epochs.

This run called sits_train(samples, sits_mlp(seed = seed)): it sets the seed, so every other setting is the default of sits_mlp() (sits source, R/sits_mlp.R, lines 104-120). The settings below are read from the run, not from the book's call.

| Setting          | Value         |
| ---------------- | ------------- |
| Layers           | 512, 512, 512 |
| Epochs, at most  | 150           |
| Batch size       | 128           |
| Validation split | 0.2           |

Loss on the part it trained on, the training samples without the validation part: 1.343 after epoch 1, 0.187 after epoch 64.

Loss on the validation part: 0.240 after epoch 64. Training stopped early, after epoch 64, with a patience of 20 epochs.

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
4. Split the labelled samples: The pipeline splits the labelled samples once, class by class, under a fixed seed: most train a model, the rest are held out to score it. Each input uses the same split. The time series keep their six bands with sits_select().
5. Train a model on each input: sits_train() trains a sits_mlp() model on the training samples of each input, under a fixed seed, and stops early when the validation loss stops falling.
6. Score each model on held-out samples: sits_classify() labels the held-out samples with each model, and sits_accuracy() compares the labels it gives with the labels the samples have.
7. Classify the study window: sits_cube() opens the study window as a cube: the monthly images for the time series model, the embeddings for the other. sits_classify() runs each model on it and gives every pixel a probability for each class.
8. Smooth the probabilities: sits_smooth() smooths the probabilities of each pixel with those of its neighbours, with the defaults of sits. One run classifies, smooths and labels the window.
9. Label each pixel: sits_label_classification() gives each pixel the class of highest probability, once from the smoothed probabilities and once from the raw ones.
10. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
8. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
9. Splits the drawn samples class by class, under a fixed seed, into training and held-out samples. `classification_split(samples_subset, 0.8, 20261011L)`
10. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
11. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
12. Opens the window's crops as a cube with sits_cube(). `encoder_window_cube(window_crops, list(source = "BDC", collection = "LANDSAT-OLI-16D"))`
13. The training and held-out samples of each input, the time series cut to six bands with sits_select(). `classification_samples(samples_subset, samples_encoded, classify_split_rows)`
14. Trains a model with sits_train() and sits_mlp() on the training samples, on the CPU, under a fixed seed. `classification_train(classify_split_samples[["series"]][["train"]], 20261011L)`
15. Labels the held-out samples with sits_classify() and scores the labels with sits_accuracy(). `classification_accuracy(classify_model_series[["model"]], classify_split_samples[["series"]][["heldout"]])`
16. Classifies the window with sits_classify(), smooths it with sits_smooth() and labels it with sits_label_classification(), on the CPU. `classification_window_run(classify_cube_series, classify_model_series[["model"]], "cache/classification/series", list(memsize = 4L, multicores = 2L))`
17. Reads the probabilities and labels sits wrote for the window. `classification_window_read(classify_files_series, rep(study_window$size, 2L), sits::sits_labels(classify_model_series[["model"]]))`
18. Writes the training record and the held-out accuracy of the model, and how its arrays are stored, for the website. `classification_export_input(input = "series", record = classify_model_series[["record"]], accuracy = classify_accuracy_series, values_per_sample = classification_values_per_sample(classify_split_samples[["series"]][["train"]]), path = "../web/src/data/classification-series.json", layout = classification_export_arrays_layout(classify_arrays_series, rep(study_window$size, 2L)), smoothing = classification_export_smooth_settings(classify_arrays_series[["labels"]]))`

The files:

- `classification-series.json`: SHA-256 `6835a431c2149c463feef6369e066978f2628ea9a6a624a8f5305ec30f7c950e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

## 4. How many held-back samples does it label correctly?

In the sits book: [Accuracy assessment of classified images](https://e-sensing.github.io/sitsbook/val_map.html#accuracy-assessment-of-classified-images).

The trained network labels the 888 samples held back. Their labels are known, so each answer can be checked.

Overall accuracy is the share of them that received their own label.

Per class, the producer's accuracy asks what share of the samples of a class were found. The user's accuracy asks what share of the samples given a class really belong to it.

The split is random within each class, not by place. A held-back sample may lie close to a training sample, so this accuracy says little about new regions.

### The figure

The held-back samples checked against their known labels, for the input you chose.

The trained network labels the held-back samples, whose labels are known. Overall accuracy is the share that received their own label. The producer's accuracy of a class is the share of its samples that were found; the user's accuracy is the share of the samples given the class that belong to it. Measured on the held-back samples, chosen at random within each class, not by place. A held-back sample may lie close to a training sample.

888 held-back samples, each with its known label.

Overall accuracy: 93.2%. Kappa 0.92.

| Class                 | Held back | Producer's accuracy | User's accuracy |
| --------------------- | --------- | ------------------- | --------------- |
| Annual crop           | 100       | 95%                 | 99%             |
| Cerradão              | 100       | 93%                 | 82%             |
| Cerrado               | 100       | 84%                 | 87%             |
| Mangrove              | 40        | 100%                | 98%             |
| Natural non-vegetated | 6         | 100%                | 100%            |
| Open cerrado          | 100       | 94%                 | 90%             |
| Pasture               | 100       | 90%                 | 96%             |
| Perennial crop        | 100       | 91%                 | 96%             |
| Silviculture          | 100       | 97%                 | 97%             |
| Sugarcane             | 100       | 96%                 | 97%             |
| Water                 | 42        | 100%                | 100%            |

Confusion matrix: each row is the class the network gave, each column the true class.

| Given class \ true class | Annual crop | Cerradão | Cerrado | Mangrove | Natural non-vegetated | Open cerrado | Pasture | Perennial crop | Silviculture | Sugarcane | Water |
| ------------------------ | ----------- | -------- | ------- | -------- | --------------------- | ------------ | ------- | -------------- | ------------ | --------- | ----- |
| Annual crop              | 95          | 0        | 0       | 0        | 0                     | 0            | 0       | 0              | 0            | 1         | 0     |
| Cerradão                 | 0           | 93       | 12      | 0        | 0                     | 0            | 0       | 8              | 1            | 0         | 0     |
| Cerrado                  | 0           | 3        | 84      | 0        | 0                     | 6            | 2       | 0              | 1            | 1         | 0     |
| Mangrove                 | 0           | 0        | 0       | 40       | 0                     | 0            | 0       | 0              | 1            | 0         | 0     |
| Natural non-vegetated    | 0           | 0        | 0       | 0        | 6                     | 0            | 0       | 0              | 0            | 0         | 0     |
| Open cerrado             | 0           | 3        | 2       | 0        | 0                     | 94           | 5       | 0              | 0            | 0         | 0     |
| Pasture                  | 2           | 0        | 0       | 0        | 0                     | 0            | 90      | 0              | 0            | 2         | 0     |
| Perennial crop           | 1           | 0        | 1       | 0        | 0                     | 0            | 2       | 91             | 0            | 0         | 0     |
| Silviculture             | 0           | 1        | 1       | 0        | 0                     | 0            | 0       | 1              | 97           | 0         | 0     |
| Sugarcane                | 2           | 0        | 0       | 0        | 0                     | 0            | 1       | 0              | 0            | 96        | 0     |
| Water                    | 0           | 0        | 0       | 0        | 0                     | 0            | 0       | 0              | 0            | 0         | 42    |

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
4. Split the labelled samples: The pipeline splits the labelled samples once, class by class, under a fixed seed: most train a model, the rest are held out to score it. Each input uses the same split. The time series keep their six bands with sits_select().
5. Train a model on each input: sits_train() trains a sits_mlp() model on the training samples of each input, under a fixed seed, and stops early when the validation loss stops falling.
6. Score each model on held-out samples: sits_classify() labels the held-out samples with each model, and sits_accuracy() compares the labels it gives with the labels the samples have.
7. Classify the study window: sits_cube() opens the study window as a cube: the monthly images for the time series model, the embeddings for the other. sits_classify() runs each model on it and gives every pixel a probability for each class.
8. Smooth the probabilities: sits_smooth() smooths the probabilities of each pixel with those of its neighbours, with the defaults of sits. One run classifies, smooths and labels the window.
9. Label each pixel: sits_label_classification() gives each pixel the class of highest probability, once from the smoothed probabilities and once from the raw ones.
10. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Splits the drawn samples class by class, under a fixed seed, into training and held-out samples. `classification_split(samples_subset, 0.8, 20261011L)`
11. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
12. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. Opens the window's crops as a cube with sits_cube(). `encoder_window_cube(window_crops, list(source = "BDC", collection = "LANDSAT-OLI-16D"))`
15. The training and held-out samples of each input, the time series cut to six bands with sits_select(). `classification_samples(samples_subset, samples_encoded, classify_split_rows)`
16. Writes the list of classes, with their titles and colours, for the website. `site_write_json(samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), "../web/src/data/classes.json")`
17. Trains a model with sits_train() and sits_mlp() on the training samples, on the CPU, under a fixed seed. `classification_train(classify_split_samples[["series"]][["train"]], 20261011L)`
18. Labels the held-out samples with sits_classify() and scores the labels with sits_accuracy(). `classification_accuracy(classify_model_series[["model"]], classify_split_samples[["series"]][["heldout"]])`
19. Classifies the window with sits_classify(), smooths it with sits_smooth() and labels it with sits_label_classification(), on the CPU. `classification_window_run(classify_cube_series, classify_model_series[["model"]], "cache/classification/series", list(memsize = 4L, multicores = 2L))`
20. Reads the probabilities and labels sits wrote for the window. `classification_window_read(classify_files_series, rep(study_window$size, 2L), sits::sits_labels(classify_model_series[["model"]]))`
21. Writes the training record and the held-out accuracy of the model, and how its arrays are stored, for the website. `classification_export_input(input = "series", record = classify_model_series[["record"]], accuracy = classify_accuracy_series, values_per_sample = classification_values_per_sample(classify_split_samples[["series"]][["train"]]), path = "../web/src/data/classification-series.json", layout = classification_export_arrays_layout(classify_arrays_series, rep(study_window$size, 2L)), smoothing = classification_export_smooth_settings(classify_arrays_series[["labels"]]))`

The files:

- `classes.json`: SHA-256 `e479d0f03cfc2258efe34563f02adc3999d25150b0f991643153d439209b2145`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series.json`: SHA-256 `6835a431c2149c463feef6369e066978f2628ea9a6a624a8f5305ec30f7c950e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

## 5. How likely is each class, pixel by pixel?

Once trained, the network is applied to the whole window. For each pixel, `sits_classify()` gives the probability of each class. The map shows one class: the darker a pixel, the higher its probability of that class.

Choose another class to see its probabilities over the same window.

Select a pixel of the window to see its probability for every class as a bar.

The values of a pixel add up to about 1, as probabilities do. The file keeps each one rounded to one of 256 steps, so the bars of a pixel add up to about 1, not exactly.

### The figure

The probability of each class for the pixels of the window, for the input you chose.

sits_classify() gives each pixel of the window a probability for each class. The file holds them.

classification-series-probs.bin holds 11 planes of 256 × 256 pixels, one for each class, with 721 kB in all.

The map shows one class at a time as a grey ramp, white for a probability of 0 and black for 1. A button for each class chooses which one.

Selecting a pixel shows its probability for every class as a bar. Select a pixel to see its probabilities.

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
4. Split the labelled samples: The pipeline splits the labelled samples once, class by class, under a fixed seed: most train a model, the rest are held out to score it. Each input uses the same split. The time series keep their six bands with sits_select().
5. Train a model on each input: sits_train() trains a sits_mlp() model on the training samples of each input, under a fixed seed, and stops early when the validation loss stops falling.
6. Classify the study window: sits_cube() opens the study window as a cube: the monthly images for the time series model, the embeddings for the other. sits_classify() runs each model on it and gives every pixel a probability for each class.
7. Smooth the probabilities: sits_smooth() smooths the probabilities of each pixel with those of its neighbours, with the defaults of sits. One run classifies, smooths and labels the window.
8. Label each pixel: sits_label_classification() gives each pixel the class of highest probability, once from the smoothed probabilities and once from the raw ones.
9. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Splits the drawn samples class by class, under a fixed seed, into training and held-out samples. `classification_split(samples_subset, 0.8, 20261011L)`
11. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
12. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. Opens the window's crops as a cube with sits_cube(). `encoder_window_cube(window_crops, list(source = "BDC", collection = "LANDSAT-OLI-16D"))`
15. The training and held-out samples of each input, the time series cut to six bands with sits_select(). `classification_samples(samples_subset, samples_encoded, classify_split_rows)`
16. Writes the list of classes, with their titles and colours, for the website. `site_write_json(samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), "../web/src/data/classes.json")`
17. Writes the classes, the samples of each in training and held out, and the window's size, for the website. `classification_export_shared(counts = classification_split_counts(classify_split_rows), labels = sits::sits_labels(samples_subset), size = rep(study_window$size, 2L), path = "../web/src/data/classification.json")`
18. Trains a model with sits_train() and sits_mlp() on the training samples, on the CPU, under a fixed seed. `classification_train(classify_split_samples[["series"]][["train"]], 20261011L)`
19. Classifies the window with sits_classify(), smooths it with sits_smooth() and labels it with sits_label_classification(), on the CPU. `classification_window_run(classify_cube_series, classify_model_series[["model"]], "cache/classification/series", list(memsize = 4L, multicores = 2L))`
20. Reads the probabilities and labels sits wrote for the window. `classification_window_read(classify_files_series, rep(study_window$size, 2L), sits::sits_labels(classify_model_series[["model"]]))`
21. Writes the window's probabilities, smoothed probabilities and both label maps, one byte per value, for the website. `classification_export_arrays(input = "series", read = classify_arrays_series, dir = "../web/public/data")`

The files:

- `classification.json`: SHA-256 `4ea04e68fdfa0adf801240f138617755152dea821e50d26adc9741c688a4bf0b`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classes.json`: SHA-256 `e479d0f03cfc2258efe34563f02adc3999d25150b0f991643153d439209b2145`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-probs.bin`: SHA-256 `56353625eee731888bdce8608e5e62420d18e5a5d09fa234ed07d4411eddd88a`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

## 6. What does smoothing change?

`sits_smooth()` applies Bayesian smoothing to the probabilities, using the neighbourhood of each pixel. The map shows the smoothed probability of one class, on the same scale as before. The pipeline kept the defaults of `sits_smooth()`.

Choose another class to see its smoothed probabilities over the window.

A pixel is labelled with the class of highest probability. The pixels marked on the map are those whose label differs between the map made from the raw probabilities and the map made from the smoothed ones.

### The figure

The smoothed probability of one class over the window, for the input you chose, and the pixels whose label changed.

sits_smooth() smooths the probabilities of the window. Bayesian smoothing with a window of 9 × 9 pixels and a smoothness of 20, the defaults of sits_smooth().

classification-series-smooth.bin holds 11 planes of 256 × 256 pixels, one for each class, with 721 kB in all.

classification-series-map-raw.bin and classification-series-map.bin hold one class code for each of the 65,536 pixels, from the raw and from the smoothed probabilities.

The map shows the smoothed probability of one class as a grey ramp, white for 0 and black for 1. A button for each class chooses which one.

The pixels whose class differs between the two maps are marked, with their count. A pixel counts when its class differs between the two maps. A pixel with no class in both does not.

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
4. Split the labelled samples: The pipeline splits the labelled samples once, class by class, under a fixed seed: most train a model, the rest are held out to score it. Each input uses the same split. The time series keep their six bands with sits_select().
5. Train a model on each input: sits_train() trains a sits_mlp() model on the training samples of each input, under a fixed seed, and stops early when the validation loss stops falling.
6. Score each model on held-out samples: sits_classify() labels the held-out samples with each model, and sits_accuracy() compares the labels it gives with the labels the samples have.
7. Classify the study window: sits_cube() opens the study window as a cube: the monthly images for the time series model, the embeddings for the other. sits_classify() runs each model on it and gives every pixel a probability for each class.
8. Smooth the probabilities: sits_smooth() smooths the probabilities of each pixel with those of its neighbours, with the defaults of sits. One run classifies, smooths and labels the window.
9. Label each pixel: sits_label_classification() gives each pixel the class of highest probability, once from the smoothed probabilities and once from the raw ones.
10. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Splits the drawn samples class by class, under a fixed seed, into training and held-out samples. `classification_split(samples_subset, 0.8, 20261011L)`
11. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
12. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. Opens the window's crops as a cube with sits_cube(). `encoder_window_cube(window_crops, list(source = "BDC", collection = "LANDSAT-OLI-16D"))`
15. The training and held-out samples of each input, the time series cut to six bands with sits_select(). `classification_samples(samples_subset, samples_encoded, classify_split_rows)`
16. Writes the list of classes, with their titles and colours, for the website. `site_write_json(samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), "../web/src/data/classes.json")`
17. Writes the classes, the samples of each in training and held out, and the window's size, for the website. `classification_export_shared(counts = classification_split_counts(classify_split_rows), labels = sits::sits_labels(samples_subset), size = rep(study_window$size, 2L), path = "../web/src/data/classification.json")`
18. Trains a model with sits_train() and sits_mlp() on the training samples, on the CPU, under a fixed seed. `classification_train(classify_split_samples[["series"]][["train"]], 20261011L)`
19. Labels the held-out samples with sits_classify() and scores the labels with sits_accuracy(). `classification_accuracy(classify_model_series[["model"]], classify_split_samples[["series"]][["heldout"]])`
20. Classifies the window with sits_classify(), smooths it with sits_smooth() and labels it with sits_label_classification(), on the CPU. `classification_window_run(classify_cube_series, classify_model_series[["model"]], "cache/classification/series", list(memsize = 4L, multicores = 2L))`
21. Reads the probabilities and labels sits wrote for the window. `classification_window_read(classify_files_series, rep(study_window$size, 2L), sits::sits_labels(classify_model_series[["model"]]))`
22. Writes the window's probabilities, smoothed probabilities and both label maps, one byte per value, for the website. `classification_export_arrays(input = "series", read = classify_arrays_series, dir = "../web/public/data")`
23. Writes the training record and the held-out accuracy of the model, and how its arrays are stored, for the website. `classification_export_input(input = "series", record = classify_model_series[["record"]], accuracy = classify_accuracy_series, values_per_sample = classification_values_per_sample(classify_split_samples[["series"]][["train"]]), path = "../web/src/data/classification-series.json", layout = classification_export_arrays_layout(classify_arrays_series, rep(study_window$size, 2L)), smoothing = classification_export_smooth_settings(classify_arrays_series[["labels"]]))`

The files:

- `classification.json`: SHA-256 `4ea04e68fdfa0adf801240f138617755152dea821e50d26adc9741c688a4bf0b`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classes.json`: SHA-256 `e479d0f03cfc2258efe34563f02adc3999d25150b0f991643153d439209b2145`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series.json`: SHA-256 `6835a431c2149c463feef6369e066978f2628ea9a6a624a8f5305ec30f7c950e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-smooth.bin`: SHA-256 `b47aad286fa6216c7002e7eae267b3231859974c13f71a261c8521b59430840e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-map-raw.bin`: SHA-256 `e6bd433ce6a25a8a9b074dbf537e9acf337825a0f63f005cb7bef42e69648e0a`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-map.bin`: SHA-256 `cc70c2b16d0d5d4d07d536e5bfba1e72f1160f6d4edb6fbd02c02322eaafd57f`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

## 7. What does the land use map look like?

After smoothing, every pixel keeps the class of highest probability. The map paints each pixel in the colour of its class, for the input you chose.

The legend lists the classes with the share of the labelled pixels that each one holds, counted from the map you see. The shares add up to 100 %.

### The figure

The land use map of the window for the input you chose, with the share of each class.

The land use map is the smoothed map.

classification-series-map.bin holds one class code for each of the 65,536 pixels of the window, 255 for a pixel with no label.

Each pixel of the window is painted in the colour of the class it was given after smoothing.

The legend gives the share of the labelled pixels that each class holds, counted from this map. The shares add up to 100 %.

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
4. Split the labelled samples: The pipeline splits the labelled samples once, class by class, under a fixed seed: most train a model, the rest are held out to score it. Each input uses the same split. The time series keep their six bands with sits_select().
5. Train a model on each input: sits_train() trains a sits_mlp() model on the training samples of each input, under a fixed seed, and stops early when the validation loss stops falling.
6. Classify the study window: sits_cube() opens the study window as a cube: the monthly images for the time series model, the embeddings for the other. sits_classify() runs each model on it and gives every pixel a probability for each class.
7. Smooth the probabilities: sits_smooth() smooths the probabilities of each pixel with those of its neighbours, with the defaults of sits. One run classifies, smooths and labels the window.
8. Label each pixel: sits_label_classification() gives each pixel the class of highest probability, once from the smoothed probabilities and once from the raw ones.
9. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Splits the drawn samples class by class, under a fixed seed, into training and held-out samples. `classification_split(samples_subset, 0.8, 20261011L)`
11. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
12. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. Opens the window's crops as a cube with sits_cube(). `encoder_window_cube(window_crops, list(source = "BDC", collection = "LANDSAT-OLI-16D"))`
15. The training and held-out samples of each input, the time series cut to six bands with sits_select(). `classification_samples(samples_subset, samples_encoded, classify_split_rows)`
16. Writes the list of classes, with their titles and colours, for the website. `site_write_json(samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), "../web/src/data/classes.json")`
17. Writes the classes, the samples of each in training and held out, and the window's size, for the website. `classification_export_shared(counts = classification_split_counts(classify_split_rows), labels = sits::sits_labels(samples_subset), size = rep(study_window$size, 2L), path = "../web/src/data/classification.json")`
18. Trains a model with sits_train() and sits_mlp() on the training samples, on the CPU, under a fixed seed. `classification_train(classify_split_samples[["series"]][["train"]], 20261011L)`
19. Classifies the window with sits_classify(), smooths it with sits_smooth() and labels it with sits_label_classification(), on the CPU. `classification_window_run(classify_cube_series, classify_model_series[["model"]], "cache/classification/series", list(memsize = 4L, multicores = 2L))`
20. Reads the probabilities and labels sits wrote for the window. `classification_window_read(classify_files_series, rep(study_window$size, 2L), sits::sits_labels(classify_model_series[["model"]]))`
21. Writes the window's probabilities, smoothed probabilities and both label maps, one byte per value, for the website. `classification_export_arrays(input = "series", read = classify_arrays_series, dir = "../web/public/data")`

The files:

- `classification.json`: SHA-256 `4ea04e68fdfa0adf801240f138617755152dea821e50d26adc9741c688a4bf0b`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classes.json`: SHA-256 `e479d0f03cfc2258efe34563f02adc3999d25150b0f991643153d439209b2145`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-map.bin`: SHA-256 `cc70c2b16d0d5d4d07d536e5bfba1e72f1160f6d4edb6fbd02c02322eaafd57f`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

Try it: [The land use map on the ground, from the time series](/interactive-images/map.md)

## Next

On this site: [From time series to a map of temporal variability](/temporal-variability.md). How does variation through time become a spatial pattern?

In the sits book: [Machine learning algorithms for image time series](https://e-sensing.github.io/sitsbook/cl_machinelearning.html), [Map accuracy assessment](https://e-sensing.github.io/sitsbook/val_map.html) and [Building regional embeddings: a full example](https://e-sensing.github.io/sitsbook/emb_example.html).