# The land use map on the ground

The land use map classified from the time series or the embeddings, as you choose: the probability of each class, the strongest class with and without smoothing, and a divider to compare the map with the image of the window and, on the embeddings, with the embeddings too.

[From step 7 of From samples to a land use map](/classification.md).

In the sits book: [Machine learning algorithms for image time series](https://e-sensing.github.io/sitsbook/cl_machinelearning.html) and [Building regional embeddings: a full example](https://e-sensing.github.io/sitsbook/emb_example.html).

Choose the time series or the embeddings, step through the probability of each class, switch smoothing on and off, show the pixels whose label changed, and swipe between the image and the map, or, on the embeddings, the embeddings too.

## The figure

The probabilities sits_classify() gave for the time series, their smoothed scores, the labelled map, and the map beside the image of the window in one month.

This text follows the time series, the image's default input. The same image with the embeddings is at [/interactive-images/map?input=embeddings](/interactive-images/map?input=embeddings).

The land use map classified from the time series, in three views: one class at a time, the strongest class, and the window on the ground.

One class shows the probability of one class for each pixel, white for 0 and black for 1, with the bars of a selected pixel; with smoothing on, it shows the smoothed score of the class and marks the pixels whose label smoothing changed. Strongest class paints each pixel in the colour of its class of highest probability, from the raw or the smoothed probabilities, and marks the changed pixels on demand. On the ground compares the map with the image of the window under a divider.

Bayesian smoothing with a window of 9 × 9 pixels and a smoothness of 20, the defaults of sits_smooth().

Image: the window in May 2017, with SWIR16, NIR08 and RED as red, green and blue. Map: the land use map after smoothing, in the colours of the classes.

The pair: Image | Map. Drag the divider, or focus it and use the arrow keys, to compare the two pictures.

classification-series-map.bin holds one class code for each of the 65,536 pixels of the window, 255 for a pixel with no label.

Each pixel of the window is painted in the colour of the class it was given after smoothing.

The legend gives the share of the labelled pixels that each class holds, counted from this map. The shares add up to 100 %.

### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Fill the missing months: Where a pixel has no value in a month, sits fills one in on a straight line between the months around it. At either end of the series it repeats the nearest value.
4. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
5. Split the labelled samples: The pipeline splits the labelled samples once, class by class, under a fixed seed: most train a model, the rest are held out to score it. Each input uses the same split. The time series keep their six bands with sits_select().
6. Train a model on each input: sits_train() trains a sits_mlp() model on the training samples of each input, under a fixed seed, and stops early when the validation loss stops falling.
7. Score each model on held-out samples: sits_classify() labels the held-out samples with each model, and sits_accuracy() compares the labels it gives with the labels the samples have.
8. Classify the study window: sits_cube() opens the study window as a cube: the monthly images for the time series model, the embeddings for the other. sits_classify() runs each model on it and gives every pixel a probability for each class.
9. Smooth the probabilities: sits_smooth() smooths the probabilities of each pixel with those of its neighbours, with the defaults of sits. One run classifies, smooths and labels the window.
10. Label each pixel: sits_label_classification() gives each pixel the class of highest probability, once from the smoothed probabilities and once from the raw ones.
11. Prepare the pictures: The pipeline picks the brightness drawn black and white, the same for every date, so the dates can be compared. Values are packed into one byte each.
12. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Splits the drawn samples class by class, under a fixed seed, into training and held-out samples. `classification_split(samples_subset, 0.8, 20261011L)`
11. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
12. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. Opens the window's crops as a cube with sits_cube(). `encoder_window_cube(window_crops, list(source = "BDC", collection = "LANDSAT-OLI-16D"))`
15. The training and held-out samples of each input, the time series cut to six bands with sits_select(). `classification_samples(samples_subset, samples_encoded, classify_split_rows)`
16. Writes the list of classes, with their titles and colours, for the website. `site_write_json(samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), "../web/src/data/classes.json")`
17. Writes the classes, the samples of each in training and held out, and the window's size, for the website. `classification_export_shared(counts = classification_split_counts(classify_split_rows), labels = sits::sits_labels(samples_subset), size = rep(study_window$size, 2L), path = "../web/src/data/classification.json")`
18. Names the bands and months of the crops. `window_crop_layers(window_crops, study_window$source, study_window$collection, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))`
19. Reads how sits scales the bands and marks missing values. `collection_band_values(study_window$source, study_window$collection, window_layers$bands)`
20. Trains a model with sits_train() and sits_mlp() on the training samples, on the CPU, under a fixed seed. `classification_train(classify_split_samples[["series"]][["train"]], 20261011L)`
21. Labels the held-out samples with sits_classify() and scores the labels with sits_accuracy(). `classification_accuracy(classify_model_series[["model"]], classify_split_samples[["series"]][["heldout"]])`
22. Classifies the window with sits_classify(), smooths it with sits_smooth() and labels it with sits_label_classification(), on the CPU. `classification_window_run(classify_cube_series, classify_model_series[["model"]], "cache/classification/series", list(memsize = 4L, multicores = 2L))`
23. Reads the stored values of the crops, missing ones as NA. `window_read_raw(window_crops, window_layers$bands, window_layers$dates, band_scale, study_window$size)`
24. Reads the probabilities and labels sits wrote for the window. `classification_window_read(classify_files_series, rep(study_window$size, 2L), sits::sits_labels(classify_model_series[["model"]]))`
25. Marks the months each pixel was observed in. `window_observed_mask(window_raw)`
26. Fills the missing months along time with sits::impute_linear(), as sits does before the encoder. `window_fill_series(window_raw, band_scale)`
27. Stores every other pixel's series in one byte per value, each band between two quantiles. `window_export_series(window_series, c(0.001, 0.999 ), 2L)`
28. Picks the reflectance drawn black and white in each colour, the same for every month. `window_export_sprite_stretch(window_series, window_layers$bands, list(red = "SWIR16", green = "NIR08", blue = "RED"), c(0.02, 0.98))`
29. Writes the window's probabilities, smoothed probabilities and both label maps, one byte per value, for the website. `classification_export_arrays(input = "series", read = classify_arrays_series, dir = "../web/public/data")`
30. Writes the training record and the held-out accuracy of the model, and how its arrays are stored, for the website. `classification_export_input(input = "series", record = classify_model_series[["record"]], accuracy = classify_accuracy_series, values_per_sample = classification_values_per_sample(classify_split_samples[["series"]][["train"]]), path = "../web/src/data/classification-series.json", layout = classification_export_arrays_layout(classify_arrays_series, rep(study_window$size, 2L)), smoothing = classification_export_smooth_settings(classify_arrays_series[["labels"]]))`
31. Draws the window in every month as false-colour images in one sheet, for the website. `site_write_png(window_export_sprite(window_series, window_layers$bands, list(red = "SWIR16", green = "NIR08", blue = "RED"), series_stretch, 32L, 6L), "../web/public/data/rgb-months.png")`
32. Writes where the window is, and how its series and monthly images are stored, for the website. `site_write_json(window_export_record(study_window, window_grid, window_layers, band_scale, window_mask, series_quantised, series_stretch, cube_files, list(repo = "e-sensing/cerrado_cube_reg_2018", revision = "refs/pr/3", commit = "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85"), list(png_scale = 2L, series_quantiles = c(0.001, 0.999), series_stride = 2L, series_units = "surface reflectance", series_rgb = list(red = "SWIR16", green = "NIR08", blue = "RED"), series_stretch = c(0.02, 0.98), sprite_columns = 6L, sprite_levels = 32L, sprite_small_tile = 220L, tour_band = "NIR08"), list(series = "series.bin", mask = "series-mask.bin", sprite = "rgb-months.png", sprite_small = "rgb-months-small.png")), "../web/src/data/window.json")`

The files:

- `classification.json`: SHA-256 `4ea04e68fdfa0adf801240f138617755152dea821e50d26adc9741c688a4bf0b`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classes.json`: SHA-256 `e479d0f03cfc2258efe34563f02adc3999d25150b0f991643153d439209b2145`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series.json`: SHA-256 `6835a431c2149c463feef6369e066978f2628ea9a6a624a8f5305ec30f7c950e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `window.json`: SHA-256 `60e731843a80b49590366d3313d69d24d23f2c23c914b42ecdff8ff53b06f9d5`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `rgb-months.png`: SHA-256 `89424cbbdeeda1cd4d46f0e9b2b6c70708250c23cf3b4a2cb4064a46d8439d8e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-probs.bin`: SHA-256 `56353625eee731888bdce8608e5e62420d18e5a5d09fa234ed07d4411eddd88a`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-smooth.bin`: SHA-256 `b47aad286fa6216c7002e7eae267b3231859974c13f71a261c8521b59430840e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-map-raw.bin`: SHA-256 `e6bd433ce6a25a8a9b074dbf537e9acf337825a0f63f005cb7bef42e69648e0a`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `classification-series-map.bin`: SHA-256 `cc70c2b16d0d5d4d07d536e5bfba1e72f1160f6d4edb6fbd02c02322eaafd57f`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

## What to try

1. **One class at a time.** Press *One class* and choose a class: white is a probability of 0, black of 1. Click a pixel, or use the arrow keys on the window, to read all its probabilities as bars.
2. **Switch smoothing on.** The picture becomes the smoothed score of the class, and the marked pixels are those whose label smoothing changed. Where do they gather?
3. **The strongest class.** Press *Strongest class* and switch smoothing off and on: the map from the raw probabilities against the map from the smoothed ones. Show the changed pixels to see where they differ.
4. **Follow a boundary on the ground.** Press *On the ground*, choose *Image | Map* and move the divider slowly across a field: where does the map's edge follow the field in the image, and where does it round a corner?
5. **From embeddings to classes.** This one needs the embeddings: choose *Embeddings* above the figure, then *Embeddings | Map*, and drag the divider: where do the map's classes follow the regions of the embeddings' colours, and where do they not?