# From a cube to labelled samples

How does a place in a cube become a sample whose label we know?

Open the book's labelled MODIS samples, then read one place through the Cerrado cube. Compare recorded regional optical series and their embeddings while keeping the same places, valid periods and supplied labels.

Picture: Recorded book sample at -57.794, -9.7573, valid 2006-09-14 to 2007-08-29, with supplied label Pasture. The dated row opens beside the NDVI median of 344 samples carrying that label, aligned by day within their reference period.

In the sits book: [Basic operations on image time series](https://e-sensing.github.io/sitsbook/ts_basics.html) and [Building regional embeddings: a full example](https://e-sensing.github.io/sitsbook/emb_example.html).

Before this: [From series to embeddings](/embeddings.md).

## 1. What is in a labelled sample?

In the sits book: [Data structure for image time series](https://e-sensing.github.io/sitsbook/ts_basics.html#data-structure-for-image-time-series).

The book reads the labelled samples of Mato Grosso from Hugging Face.

The book's code:

```
samples_matogrosso_mod13q1 <- sits_from_hf(
  repo = "e-sensing/samples_book",
  file = "samples_matogrosso_mod13q1.parquet",
  type = "dataset"
)
```

Where the sits book writes it: [ts_basics.qmd, lines 61–65](https://github.com/e-sensing/sitsbook/blob/bc75e250e50bc6fb8222593973b8dfe3b692201f/ts_basics.qmd#L61-L65).

The book begins with 1,837 samples of Mato Grosso. A sample names a place, a period and a label.

The last cell holds the measurements through that period: a time series.

Many places carry this label. Drawn together, their series show what they share and how much they differ.

`sits_patterns()` fits one curve to each class. It summarizes what these samples share.

Choose a band or a class. These are the book's MODIS samples, collected in different years.

### The figure

Aligned by day within each period. The book's MODIS samples were collected in different years.

The book's published MODIS set contains 1,837 samples. The original first row carries the supplied label Pasture. The figure opens with band NDVI and the Pasture class summary (344 samples). The class and band choices change the named summary; the original exemplar stays named separately. Aligned by day within each period. Source dates from different years are retained below; the fitted pattern has its own source calendar. No common observation year is implied.

| Field          | Recorded value |
| -------------- | -------------- |
| Longitude      | -57.794        |
| Latitude       | -9.7573        |
| Start date     | 2006-09-14     |
| End date       | 2007-08-29     |
| Supplied label | Pasture        |
| Cube           | bdc_cube       |

| Class      | Samples | Reference first date | Reference last date |
| ---------- | ------- | -------------------- | ------------------- |
| Cerrado    | 379     | 2000-09-13           | 2001-08-29          |
| Forest     | 131     | 2008-09-13           | 2009-08-29          |
| Pasture    | 344     | 2006-09-14           | 2007-08-29          |
| Soy corn   | 364     | 2014-09-14           | 2015-08-29          |
| Soy cotton | 352     | 2015-09-14           | 2016-08-28          |
| Soy fallow | 87      | 2006-09-14           | 2007-08-29          |
| Soy millet | 180     | 2014-09-14           | 2015-08-29          |

| Actual sample date | Elapsed day | NDVI                | EVI                 | NIR                 | MIR                 |
| ------------------ | ----------- | ------------------- | ------------------- | ------------------- | ------------------- |
| 2006-09-14         | 0           | 0.4995              | 0.26280000000000003 | 0.2298              | 0.13920000000000002 |
| 2006-09-30         | 16          | 0.4853              | 0.3299              | 0.35850000000000004 | 0.1608              |
| 2006-10-16         | 32          | 0.7161000000000001  | 0.39680000000000004 | 0.2642              | 0.0757              |
| 2006-11-01         | 48          | 0.6536000000000001  | 0.41500000000000004 | 0.3321              | 0.12390000000000001 |
| 2006-11-17         | 64          | 0.5911000000000001  | 0.43320000000000003 | 0.4001              | 0.17220000000000002 |
| 2006-12-03         | 80          | 0.6623              | 0.4388              | 0.34750000000000003 | 0.1253              |
| 2006-12-19         | 96          | 0.7336              | 0.4445              | 0.2948              | 0.0784              |
| 2007-01-01         | 109         | 0.739               | 0.5015000000000001  | 0.3478              | 0.0887              |
| 2007-01-17         | 125         | 0.7679              | 0.5256000000000001  | 0.3512              | 0.0761              |
| 2007-02-02         | 141         | 0.7968000000000001  | 0.5498000000000001  | 0.3547              | 0.0634              |
| 2007-02-18         | 157         | 0.7982              | 0.5334              | 0.3637              | 0.0707              |
| 2007-03-06         | 173         | 0.7763              | 0.5142              | 0.3486              | 0.0732              |
| 2007-03-22         | 189         | 0.7543000000000001  | 0.4949              | 0.3335              | 0.0758              |
| 2007-04-07         | 205         | 0.5025000000000001  | 0.3667              | 0.454               | 0.113               |
| 2007-04-23         | 221         | 0.7458              | 0.3904              | 0.2384              | 0.0437              |
| 2007-05-09         | 237         | 0.7291000000000001  | 0.5442              | 0.3793              | 0.0947              |
| 2007-05-25         | 253         | 0.6806              | 0.4495              | 0.3294              | 0.0918              |
| 2007-06-10         | 269         | 0.5938              | 0.37010000000000004 | 0.30920000000000003 | 0.1136              |
| 2007-06-26         | 285         | 0.5018              | 0.3019              | 0.28250000000000003 | 0.13390000000000002 |
| 2007-07-12         | 301         | 0.5389              | 0.31720000000000004 | 0.2757              | 0.1293              |
| 2007-07-28         | 317         | 0.4645              | 0.249               | 0.2366              | 0.14930000000000002 |
| 2007-08-13         | 333         | 0.44010000000000005 | 0.23700000000000002 | 0.2341              | 0.1529              |
| 2007-08-29         | 349         | 0.31010000000000004 | 0.1898              | 0.24880000000000002 | 0.1774              |

| Source date | Day within period | Median | Quartile 25 | Quartile 75 |
| ----------- | ----------------- | ------ | ----------- | ----------- |
| 2006-09-14  | 0                 | 0.3565 | 0.3167      | 0.4209      |
| 2006-09-30  | 16                | 0.4119 | 0.3444      | 0.4896      |
| 2006-10-16  | 32                | 0.47   | 0.3969      | 0.5428      |
| 2006-11-01  | 48                | 0.553  | 0.4754      | 0.6327      |
| 2006-11-17  | 64                | 0.583  | 0.5218      | 0.659       |
| 2006-12-03  | 80                | 0.6178 | 0.5581      | 0.6924      |
| 2006-12-19  | 96                | 0.6546 | 0.5926      | 0.7016      |
| 2007-01-01  | 109               | 0.6608 | 0.6077      | 0.7114      |
| 2007-01-17  | 125               | 0.663  | 0.6018      | 0.7144      |
| 2007-02-02  | 141               | 0.665  | 0.5956      | 0.7255      |
| 2007-02-18  | 157               | 0.6741 | 0.6075      | 0.7238      |
| 2007-03-06  | 173               | 0.6931 | 0.6336      | 0.7381      |
| 2007-03-22  | 189               | 0.686  | 0.6224      | 0.7316      |
| 2007-04-07  | 205               | 0.6792 | 0.6252      | 0.727       |
| 2007-04-23  | 221               | 0.672  | 0.6092      | 0.7194      |
| 2007-05-09  | 237               | 0.6332 | 0.5719      | 0.6904      |
| 2007-05-25  | 253               | 0.6013 | 0.5319      | 0.664       |
| 2007-06-10  | 269               | 0.542  | 0.4692      | 0.608       |
| 2007-06-26  | 285               | 0.4722 | 0.4136      | 0.5389      |
| 2007-07-12  | 301               | 0.4171 | 0.3636      | 0.4845      |
| 2007-07-28  | 317               | 0.3852 | 0.3387      | 0.436       |
| 2007-08-13  | 333               | 0.3496 | 0.3171      | 0.397       |
| 2007-08-29  | 349               | 0.3484 | 0.3148      | 0.3898      |

| Pattern source date | Day within period | sits_patterns() value |
| ------------------- | ----------------- | --------------------- |
| 2006-09-14          | 0                 | 0.3726                |
| 2006-09-22          | 8                 | 0.4004                |
| 2006-09-30          | 16                | 0.4282                |
| 2006-10-08          | 24                | 0.4561                |
| 2006-10-16          | 32                | 0.4839                |
| 2006-10-24          | 40                | 0.5113                |
| 2006-11-01          | 48                | 0.5376                |
| 2006-11-09          | 56                | 0.5622                |
| 2006-11-17          | 64                | 0.5845                |
| 2006-11-25          | 72                | 0.6037                |
| 2006-12-03          | 80                | 0.6195                |
| 2006-12-11          | 88                | 0.6317                |
| 2006-12-19          | 96                | 0.6405                |
| 2006-12-27          | 104               | 0.6465                |
| 2007-01-04          | 112               | 0.6502                |
| 2007-01-12          | 120               | 0.6526                |
| 2007-01-20          | 128               | 0.6545                |
| 2007-01-28          | 136               | 0.6566                |
| 2007-02-05          | 144               | 0.6594                |
| 2007-02-13          | 152               | 0.6629                |
| 2007-02-21          | 160               | 0.6668                |
| 2007-03-01          | 168               | 0.6707                |
| 2007-03-09          | 176               | 0.674                 |
| 2007-03-17          | 184               | 0.676                 |
| 2007-03-25          | 192               | 0.6762                |
| 2007-04-02          | 200               | 0.6741                |
| 2007-04-10          | 208               | 0.6695                |
| 2007-04-18          | 216               | 0.6623                |
| 2007-04-26          | 224               | 0.6522                |
| 2007-05-04          | 232               | 0.6392                |
| 2007-05-12          | 240               | 0.6231                |
| 2007-05-20          | 248               | 0.604                 |
| 2007-05-28          | 256               | 0.5818                |
| 2007-06-05          | 264               | 0.5568                |
| 2007-06-13          | 272               | 0.5296                |
| 2007-06-21          | 280               | 0.5011                |
| 2007-06-29          | 288               | 0.4726                |
| 2007-07-07          | 296               | 0.4454                |
| 2007-07-15          | 304               | 0.4209                |
| 2007-07-23          | 312               | 0.4001                |
| 2007-07-31          | 320               | 0.3836                |
| 2007-08-08          | 328               | 0.3712                |
| 2007-08-16          | 336               | 0.3622                |
| 2007-08-24          | 344               | 0.3554                |

#### The dates of this sample

Original calendar dates and all bands of the same first row. The chart aligns each curve by elapsed days within its own reference period.

| Actual date | Elapsed day | NDVI   | EVI    | NIR    | MIR    |
| ----------- | ----------- | ------ | ------ | ------ | ------ |
| 2006-09-14  | 0           | 0.4995 | 0.2628 | 0.2298 | 0.1392 |
| 2006-09-30  | 16          | 0.4853 | 0.3299 | 0.3585 | 0.1608 |
| 2006-10-16  | 32          | 0.7161 | 0.3968 | 0.2642 | 0.0757 |
| 2006-11-01  | 48          | 0.6536 | 0.4150 | 0.3321 | 0.1239 |
| 2006-11-17  | 64          | 0.5911 | 0.4332 | 0.4001 | 0.1722 |
| 2006-12-03  | 80          | 0.6623 | 0.4388 | 0.3475 | 0.1253 |
| 2006-12-19  | 96          | 0.7336 | 0.4445 | 0.2948 | 0.0784 |
| 2007-01-01  | 109         | 0.7390 | 0.5015 | 0.3478 | 0.0887 |
| 2007-01-17  | 125         | 0.7679 | 0.5256 | 0.3512 | 0.0761 |
| 2007-02-02  | 141         | 0.7968 | 0.5498 | 0.3547 | 0.0634 |
| 2007-02-18  | 157         | 0.7982 | 0.5334 | 0.3637 | 0.0707 |
| 2007-03-06  | 173         | 0.7763 | 0.5142 | 0.3486 | 0.0732 |
| 2007-03-22  | 189         | 0.7543 | 0.4949 | 0.3335 | 0.0758 |
| 2007-04-07  | 205         | 0.5025 | 0.3667 | 0.4540 | 0.1130 |
| 2007-04-23  | 221         | 0.7458 | 0.3904 | 0.2384 | 0.0437 |
| 2007-05-09  | 237         | 0.7291 | 0.5442 | 0.3793 | 0.0947 |
| 2007-05-25  | 253         | 0.6806 | 0.4495 | 0.3294 | 0.0918 |
| 2007-06-10  | 269         | 0.5938 | 0.3701 | 0.3092 | 0.1136 |
| 2007-06-26  | 285         | 0.5018 | 0.3019 | 0.2825 | 0.1339 |
| 2007-07-12  | 301         | 0.5389 | 0.3172 | 0.2757 | 0.1293 |
| 2007-07-28  | 317         | 0.4645 | 0.2490 | 0.2366 | 0.1493 |
| 2007-08-13  | 333         | 0.4401 | 0.2370 | 0.2341 | 0.1529 |
| 2007-08-29  | 349         | 0.3101 | 0.1898 | 0.2488 | 0.1774 |

Pinned source: e-sensing/samples_book, revision 4945eb700f902f63963a84f2265ec789e2ebc3de. This site's pipeline reads the verified local Parquet with sits_from_parquet(). It computes the class summaries and sits_patterns() curves.

#### How it was made

1. Read the book's samples: The pipeline verifies the pinned Mato Grosso Parquet, reads its supplied labels, and computes aligned summaries and sits_patterns() curves.
2. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Verifies the pinned book Parquet's bytes. `hf_pinned_file(list(repo = "e-sensing/samples_book", type = "dataset", revision = "4945eb700f902f63963a84f2265ec789e2ebc3de", file = "samples_matogrosso_mod13q1.parquet", bytes = 481117, sha256 = "d303ca01bc556a23c3f9da10532f17effbac08b3ae74fb8b23ef619f7331a21e"))`
2. Reads the book's supplied labelled samples. `samples_book_read(samples_book_file)`
3. Computes the book samples' sits_patterns() curves. `samples_book_pattern_data(samples_book)`
4. Keeps the book sample axes and exact decimal grid. `samples_book_values(samples_book)`
5. Aligns each class and computes medians and quartiles. `samples_book_class_curves(samples_book)`
6. Writes the pinned book samples and computed curves. `site_write_json(samples_book_json(samples_book_record, list(together = samples_book_together, patterns = samples_book_patterns), list(source = list(repo = "e-sensing/samples_book", type = "dataset", revision = "4945eb700f902f63963a84f2265ec789e2ebc3de", file = "samples_matogrosso_mod13q1.parquet", bytes = 481117, sha256 = "d303ca01bc556a23c3f9da10532f17effbac08b3ae74fb8b23ef619f7331a21e"), titles = list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated"))), "../web/src/data/series-book.json")`
7. Writes the exact book sample/band/date integer array. `samples_book_write_values(samples_book_record, "../web/public/data/series-samples.bin")`

The files:

- `series-book.json`: SHA-256 `c6f8d0f8a3b0529c6e59701c1bb711b1539818ab2a97bbd03719ea59d41a774e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `series-samples.bin`: SHA-256 `c82a15efc85654c86c2538919c3b9d24b410a4045c5900b2b56bd2abb23947d3`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

Try it: [Patterns of labelled samples](/interactive-images/time-series-patterns.md)

## 2. How does one place become a time series?

In the sits book: [Time series from data cubes](https://e-sensing.github.io/sitsbook/ts_basics.html#time-series-from-data-cubes).

Here sits reads one place through the months of the Cerrado cube.

The book's code:

```
series <- sits_get_data(cube    = raster_cube,
                        samples = sample_latlong)
```

Where the sits book writes it: [ts_basics.qmd, lines 490–491](https://github.com/e-sensing/sitsbook/blob/bc75e250e50bc6fb8222593973b8dfe3b692201f/ts_basics.qmd#L490-L491).

The book's call uses Sinop. Here we ran it on one point in the cached Cerrado window.

Now the Cerrado, with Landsat images. The marked place lies inside the small window.

The same pixel is present in every month's image.

Read down the column: 24 months in 6 bands.

`sits_get_data()` returned this row. Its label is NoClass because we supplied only a location.

### The figure

One sits_get_data() call, run by this site's pipeline on the cached Cerrado window. The regional samples shown next are a separate dataset.

One extracted sits_get_data() result from the cached Cerrado window (BDC · LANDSAT-OLI-16D). The marked native pixel is column 128, row 128 (zero-based); the image shows 2017-05-01. The cube diagram follows that same pixel through all 24 dates. 6 band lines show surface reflectance, with the original dates on their axis. The returned label is NoClass: only a location was supplied. The book's two-line extraction call uses Sinop; this result is our local Cerrado adaptation. It matches the cached window exactly on every observed value; imputed values differ by at most the last binary digit, because sits imputes before it scales. The point's coordinates are rounded to 15 significant digits in the table below; the figure shows 6 decimals. The regional records shown next come from a separate published sample file; they were not extracted at this window point.

| Returned field | Value             |
| -------------- | ----------------- |
| longitude      | -57.8030536415983 |
| latitude       | -13.8304516793958 |
| start_date     | 2017-01-01        |
| end_date       | 2018-12-01        |
| label          | NoClass           |
| cube           | LANDSAT-OLI-16D   |

| Actual date | BLUE                 | GREEN                | RED                  | NIR08               | SWIR16              | SWIR22              |
| ----------- | -------------------- | -------------------- | -------------------- | ------------------- | ------------------- | ------------------- |
| 2017-01-01  | 0.0194               | 0.0351               | 0.030100000000000002 | 0.1932              | 0.15380000000000002 | 0.0699              |
| 2017-02-01  | 0.0194               | 0.0351               | 0.030100000000000002 | 0.1932              | 0.15380000000000002 | 0.0699              |
| 2017-03-01  | 0.0194               | 0.0351               | 0.030100000000000002 | 0.1932              | 0.15380000000000002 | 0.0699              |
| 2017-04-01  | 0.0194               | 0.0351               | 0.030100000000000002 | 0.1932              | 0.15380000000000002 | 0.0699              |
| 2017-05-01  | 0.0194               | 0.0351               | 0.030100000000000002 | 0.1932              | 0.15380000000000002 | 0.0699              |
| 2017-06-01  | 0.0201               | 0.038                | 0.0349               | 0.1917              | 0.16110000000000002 | 0.0763              |
| 2017-07-01  | 0.0222               | 0.044700000000000004 | 0.0441               | 0.21480000000000002 | 0.1773              | 0.08800000000000001 |
| 2017-08-01  | 0.0256               | 0.0516               | 0.049300000000000004 | 0.2368              | 0.2024              | 0.0999              |
| 2017-09-01  | 0.024133333333333336 | 0.051000000000000004 | 0.0446               | 0.26383333333333336 | 0.19723333333333334 | 0.09263333333333335 |
| 2017-10-01  | 0.02266666666666667  | 0.0504               | 0.039900000000000005 | 0.29086666666666666 | 0.1920666666666667  | 0.08536666666666666 |
| 2017-11-01  | 0.0212               | 0.049800000000000004 | 0.0352               | 0.3179              | 0.1869              | 0.0781              |
| 2017-12-01  | 0.019700000000000002 | 0.04685              | 0.0322               | 0.30525             | 0.17675000000000002 | 0.0717              |
| 2018-01-01  | 0.0182               | 0.0439               | 0.0292               | 0.2926              | 0.1666              | 0.0653              |
| 2018-02-01  | 0.0182               | 0.04415              | 0.03155              | 0.2773              | 0.16265000000000002 | 0.0673              |
| 2018-03-01  | 0.0182               | 0.0444               | 0.0339               | 0.262               | 0.1587              | 0.0693              |
| 2018-04-01  | 0.0189               | 0.04                 | 0.030500000000000003 | 0.24180000000000001 | 0.15525             | 0.06655             |
| 2018-05-01  | 0.0196               | 0.0356               | 0.027100000000000003 | 0.22160000000000002 | 0.15180000000000002 | 0.06380000000000001 |
| 2018-06-01  | 0.0189               | 0.0367               | 0.0317               | 0.2068              | 0.15760000000000002 | 0.0728              |
| 2018-07-01  | 0.0194               | 0.041                | 0.0367               | 0.21150000000000002 | 0.16160000000000002 | 0.07440000000000001 |
| 2018-08-01  | 0.022500000000000003 | 0.046900000000000004 | 0.0437               | 0.23370000000000002 | 0.1822              | 0.0881              |
| 2018-09-01  | 0.0221               | 0.0533               | 0.0432               | 0.29450000000000004 | 0.19820000000000002 | 0.0878              |
| 2018-10-01  | 0.021400000000000002 | 0.054200000000000005 | 0.039                | 0.3048              | 0.1777              | 0.0755              |
| 2018-11-01  | 0.021400000000000002 | 0.054200000000000005 | 0.039                | 0.3048              | 0.1777              | 0.0755              |
| 2018-12-01  | 0.021400000000000002 | 0.054200000000000005 | 0.039                | 0.3048              | 0.1777              | 0.0755              |

#### How it was made

1. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
2. Fill the missing months: Where a pixel has no value in a month, sits fills one in on a straight line between the months around it. At either end of the series it repeats the nearest value.
3. Read one place: sits_get_data() reads one centre pixel from the cached cube. It matches the cached window exactly on every observed value; imputed values differ by at most the last binary digit, because sits imputes before it scales.
4. Prepare the pictures: The pipeline picks the brightness drawn black and white, the same for every date, so the dates can be compared. Values are packed into one byte each.
5. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
3. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
4. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
5. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
6. Names the bands and months of the crops. `window_crop_layers(window_crops, study_window$source, study_window$collection, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))`
7. Reads how sits scales the bands and marks missing values. `collection_band_values(study_window$source, study_window$collection, window_layers$bands)`
8. Reads the stored values of the crops, missing ones as NA. `window_read_raw(window_crops, window_layers$bands, window_layers$dates, band_scale, study_window$size)`
9. Marks the months each pixel was observed in. `window_observed_mask(window_raw)`
10. Fills the missing months along time with sits::impute_linear(), as sits does before the encoder. `window_fill_series(window_raw, band_scale)`
11. Reads and verifies the actual centre-point sits_get_data() result. `samples_window_check(samples_window_read(window_crops, window_grid, c(list(source = "BDC", collection = "LANDSAT-OLI-16D"), list(layers = window_layers))), window_series, list(layers = window_layers, raw = window_raw, scale = band_scale))`
12. Stores every other pixel's series in one byte per value, each band between two quantiles. `window_export_series(window_series, c(0.001, 0.999 ), 2L)`
13. Picks the reflectance drawn black and white in each colour, the same for every month. `window_export_sprite_stretch(window_series, window_layers$bands, list(red = "SWIR16", green = "NIR08", blue = "RED"), c(0.02, 0.98))`
14. Writes the actual returned centre-point row. `site_write_json(samples_window_json(samples_window_point), "../web/src/data/labelled-window-point.json")`
15. Writes the unchanged full-precision point values. `samples_window_write(samples_window_point, window_layers, "../web/public/data/labelled-window-point.bin")`
16. Draws the window in every month as false-colour images in one sheet, for the website. `site_write_png(window_export_sprite(window_series, window_layers$bands, list(red = "SWIR16", green = "NIR08", blue = "RED"), series_stretch, 32L, 6L), "../web/public/data/rgb-months.png")`
17. Writes where the window is, and how its series and monthly images are stored, for the website. `site_write_json(window_export_record(study_window, window_grid, window_layers, band_scale, window_mask, series_quantised, series_stretch, cube_files, list(repo = "e-sensing/cerrado_cube_reg_2018", revision = "refs/pr/3", commit = "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85"), list(png_scale = 2L, series_quantiles = c(0.001, 0.999), series_stride = 2L, series_units = "surface reflectance", series_rgb = list(red = "SWIR16", green = "NIR08", blue = "RED"), series_stretch = c(0.02, 0.98), sprite_columns = 6L, sprite_levels = 32L, sprite_small_tile = 220L, tour_band = "NIR08"), list(series = "series.bin", mask = "series-mask.bin", sprite = "rgb-months.png", sprite_small = "rgb-months-small.png")), "../web/src/data/window.json")`

The files:

- `window.json`: SHA-256 `60e731843a80b49590366d3313d69d24d23f2c23c914b42ecdff8ff53b06f9d5`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `rgb-months.png`: SHA-256 `89424cbbdeeda1cd4d46f0e9b2b6c70708250c23cf3b4a2cb4064a46d8439d8e`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-window-point.json`: SHA-256 `6ed365d6dc0a933994fff4e77b4a36f332598f3522ae9cf16f1c0b14ed89b0b8`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-window-point.bin`: SHA-256 `01e6c63f64b12cbcd6878eb9157de4c2c6b5d2d2fe38e059fc30964d8399d2c0`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

Try it: [Any pixel of the Sinop cube, as a time series](/interactive-images/time-series-location.md)

## 3. Do all places behave the same?

In the sits book: [Time series from data cubes](https://e-sensing.github.io/sitsbook/ts_basics.html#time-series-from-data-cubes).

The book reads recorded optical series for labelled places across the Cerrado.

The book's code:

```
labelled_samples <- sits_from_hf(
    repo = "e-sensing/samples_cerrado",
    file = "labelled_samples_cerrado.parquet",
    type = "dataset"
)
```

Where the sits book writes it: [emb_example.qmd, lines 551–555](https://github.com/e-sensing/sitsbook/blob/bc75e250e50bc6fb8222593973b8dfe3b692201f/emb_example.qmd#L551-L555).

`sits_from_hf()` reads series that were already extracted; their original extraction is undocumented.

These places lie across the Cerrado. They are outside the small window; the map shows both footprints.

This is one place's recorded NIR08 series. We read it from the published sample file.

Another place, another curve. Their Euclidean distance in NIR08 measures the difference.

Choose a place or a band. A shape alone is not a supplied label.

### The figure

Recorded optical series from the published sample file; their original extraction is undocumented. The map is drawn in geographic coordinates, with no basemap; a square marks the study window at its true place, drawn larger than the window.

The book reads recorded optical series for labelled places across the Cerrado with sits_from_hf(). They are recorded optical series from the published sample file: e-sensing/samples_cerrado, file labelled_samples_cerrado.parquet, revision fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8. For this site, the pipeline read the verified local copy of that published file. The original optical extraction is undocumented. Each series covers 2017-01-01 to 2018-12-01, 24 monthly dates. The locator draws every place as a dot in geographic coordinates (Longitude/latitude, equirectangular), with no basemap or downloaded tiles, ordered by row id. The places span longitudes 59.9401°W to 41.5409°W and latitudes 24.4117°S to 2.3509°S. The window's true footprint spans longitudes 57.8397°W to 57.7667°W and latitudes 13.8650°S to 13.7957°S, centred at 57.8032°W, 13.8303°S. At that scale it is under a pixel, so a square of fixed size labelled "Study window (step 2)" marks it at its true place, and the note under the map says so: Study window drawn larger than its true 7.7 km. Place A is Place 5 (56.6922°W, 21.9743°S): the smallest row id. Place B is Place 3313 (55.5833°W, 14.8447°S): the series farthest from A's in NIR08, the smaller row id on a tie. The pipeline chose both without labels. Their Euclidean distance in NIR08 is 1.73790928805275. At the last beat the reader may choose any place, listed by row id, or any band; the distance to A is then measured in the browser. A shape alone is not a supplied label.

| Date       | A, Place 5          | B, Place 3313         |
| ---------- | ------------------- | --------------------- |
| 2017-01-01 | 0.39630000000000004 | 0.0039000000000000003 |
| 2017-02-01 | 0.39630000000000004 | 0.0039000000000000003 |
| 2017-03-01 | 0.3356              | 0.0039000000000000003 |
| 2017-04-01 | 0.3295              | 0.0039000000000000003 |
| 2017-05-01 | 0.3242              | 0.0039000000000000003 |
| 2017-06-01 | 0.3189              | 0.0039000000000000003 |
| 2017-07-01 | 0.2985              | 0.0039000000000000003 |
| 2017-08-01 | 0.314               | 0.0281                |
| 2017-09-01 | 0.3015              | 0.0158                |
| 2017-10-01 | 0.34347500000000003 | 0.009250000000000001  |
| 2017-11-01 | 0.38545             | 0.0027                |
| 2017-12-01 | 0.427425            | 0.00224               |
| 2018-01-01 | 0.46940000000000004 | 0.0017800000000000001 |
| 2018-02-01 | 0.379               | 0.00132               |
| 2018-03-01 | 0.3022              | 0.0008600000000000002 |
| 2018-04-01 | 0.3502              | 0.0004                |
| 2018-05-01 | 0.3351              | 0.0043                |
| 2018-06-01 | 0.2939              | 0.0017000000000000001 |
| 2018-07-01 | 0.2881              | 0.004200000000000001  |
| 2018-08-01 | 0.303025            | 0.0068000000000000005 |
| 2018-09-01 | 0.31795             | 0.039                 |
| 2018-10-01 | 0.33287500000000003 | 0.0161                |
| 2018-11-01 | 0.3478              | 0.9312                |
| 2018-12-01 | 0.3884              | 0.0021000000000000003 |

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Fill the missing months: Where a pixel has no value in a month, sits fills one in on a straight line between the months around it. At either end of the series it repeats the nearest value.
4. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
5. Keep the regional sample IDs: The pipeline retains recorded optical series from the published sample file and their cached encodings. The original optical extraction is undocumented; both forms keep the same IDs.
6. Prepare the pictures: The pipeline picks the brightness drawn black and white, the same for every date, so the dates can be compared. Values are packed into one byte each.
7. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
11. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
12. Writes the window's edge as a closed ring of longitudes and latitudes. `site_write_json(samples_window_outline(window_grid), "../web/src/data/labelled-window-outline.json")`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. Names the bands and months of the crops. `window_crop_layers(window_crops, study_window$source, study_window$collection, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))`
15. Reads how sits scales the bands and marks missing values. `collection_band_values(study_window$source, study_window$collection, window_layers$bands)`
16. Binds recorded optical and cached encoded rows by their complete identity. `samples_cohort_record(samples_subset, samples_encoded, list(bands = window_layers[["bands"]], choice_band = "NIR08", classes = samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), source = list( repo = "e-sensing/samples_cerrado", commit = "fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8", file = "labelled_samples_cerrado.parquet")))`
17. Checks exact storage in the declared band order. `samples_storage_order(samples_storage_check(labelled_sample_record[["optical"]], samples_grid_metadata(list(crops = window_crops), "cache/src/sits-78d26eb07370874192911854b48b60ebb1d668c8")), labelled_sample_record[["bands"]])`
18. Writes original IDs and full-precision coordinates. `samples_cohort_points(labelled_sample_record, "../web/public/data/labelled-sample-points.bin")`
19. Reads the stored values of the crops, missing ones as NA. `window_read_raw(window_crops, window_layers$bands, window_layers$dates, band_scale, study_window$size)`
20. Writes each recorded optical band exactly in sample/date order. `samples_cohort_optical(labelled_sample_record, labelled_sample_storage, "../web/public/data")`
21. Writes axes, counts and descriptors for the common recorded cohort. `site_write_json(samples_cohort_json(labelled_sample_record, labelled_sample_storage, list(source = list(repo = "e-sensing/samples_cerrado", commit = "fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8", file = "labelled_samples_cerrado.parquet"), choice_band = "NIR08", titles = list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated"), settings = list(samples = list(per_class = 500L, period_start = "2017-01-01", train_frac = 0.8), seed = 20261011L, encoder = list(repo = "e-sensing/encoders_cerrado", type = "model", commit = "fbcea998a4173854a6a7d7cbd3bbeae4496de0c5", files = list(lejepa = "ssl_lejepa_tcnn_model_2017_2024.rds"))))), "../web/src/data/labelled-samples.json")`
22. Marks the months each pixel was observed in. `window_observed_mask(window_raw)`
23. Fills the missing months along time with sits::impute_linear(), as sits does before the encoder. `window_fill_series(window_raw, band_scale)`
24. Stores every other pixel's series in one byte per value, each band between two quantiles. `window_export_series(window_series, c(0.001, 0.999 ), 2L)`
25. Picks the reflectance drawn black and white in each colour, the same for every month. `window_export_sprite_stretch(window_series, window_layers$bands, list(red = "SWIR16", green = "NIR08", blue = "RED"), c(0.02, 0.98))`
26. Writes where the window is, and how its series and monthly images are stored, for the website. `site_write_json(window_export_record(study_window, window_grid, window_layers, band_scale, window_mask, series_quantised, series_stretch, cube_files, list(repo = "e-sensing/cerrado_cube_reg_2018", revision = "refs/pr/3", commit = "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85"), list(png_scale = 2L, series_quantiles = c(0.001, 0.999), series_stride = 2L, series_units = "surface reflectance", series_rgb = list(red = "SWIR16", green = "NIR08", blue = "RED"), series_stretch = c(0.02, 0.98), sprite_columns = 6L, sprite_levels = 32L, sprite_small_tile = 220L, tour_band = "NIR08"), list(series = "series.bin", mask = "series-mask.bin", sprite = "rgb-months.png", sprite_small = "rgb-months-small.png")), "../web/src/data/window.json")`

The files:

- `labelled-samples.json`: SHA-256 `19c17c7964f2917f2ac50dcd32a7c7e37255c0f41b834b2c6e5ac8cfac84862c`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-window-outline.json`: SHA-256 `16a450cec894634f771b9ecb3bf6e6ff0341c7addc39544575c3412f1b373377`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `window.json`: SHA-256 `60e731843a80b49590366d3313d69d24d23f2c23c914b42ecdff8ff53b06f9d5`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-sample-points.bin`: SHA-256 `f30f63bfdccf84c25268347a694c4de1864d2d3e70a645d691f10cdccbefcf68`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-BLUE.bin`: SHA-256 `85c4ed0646d84e38da8e45a754299324fa86947bbba1787630186bdc80d69bb8`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-GREEN.bin`: SHA-256 `1d437e78acfe8ae2375085ce2d43e6015bd1ecb712cd1ff0992d39ce89929385`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-RED.bin`: SHA-256 `052c0eda2adc4e1fabfde60d48e43963e0797cda0282631118c11ba74ab0ff09`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-NIR08.bin`: SHA-256 `862bc52a73cdf4d585a918da2cb848e1908b37dabf2c6a02e2699f7a61bea1c5`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-SWIR16.bin`: SHA-256 `117a96c40441a5d3b81671beb61cffe4a1a9fee20e83b8c22f5ebe68176c5a4d`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-SWIR22.bin`: SHA-256 `678ca9c394ace2199b4042f7802bf7d974890fefbc6e897e83e94f6ddb998165`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

## 4. Where does a sample's label come from?

In the sits book: [Data structure for image time series](https://e-sensing.github.io/sitsbook/ts_basics.html#data-structure-for-image-time-series).

A label belongs to a place for a stated period.

The published sample already carries this label. The curve did not assign it.

These regional records carry 11 labels. The comparison keeps the same 4,443 samples.

The label holds for this period. Open the Sinop example to compare other years.

### The figure

Recorded series and supplied labels from the published sample file; their original extraction is undocumented. The map is drawn in geographic coordinates, with no basemap; a square marks the study window at its true place, drawn larger than the window.

The selected record is place A, Place 5 (56.6922°W, 21.9743°S), the record step 3 opened with. Its start_date and end_date state the period 2017-01-01 to 2018-12-01; every record of the file shares it. Its supplied label is Cerradão (Cerradao). The published sample already carries this label; the curve did not assign it. The figure shows A's unchanged NIR08 series beside the record. These regional records carry 11 labels; the comparison keeps the same 4,443 samples. On the map each place takes its class colour from the published dictionary. A square labelled "Study window (step 2)" marks step 2's window at its true place: Study window drawn larger than its true 7.7 km. The records are recorded optical series from the published sample file: e-sensing/samples_cerrado, file labelled_samples_cerrado.parquet, revision fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8. The original optical extraction is undocumented. The labels come with the published file; this site checked neither them nor the extraction. At the last beat the reader may pick any place, listed by row id, to read its record and series.

| Field      | Recorded value      |
| ---------- | ------------------- |
| longitude  | -56.69219601305513  |
| latitude   | -21.97433982789688  |
| start_date | 2017-01-01          |
| end_date   | 2018-12-01          |
| label      | Cerradão (Cerradao) |

| Label                 | Name in the file | Samples |
| --------------------- | ---------------- | ------- |
| Annual crop           | Annual_Crop      | 500     |
| Cerradão              | Cerradao         | 500     |
| Cerrado               | Cerrado          | 500     |
| Mangrove              | Mangrove         | 200     |
| Natural non-vegetated | Nat_NonVeg       | 32      |
| Open cerrado          | Open_Cerrado     | 500     |
| Pasture               | Pasture          | 500     |
| Perennial crop        | Perennial_Crop   | 500     |
| Silviculture          | Silviculture     | 500     |
| Sugarcane             | Sugarcane        | 500     |
| Water                 | Water            | 211     |

| Date       | A, Place 5, NIR08   |
| ---------- | ------------------- |
| 2017-01-01 | 0.39630000000000004 |
| 2017-02-01 | 0.39630000000000004 |
| 2017-03-01 | 0.3356              |
| 2017-04-01 | 0.3295              |
| 2017-05-01 | 0.3242              |
| 2017-06-01 | 0.3189              |
| 2017-07-01 | 0.2985              |
| 2017-08-01 | 0.314               |
| 2017-09-01 | 0.3015              |
| 2017-10-01 | 0.34347500000000003 |
| 2017-11-01 | 0.38545             |
| 2017-12-01 | 0.427425            |
| 2018-01-01 | 0.46940000000000004 |
| 2018-02-01 | 0.379               |
| 2018-03-01 | 0.3022              |
| 2018-04-01 | 0.3502              |
| 2018-05-01 | 0.3351              |
| 2018-06-01 | 0.2939              |
| 2018-07-01 | 0.2881              |
| 2018-08-01 | 0.303025            |
| 2018-09-01 | 0.31795             |
| 2018-10-01 | 0.33287500000000003 |
| 2018-11-01 | 0.3478              |
| 2018-12-01 | 0.3884              |

#### Other years: the separate Sinop example

The book's Sinop example is a separate dataset, with its own places in Mato Grosso, read from its own CSV file. It is not part of the Cerrado samples of this step: no Sinop value is drawn on this curve or this map.

A label holds for the period written in its sample. Setting the same place's series in another year beside it gives a distance between the two series. A distance, or a threshold on it, is a measure of how much they differ, never a new label.

To compare a Sinop place across its years, open [the interactive image of the Sinop years](/interactive-images/time-series-years.md).

The Sinop CSV example is in the sits book: [Time series from data cubes](https://e-sensing.github.io/sitsbook/ts_basics.html#time-series-from-data-cubes).

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Fill the missing months: Where a pixel has no value in a month, sits fills one in on a straight line between the months around it. At either end of the series it repeats the nearest value.
4. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
5. Keep the regional sample IDs: The pipeline retains recorded optical series from the published sample file and their cached encodings. The original optical extraction is undocumented; both forms keep the same IDs.
6. Prepare the pictures: The pipeline picks the brightness drawn black and white, the same for every date, so the dates can be compared. Values are packed into one byte each.
7. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
11. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
12. Writes the window's edge as a closed ring of longitudes and latitudes. `site_write_json(samples_window_outline(window_grid), "../web/src/data/labelled-window-outline.json")`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. Names the bands and months of the crops. `window_crop_layers(window_crops, study_window$source, study_window$collection, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))`
15. Reads how sits scales the bands and marks missing values. `collection_band_values(study_window$source, study_window$collection, window_layers$bands)`
16. Binds recorded optical and cached encoded rows by their complete identity. `samples_cohort_record(samples_subset, samples_encoded, list(bands = window_layers[["bands"]], choice_band = "NIR08", classes = samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), source = list( repo = "e-sensing/samples_cerrado", commit = "fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8", file = "labelled_samples_cerrado.parquet")))`
17. Checks exact storage in the declared band order. `samples_storage_order(samples_storage_check(labelled_sample_record[["optical"]], samples_grid_metadata(list(crops = window_crops), "cache/src/sits-78d26eb07370874192911854b48b60ebb1d668c8")), labelled_sample_record[["bands"]])`
18. Writes supplied labels separately in the common row order. `samples_cohort_labels(labelled_sample_record, "../web/public/data/labelled-sample-labels.bin")`
19. Writes original IDs and full-precision coordinates. `samples_cohort_points(labelled_sample_record, "../web/public/data/labelled-sample-points.bin")`
20. Reads the stored values of the crops, missing ones as NA. `window_read_raw(window_crops, window_layers$bands, window_layers$dates, band_scale, study_window$size)`
21. Writes each recorded optical band exactly in sample/date order. `samples_cohort_optical(labelled_sample_record, labelled_sample_storage, "../web/public/data")`
22. Writes axes, counts and descriptors for the common recorded cohort. `site_write_json(samples_cohort_json(labelled_sample_record, labelled_sample_storage, list(source = list(repo = "e-sensing/samples_cerrado", commit = "fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8", file = "labelled_samples_cerrado.parquet"), choice_band = "NIR08", titles = list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated"), settings = list(samples = list(per_class = 500L, period_start = "2017-01-01", train_frac = 0.8), seed = 20261011L, encoder = list(repo = "e-sensing/encoders_cerrado", type = "model", commit = "fbcea998a4173854a6a7d7cbd3bbeae4496de0c5", files = list(lejepa = "ssl_lejepa_tcnn_model_2017_2024.rds"))))), "../web/src/data/labelled-samples.json")`
23. Marks the months each pixel was observed in. `window_observed_mask(window_raw)`
24. Fills the missing months along time with sits::impute_linear(), as sits does before the encoder. `window_fill_series(window_raw, band_scale)`
25. Stores every other pixel's series in one byte per value, each band between two quantiles. `window_export_series(window_series, c(0.001, 0.999 ), 2L)`
26. Picks the reflectance drawn black and white in each colour, the same for every month. `window_export_sprite_stretch(window_series, window_layers$bands, list(red = "SWIR16", green = "NIR08", blue = "RED"), c(0.02, 0.98))`
27. Writes where the window is, and how its series and monthly images are stored, for the website. `site_write_json(window_export_record(study_window, window_grid, window_layers, band_scale, window_mask, series_quantised, series_stretch, cube_files, list(repo = "e-sensing/cerrado_cube_reg_2018", revision = "refs/pr/3", commit = "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85"), list(png_scale = 2L, series_quantiles = c(0.001, 0.999), series_stride = 2L, series_units = "surface reflectance", series_rgb = list(red = "SWIR16", green = "NIR08", blue = "RED"), series_stretch = c(0.02, 0.98), sprite_columns = 6L, sprite_levels = 32L, sprite_small_tile = 220L, tour_band = "NIR08"), list(series = "series.bin", mask = "series-mask.bin", sprite = "rgb-months.png", sprite_small = "rgb-months-small.png")), "../web/src/data/window.json")`

The files:

- `labelled-samples.json`: SHA-256 `19c17c7964f2917f2ac50dcd32a7c7e37255c0f41b834b2c6e5ac8cfac84862c`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-window-outline.json`: SHA-256 `16a450cec894634f771b9ecb3bf6e6ff0341c7addc39544575c3412f1b373377`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `window.json`: SHA-256 `60e731843a80b49590366d3313d69d24d23f2c23c914b42ecdff8ff53b06f9d5`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-sample-points.bin`: SHA-256 `f30f63bfdccf84c25268347a694c4de1864d2d3e70a645d691f10cdccbefcf68`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-sample-labels.bin`: SHA-256 `580acce299741ace3450ff8dbfcaae394e1cdf12b652da53d78e5fd7d4ec6b38`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-NIR08.bin`: SHA-256 `862bc52a73cdf4d585a918da2cb848e1908b37dabf2c6a02e2699f7a61bea1c5`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

Try it: [One labelled Sinop place, every year](/interactive-images/time-series-years.md)

## 5. What changes when a sample is encoded?

In the sits book: [Classifying EO embeddings for land use and land cover applications](https://e-sensing.github.io/sitsbook/emb_example.html#classifying-eo-embeddings-for-land-use-and-land-cover-applications).

The book passes the samples' optical series to the encoder.

The book's code:

```
samples_lejepa <- sits_encode(
    data = labelled_samples_red,
    encoder = lejepa_encoder,
    multicores = 8,
    gpu_memory = 16,
    batch_size = 512,
    verbose = TRUE
)
```

Where the sits book writes it: [emb_example.qmd, lines 601–608](https://github.com/e-sensing/sitsbook/blob/bc75e250e50bc6fb8222593973b8dfe3b692201f/emb_example.qmd#L601-L608).

The book encodes a random fraction of its samples. Here the pipeline encoded the 4,443 samples valid from 2017-01-01, drawn per class with a fixed seed.

This is the sample's recorded NIR08 series, one of the 6 bands the encoder reads.

`sits_encode()` reads all 6 bands over 24 months. The encoder connection here is a diagram.

64 values come back. The location, valid period and supplied label stay with the sample.

Switch forms for the same samples. One axis follows months; the other follows dimensions.

Compare one class in either form. Embedding dimensions do not mark seasons.

### The figure

Recorded optical series from the published sample file; their original extraction is undocumented. Their embeddings come from this site's pipeline running sits_encode() with the book's encoder. The encoder is drawn as a diagram. The map is drawn in geographic coordinates, with no basemap; a square marks the study window at its true place, drawn larger than the window.

The step follows place A, Place 5 (56.6922°W, 21.9743°S), labelled Cerradão (Cerradao). The book passes the samples' optical series to the encoder. The figure first shows A's recorded NIR08 series, one of the 6 bands the encoder reads (BLUE, GREEN, RED, NIR08, SWIR16 and SWIR22). sits_encode() reads all 6 bands over 24 months; the encoder connection drawn on the page is a diagram. 64 values come back, one per embedding dimension. The location, valid period (2017-01-01 to 2018-12-01) and supplied label stay with the sample. Embedding dimensions are not dates: the figure draws them as separate dots, never a line, and they do not mark seasons. The optical series are recorded optical series from the published sample file: e-sensing/samples_cerrado, file labelled_samples_cerrado.parquet, revision fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8. The original optical extraction is undocumented. The embeddings come from this site's pipeline running sits_encode() with the book's encoder on that subset. The book encodes a random fraction of its samples. Here the pipeline encoded the 4,443 samples valid from 2017-01-01, drawn per class up to a fixed number with a fixed seed, so this subset is not the book's draw. A's class holds 500 of them. From the switch on, the reader may show the same samples in either form, pick an optical band, and compare one class's spread in either form.

| Field      | Recorded value      |
| ---------- | ------------------- |
| row_id     | 5                   |
| longitude  | -56.69219601305513  |
| latitude   | -21.97433982789688  |
| start_date | 2017-01-01          |
| end_date   | 2018-12-01          |
| label      | Cerradão (Cerradao) |

| Date       | A, Place 5, NIR08   |
| ---------- | ------------------- |
| 2017-01-01 | 0.39630000000000004 |
| 2017-02-01 | 0.39630000000000004 |
| 2017-03-01 | 0.3356              |
| 2017-04-01 | 0.3295              |
| 2017-05-01 | 0.3242              |
| 2017-06-01 | 0.3189              |
| 2017-07-01 | 0.2985              |
| 2017-08-01 | 0.314               |
| 2017-09-01 | 0.3015              |
| 2017-10-01 | 0.34347500000000003 |
| 2017-11-01 | 0.38545             |
| 2017-12-01 | 0.427425            |
| 2018-01-01 | 0.46940000000000004 |
| 2018-02-01 | 0.379               |
| 2018-03-01 | 0.3022              |
| 2018-04-01 | 0.3502              |
| 2018-05-01 | 0.3351              |
| 2018-06-01 | 0.2939              |
| 2018-07-01 | 0.2881              |
| 2018-08-01 | 0.303025            |
| 2018-09-01 | 0.31795             |
| 2018-10-01 | 0.33287500000000003 |
| 2018-11-01 | 0.3478              |
| 2018-12-01 | 0.3884              |

| Dimension | A, Place 5, embedding value |
| --------- | --------------------------- |
| EMB01     | 1.2330178022384644          |
| EMB02     | -1.3900307416915894         |
| EMB03     | -0.5681504607200623         |
| EMB04     | 0.13382433354854584         |
| EMB05     | 0.061987388879060745        |
| EMB06     | -1.0852817296981812         |
| EMB07     | 0.84416663646698            |
| EMB08     | -2.3015835285186768         |
| EMB09     | -1.5573960542678833         |
| EMB10     | -0.5035130977630615         |
| EMB11     | 0.285387247800827           |
| EMB12     | 0.7860788702964783          |
| EMB13     | -1.713313102722168          |
| EMB14     | -0.5290852189064026         |
| EMB15     | -0.12984433770179749        |
| EMB16     | 0.8999024033546448          |
| EMB17     | -0.017512788996100426       |
| EMB18     | -1.1728001832962036         |
| EMB19     | -0.0707332119345665         |
| EMB20     | -0.7991218566894531         |
| EMB21     | -3.5356178283691406         |
| EMB22     | -1.5985499620437622         |
| EMB23     | -1.6009892225265503         |
| EMB24     | 2.198596715927124           |
| EMB25     | -0.16945622861385345        |
| EMB26     | 1.632704734802246           |
| EMB27     | -0.9757488369941711         |
| EMB28     | -2.152275800704956          |
| EMB29     | 0.31884872913360596         |
| EMB30     | -0.11374634504318237        |
| EMB31     | 1.2201240062713623          |
| EMB32     | -0.6432758569717407         |
| EMB33     | 0.427428662776947           |
| EMB34     | -1.4713659286499023         |
| EMB35     | 1.921845555305481           |
| EMB36     | -0.14998337626457214        |
| EMB37     | 3.6155354976654053          |
| EMB38     | -0.014483272098004818       |
| EMB39     | 1.3911150693893433          |
| EMB40     | 0.9809101819992065          |
| EMB41     | 1.695563793182373           |
| EMB42     | 0.10549647361040115         |
| EMB43     | -0.7289964556694031         |
| EMB44     | 1.615289568901062           |
| EMB45     | 1.9130191802978516          |
| EMB46     | 0.7141991853713989          |
| EMB47     | -1.6628077030181885         |
| EMB48     | 2.5517265796661377          |
| EMB49     | -0.6360418796539307         |
| EMB50     | -2.0116565227508545         |
| EMB51     | 0.015381492674350739        |
| EMB52     | -0.6779536604881287         |
| EMB53     | 0.8639090061187744          |
| EMB54     | 1.8074896335601807          |
| EMB55     | -1.6300450563430786         |
| EMB56     | 0.7932793498039246          |
| EMB57     | 0.21893301606178284         |
| EMB58     | -0.8145778775215149         |
| EMB59     | 2.004373073577881           |
| EMB60     | 1.0559161901474             |
| EMB61     | -1.9200150966644287         |
| EMB62     | 0.6940257549285889          |
| EMB63     | -1.0361239910125732         |
| EMB64     | 0.31322863698005676         |

| Date       | Lower quartile      | Median              | Upper quartile      |
| ---------- | ------------------- | ------------------- | ------------------- |
| 2017-01-01 | 0.2781              | 0.32165             | 0.358425            |
| 2017-02-01 | 0.2792              | 0.315825            | 0.3547              |
| 2017-03-01 | 0.28015             | 0.3096166666666667  | 0.34563333333333335 |
| 2017-04-01 | 0.263575            | 0.2913              | 0.3214              |
| 2017-05-01 | 0.25225000000000003 | 0.27592500000000003 | 0.3064              |
| 2017-06-01 | 0.24532500000000002 | 0.26880000000000004 | 0.29510000000000003 |
| 2017-07-01 | 0.2447              | 0.2683              | 0.29450000000000004 |
| 2017-08-01 | 0.25062500000000004 | 0.2813              | 0.3082125           |
| 2017-09-01 | 0.26755             | 0.2977              | 0.3259              |
| 2017-10-01 | 0.28703750000000006 | 0.31735             | 0.3504333333333333  |
| 2017-11-01 | 0.289175            | 0.33119375          | 0.3748              |
| 2017-12-01 | 0.3012275           | 0.3329              | 0.37400625          |
| 2018-01-01 | 0.30202000000000007 | 0.33332333333333336 | 0.36973750000000005 |
| 2018-02-01 | 0.29460000000000003 | 0.32291250000000005 | 0.357225            |
| 2018-03-01 | 0.28490000000000004 | 0.3128714285714286  | 0.34923333333333334 |
| 2018-04-01 | 0.263375            | 0.28685000000000005 | 0.32585             |
| 2018-05-01 | 0.254975            | 0.2773              | 0.30985             |
| 2018-06-01 | 0.24305000000000002 | 0.26825             | 0.297825            |
| 2018-07-01 | 0.243675            | 0.26715             | 0.29275             |
| 2018-08-01 | 0.25025             | 0.278825            | 0.310025            |
| 2018-09-01 | 0.276               | 0.3054              | 0.331725            |
| 2018-10-01 | 0.2942              | 0.331               | 0.361575            |
| 2018-11-01 | 0.301125            | 0.33906666666666674 | 0.37960625000000003 |
| 2018-12-01 | 0.31085             | 0.35185             | 0.3884              |

| Dimension | Lower quartile       | Median                | Upper quartile       |
| --------- | -------------------- | --------------------- | -------------------- |
| EMB01     | -0.1010567955672741  | 0.6903464198112488    | 1.542946696281433    |
| EMB02     | -0.8224388808012009  | -0.3278713971376419   | 0.03219197876751423  |
| EMB03     | -0.5026855617761612  | 0.405516654253006     | 1.1137759685516357   |
| EMB04     | -0.5799941122531891  | 0.5712794661521912    | 1.7421443462371826   |
| EMB05     | -3.369974732398987   | -2.23770809173584     | -1.5808860659599304  |
| EMB06     | -1.0276636183261871  | -0.43521231412887573  | 0.17904626950621605  |
| EMB07     | 0.20523467287421227  | 0.722084105014801     | 1.2673892080783844   |
| EMB08     | -1.2911924421787262  | -0.951613575220108    | -0.4867357760667801  |
| EMB09     | -1.276538908481598   | -0.17457542568445206  | 0.5484316349029541   |
| EMB10     | -2.2763607501983643  | -1.3987252116203308   | -0.6845518052577972  |
| EMB11     | 0.05352824553847313  | 1.0510995388031006    | 1.6758215725421906   |
| EMB12     | -1.2163829505443573  | -0.10990378260612488  | 0.6446310430765152   |
| EMB13     | -0.3276621401309967  | 0.590567409992218     | 1.3393518924713135   |
| EMB14     | -1.2953938841819763  | -0.4749186187982559   | 0.031965839210897684 |
| EMB15     | -3.2447964549064636  | -2.3276926279067993   | -1.5126361846923828  |
| EMB16     | -2.122460126876831   | -1.155880868434906    | -0.02705510938540101 |
| EMB17     | 1.6619341671466827   | 2.4177554845809937    | 3.1569390296936035   |
| EMB18     | -3.045407772064209   | -2.3984689712524414   | -1.6562288403511047  |
| EMB19     | -0.34316933155059814 | 0.17465977370738983   | 0.7103186696767807   |
| EMB20     | -0.458406038582325   | 0.7881646156311035    | 1.9392657279968262   |
| EMB21     | -2.9781250953674316  | -2.3538626432418823   | -1.7434381544589996  |
| EMB22     | -2.2531129717826843  | -1.683214247226715    | -1.0746824145317078  |
| EMB23     | -2.280011534690857   | -1.491638422012329    | -0.3708733469247818  |
| EMB24     | -1.0174265503883362  | -0.36722569167613983  | 0.5113553553819656   |
| EMB25     | -3.0637505054473877  | -2.3182730674743652   | -1.6988528668880463  |
| EMB26     | 3.372021973133087    | 3.7516742944717407    | 4.216524004936218    |
| EMB27     | -2.0002644658088684  | -1.1126003861427307   | -0.34505459666252136 |
| EMB28     | -0.23068254068493843 | 0.25353823602199554   | 0.873411625623703    |
| EMB29     | 0.9155551046133041   | 1.870758295059204     | 2.6216378211975098   |
| EMB30     | 0.8995515555143356   | 1.8964070677757263    | 2.719979226589203    |
| EMB31     | -0.9311211854219437  | -0.031039075925946236 | 0.8362912386655807   |
| EMB32     | -2.1230639815330505  | -1.6735978722572327   | -1.2758153676986694  |
| EMB33     | -0.6135038286447525  | 0.009279798017814755  | 0.3242737427353859   |
| EMB34     | -1.349402278661728   | -0.43534378707408905  | 0.07324598357081413  |
| EMB35     | 2.222937762737274    | 2.680785894393921     | 3.209222137928009    |
| EMB36     | -0.5838613659143448  | 0.26453743875026703   | 1.402107149362564    |
| EMB37     | 1.1195348501205444   | 2.028523802757263     | 3.450495421886444    |
| EMB38     | -0.3515005558729172  | 0.3035416603088379    | 1.2137926518917084   |
| EMB39     | 1.1786717176437378   | 1.8317095637321472    | 2.5384527444839478   |
| EMB40     | 0.8066371977329254   | 1.5032787919044495    | 2.247910439968109    |
| EMB41     | -0.9307397753000259  | 0.09255184605717659   | 1.5473872125148773   |
| EMB42     | -1.162200778722763   | -0.6568270325660706   | -0.2519182376563549  |
| EMB43     | 0.23149283975362778  | 1.2159401178359985    | 1.8777713477611542   |
| EMB44     | 1.3731281161308289   | 2.2070740461349487    | 3.311392307281494    |
| EMB45     | -0.6905752271413803  | -0.3438557982444763   | 0.13652799278497696  |
| EMB46     | -1.3971263468265533  | -0.6423064768314362   | -0.11140730790793896 |
| EMB47     | -1.7915675938129425  | -1.2789053916931152   | -0.6700140088796616  |
| EMB48     | -0.17994745448231697 | 0.4039788991212845    | 1.0786371529102325   |
| EMB49     | -0.26559919118881226 | 0.09459185600280762   | 0.3986098915338516   |
| EMB50     | -1.4071923196315765  | -0.9409140348434448   | -0.6671516448259354  |
| EMB51     | -0.8358849436044693  | -0.3874433934688568   | 0.2927369400858879   |
| EMB52     | -0.43168240785598755 | 0.46288450062274933   | 1.1314449906349182   |
| EMB53     | 0.3170604407787323   | 0.6349593698978424    | 1.0617763996124268   |
| EMB54     | -1.4677501618862152  | -0.4842316806316376   | 0.7321304082870483   |
| EMB55     | -0.4563753828406334  | 0.011840142775326967  | 0.5591948479413986   |
| EMB56     | -1.4872047007083893  | -0.7981394529342651   | -0.2618955299258232  |
| EMB57     | -2.1148524284362793  | -1.3212993144989014   | -0.8181926608085632  |
| EMB58     | -1.2974109053611755  | -0.7119967341423035   | -0.21835358813405037 |
| EMB59     | 1.1449319124221802   | 1.6727259159088135    | 2.2647478580474854   |
| EMB60     | -0.24845238029956818 | 0.3579731583595276    | 0.8864647150039673   |
| EMB61     | -2.368748664855957   | -1.8326662182807922   | -1.3054219186306     |
| EMB62     | 0.40922053903341293  | 1.0136563181877136    | 1.816453367471695    |
| EMB63     | -0.5398180484771729  | -0.16388067603111267  | 0.2647358775138855   |
| EMB64     | -2.750809907913208   | -2.156503200531006    | -1.2781843543052673  |

#### A's 6 bands on every date

| Date       | BLUE                 | GREEN                | RED                  | NIR08               | SWIR16              | SWIR22              |
| ---------- | -------------------- | -------------------- | -------------------- | ------------------- | ------------------- | ------------------- |
| 2017-01-01 | 0.029400000000000003 | 0.06280000000000001  | 0.0567               | 0.39630000000000004 | 0.2071              | 0.11220000000000001 |
| 2017-02-01 | 0.029400000000000003 | 0.06280000000000001  | 0.0567               | 0.39630000000000004 | 0.2071              | 0.11220000000000001 |
| 2017-03-01 | 0.027600000000000003 | 0.0536               | 0.0599               | 0.3356              | 0.2126              | 0.1199              |
| 2017-04-01 | 0.025400000000000002 | 0.0517               | 0.043000000000000003 | 0.3295              | 0.18660000000000002 | 0.1023              |
| 2017-05-01 | 0.02385              | 0.048                | 0.038900000000000004 | 0.3242              | 0.17815             | 0.08940000000000001 |
| 2017-06-01 | 0.0223               | 0.0443               | 0.034800000000000005 | 0.3189              | 0.16970000000000002 | 0.0765              |
| 2017-07-01 | 0.0292               | 0.054700000000000006 | 0.0684               | 0.2985              | 0.2024              | 0.11230000000000001 |
| 2017-08-01 | 0.0422               | 0.07880000000000001  | 0.0819               | 0.314               | 0.2615              | 0.167               |
| 2017-09-01 | 0.0429               | 0.06960000000000001  | 0.0777               | 0.3015              | 0.2359              | 0.14850000000000002 |
| 2017-10-01 | 0.038625             | 0.06445000000000001  | 0.065475             | 0.34347500000000003 | 0.2215              | 0.127675            |
| 2017-11-01 | 0.03435              | 0.059300000000000005 | 0.053250000000000006 | 0.38545             | 0.2071              | 0.10685             |
| 2017-12-01 | 0.030075             | 0.054150000000000004 | 0.041025             | 0.427425            | 0.1927              | 0.086025            |
| 2018-01-01 | 0.0258               | 0.049                | 0.028800000000000003 | 0.46940000000000004 | 0.17830000000000001 | 0.06520000000000001 |
| 2018-02-01 | 0.0166               | 0.0424               | 0.025                | 0.379               | 0.16110000000000002 | 0.0632              |
| 2018-03-01 | 0.0286               | 0.0489               | 0.048600000000000004 | 0.3022              | 0.2136              | 0.10930000000000001 |
| 2018-04-01 | 0.0196               | 0.0432               | 0.028900000000000002 | 0.3502              | 0.1726              | 0.0709              |
| 2018-05-01 | 0.017                | 0.0419               | 0.0303               | 0.3351              | 0.1724              | 0.075               |
| 2018-06-01 | 0.0219               | 0.0429               | 0.0368               | 0.2939              | 0.1821              | 0.083               |
| 2018-07-01 | 0.0267               | 0.0455               | 0.0405               | 0.2881              | 0.192               | 0.09390000000000001 |
| 2018-08-01 | 0.030425             | 0.05185              | 0.049850000000000005 | 0.303025            | 0.20650000000000002 | 0.10687500000000001 |
| 2018-09-01 | 0.03415              | 0.0582               | 0.0592               | 0.31795             | 0.221               | 0.11985000000000001 |
| 2018-10-01 | 0.037875             | 0.06455              | 0.06855              | 0.33287500000000003 | 0.23550000000000001 | 0.132825            |
| 2018-11-01 | 0.041600000000000005 | 0.0709               | 0.0779               | 0.3478              | 0.25                | 0.1458              |
| 2018-12-01 | 0.034300000000000004 | 0.0636               | 0.058800000000000005 | 0.3884              | 0.219               | 0.12100000000000001 |

#### How it was made

1. Read the labelled samples: The pipeline reads the labelled samples: time series, each with its land cover class. It lists the classes and gives each the colour sits draws it in.
2. Cut out the study window: The pipeline finds the window in its tile of a regular cube and cuts it out of every band and month. It keeps the stored values and marks the months a pixel has none.
3. Fill the missing months: Where a pixel has no value in a month, sits fills one in on a straight line between the months around it. At either end of the series it repeats the nearest value.
4. Encode the labelled samples: The pipeline draws the same number of labelled samples from each class of the period of the embeddings cube, and sits_encode() turns each into its 64 numbers with the published encoder.
5. Keep the regional sample IDs: The pipeline retains recorded optical series from the published sample file and their cached encodings. The original optical extraction is undocumented; both forms keep the same IDs.
6. Prepare the pictures: The pipeline picks the brightness drawn black and white, the same for every date, so the dates can be compared. Values are packed into one byte each.
7. Write the files: The pipeline writes the files the website reads, each with a record of its sources and of how closely a rerun repeats it.

#### Reproducibility details

- A rerun on aarch64-apple-darwin23 gave the same files, byte for byte.

Every step of the pipeline:

1. Lists the band files of the regular Landsat cube on Hugging Face, at a fixed commit. `hf_cube_files("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85")`
2. The published encoder of the Cerrado, as kept from Hugging Face. `hf_cache_path("e-sensing/encoders_cerrado", "ssl_lejepa_tcnn_model_2017_2024.rds", "model")`
3. The labelled Cerrado samples, as kept from Hugging Face. `hf_cache_path("e-sensing/samples_cerrado", "labelled_samples_cerrado.parquet", "dataset")`
4. The study window, chosen as pipeline/legacy/R/01_window.R describes. `list(tile = "009010", col_off = 2208L, row_off = 1536L, size = 256L, resolution = 30L, source = "BDC", collection = "LANDSAT-OLI-16D")`
5. Reads the samples into a table, one row per time series. `samples_read_cached(samples_file)`
6. Reads the grid of the window's tile from a band file. `grid_read_tile(hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", cube_files$file[cube_files$tile == study_window$tile][[1L]]))`
7. Lists the land cover classes found in the samples. `sits::sits_labels(samples)`
8. The labelled samples of the period, the same number of each class, with their place in the period. `encoder_draw_per_class(encoder_period_samples(samples, "2017-01-01"), per_class = 500L, seed = 20261011L)`
9. Places the window in the tile's grid. `window_place(tile_grid, study_window)`
10. Gives each class the colour sits uses to draw it. `samples_class_colors(labels)`
11. The drawn samples as the published encoder embeds them: 64 numbers each. `encoder_encode_samples(samples_subset, encoder_file, list(device = "default", memsize = 4L, multicores = 2L))`
12. Writes the window's edge as a closed ring of longitudes and latitudes. `site_write_json(samples_window_outline(window_grid), "../web/src/data/labelled-window-outline.json")`
13. Crops the window out of the cube's band files by georeference, keeping crops already made. `window_crop_files("cache/windows/window_009010_c2208_r1536/reg", hf_resolve_url("e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", hf_tile_files(cube_files, study_window$tile, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))), window_grid, "e-sensing/cerrado_cube_reg_2018", "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85", 8L, crop = window_crop_http_files)`
14. Names the bands and months of the crops. `window_crop_layers(window_crops, study_window$source, study_window$collection, c("2017-01-01", "2017-02-01", "2017-03-01", "2017-04-01", "2017-05-01", "2017-06-01", "2017-07-01", "2017-08-01", "2017-09-01", "2017-10-01", "2017-11-01", "2017-12-01", "2018-01-01", "2018-02-01", "2018-03-01", "2018-04-01", "2018-05-01", "2018-06-01", "2018-07-01", "2018-08-01", "2018-09-01", "2018-10-01", "2018-11-01", "2018-12-01"))`
15. Reads how sits scales the bands and marks missing values. `collection_band_values(study_window$source, study_window$collection, window_layers$bands)`
16. Binds recorded optical and cached encoded rows by their complete identity. `samples_cohort_record(samples_subset, samples_encoded, list(bands = window_layers[["bands"]], choice_band = "NIR08", classes = samples_class_table(labels, colors, list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated")), source = list( repo = "e-sensing/samples_cerrado", commit = "fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8", file = "labelled_samples_cerrado.parquet")))`
17. Checks exact storage in the declared band order. `samples_storage_order(samples_storage_check(labelled_sample_record[["optical"]], samples_grid_metadata(list(crops = window_crops), "cache/src/sits-78d26eb07370874192911854b48b60ebb1d668c8")), labelled_sample_record[["bands"]])`
18. Writes exact float32 values from the cached encoded cohort. `samples_cohort_embeddings(labelled_sample_record, "../web/public/data/labelled-samples-embeddings.bin")`
19. Writes supplied labels separately in the common row order. `samples_cohort_labels(labelled_sample_record, "../web/public/data/labelled-sample-labels.bin")`
20. Writes original IDs and full-precision coordinates. `samples_cohort_points(labelled_sample_record, "../web/public/data/labelled-sample-points.bin")`
21. Reads the stored values of the crops, missing ones as NA. `window_read_raw(window_crops, window_layers$bands, window_layers$dates, band_scale, study_window$size)`
22. Writes each recorded optical band exactly in sample/date order. `samples_cohort_optical(labelled_sample_record, labelled_sample_storage, "../web/public/data")`
23. Writes axes, counts and descriptors for the common recorded cohort. `site_write_json(samples_cohort_json(labelled_sample_record, labelled_sample_storage, list(source = list(repo = "e-sensing/samples_cerrado", commit = "fc6cd2da2a8b6f342dcf5cac721fcf3d5a05cbe8", file = "labelled_samples_cerrado.parquet"), choice_band = "NIR08", titles = list(Cerradao = "Cerradão", Nat_NonVeg = "Natural non-vegetated"), settings = list(samples = list(per_class = 500L, period_start = "2017-01-01", train_frac = 0.8), seed = 20261011L, encoder = list(repo = "e-sensing/encoders_cerrado", type = "model", commit = "fbcea998a4173854a6a7d7cbd3bbeae4496de0c5", files = list(lejepa = "ssl_lejepa_tcnn_model_2017_2024.rds"))))), "../web/src/data/labelled-samples.json")`
24. Marks the months each pixel was observed in. `window_observed_mask(window_raw)`
25. Fills the missing months along time with sits::impute_linear(), as sits does before the encoder. `window_fill_series(window_raw, band_scale)`
26. Stores every other pixel's series in one byte per value, each band between two quantiles. `window_export_series(window_series, c(0.001, 0.999 ), 2L)`
27. Picks the reflectance drawn black and white in each colour, the same for every month. `window_export_sprite_stretch(window_series, window_layers$bands, list(red = "SWIR16", green = "NIR08", blue = "RED"), c(0.02, 0.98))`
28. Writes where the window is, and how its series and monthly images are stored, for the website. `site_write_json(window_export_record(study_window, window_grid, window_layers, band_scale, window_mask, series_quantised, series_stretch, cube_files, list(repo = "e-sensing/cerrado_cube_reg_2018", revision = "refs/pr/3", commit = "af1d4f67fee0fcaba35941f4ac5d7d9e87d4fa85"), list(png_scale = 2L, series_quantiles = c(0.001, 0.999), series_stride = 2L, series_units = "surface reflectance", series_rgb = list(red = "SWIR16", green = "NIR08", blue = "RED"), series_stretch = c(0.02, 0.98), sprite_columns = 6L, sprite_levels = 32L, sprite_small_tile = 220L, tour_band = "NIR08"), list(series = "series.bin", mask = "series-mask.bin", sprite = "rgb-months.png", sprite_small = "rgb-months-small.png")), "../web/src/data/window.json")`

The files:

- `labelled-samples.json`: SHA-256 `19c17c7964f2917f2ac50dcd32a7c7e37255c0f41b834b2c6e5ac8cfac84862c`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-window-outline.json`: SHA-256 `16a450cec894634f771b9ecb3bf6e6ff0341c7addc39544575c3412f1b373377`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `window.json`: SHA-256 `60e731843a80b49590366d3313d69d24d23f2c23c914b42ecdff8ff53b06f9d5`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-sample-points.bin`: SHA-256 `f30f63bfdccf84c25268347a694c4de1864d2d3e70a645d691f10cdccbefcf68`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-sample-labels.bin`: SHA-256 `580acce299741ace3450ff8dbfcaae394e1cdf12b652da53d78e5fd7d4ec6b38`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-BLUE.bin`: SHA-256 `85c4ed0646d84e38da8e45a754299324fa86947bbba1787630186bdc80d69bb8`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-GREEN.bin`: SHA-256 `1d437e78acfe8ae2375085ce2d43e6015bd1ecb712cd1ff0992d39ce89929385`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-RED.bin`: SHA-256 `052c0eda2adc4e1fabfde60d48e43963e0797cda0282631118c11ba74ab0ff09`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-NIR08.bin`: SHA-256 `862bc52a73cdf4d585a918da2cb848e1908b37dabf2c6a02e2699f7a61bea1c5`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-SWIR16.bin`: SHA-256 `117a96c40441a5d3b81671beb61cffe4a1a9fee20e83b8c22f5ebe68176c5a4d`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-SWIR22.bin`: SHA-256 `678ca9c394ace2199b4042f7802bf7d974890fefbc6e897e83e94f6ddb998165`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.
- `labelled-samples-embeddings.bin`: SHA-256 `4b8f135a9977b2f40d8fea516a098188ddc3cc959883cc2361e722cc51eb963d`. A rerun on aarch64-apple-darwin23 gave the same file, byte for byte.

To make them again: `cd pipeline && Rscript make.R`.

## Next

On this site: [From samples to a land use map](/classification.md). How do labelled samples become a map of land use?

In the sits book: [Basic operations on image time series](https://e-sensing.github.io/sitsbook/ts_basics.html) and [Building regional embeddings: a full example](https://e-sensing.github.io/sitsbook/emb_example.html).