# Similar places

**Task.** Which places look like this one to the encoder?

Each pixel of a 256 × 256 window in the Cerrado, near 13.8° S, 57.8° W, has an embedding: 64 numbers the LeJEPA encoder from sits (`sits_ssl_lejepa`, pretrained on 647,967 time series) gives its time series, for the 2018 map. Pick a pixel and the page finds the pixels whose embeddings are closest to it, by Euclidean or cosine distance, and shows their land cover on the map sits made from the same embeddings.

Journey: [From series to embeddings, step 5: What are these 64 numbers?](/embeddings#vector).

**Data.** Computed in your browser from recorded data.

## The sits call

```r
# cube: e-sensing/cerrado_cube_reg_2018,
#   a 256 × 256 window of tile 009010
# encoder: e-sensing/encoders_cerrado,
#   ssl_lejepa_tcnn_model_2017_2024.rds
sits_encode(data = cube, encoder = encoder)
```

Read in sits 2.0.0 (78d26eb), R/sits_encode.R:304.

Run again on this window, it gave the published embeddings back, all 4,194,304 values the same. No sits function finds the pixels of a cube closest to one pixel: this page computes the distances in your browser.

The page shows the study window of the site, 256 × 256 pixels (65,536), in the Cerrado of Brazil. One pixel is selected and marked with a crosshair. The pixels whose embedding is closest to it, the similar pixels, are left clear and the others are dimmed. A slider labelled Closest sets how many stay clear, from 1% to 50% of the window, starting at 5%. Moving the pointer over the picture shows the result for the pixel under it; a click, a tap or the arrow keys keep a pixel. A switch labelled Picture shows under the similar pixels the satellite image of May 2017 or the land cover map sits drew of the window. A panel lists the land cover classes of the similar pixels, taken from that map, as shares of the set, and says what share has the class of the selected pixel. A switch labelled Measure chooses how closeness is measured, Euclidean (the default) or Cosine.

Close can mean two things. Euclidean counts how far apart the 64 values of two embeddings are, so it also counts how large they are. Cosine compares the pattern of the values and ignores how large they are, so two pixels whose lists have the same shape but different sizes are close for Cosine and may be far for Euclidean. For Cosine, the largest distance in the tables is the cosine distance, 0 for the same pattern and larger the more the patterns differ. With either measure the set for a share is exactly that share of the 65,536 pixels, the nearest first, the selected pixel included.

## The first state of the page

The page opens with the pixel at the centre of the window selected (column 129, row 129), which the map labels Cerrado.

The same state with each measure.

### Euclidean

| Closest | Pixels | Largest distance | Same class as the selected pixel | Largest classes |
| --- | --- | --- | --- | --- |
| 1% | 655 | 4.64 | > 99% | Cerrado > 99%, Cerradão < 1% |
| 5% | 3,277 | 7.54 | > 99% | Cerrado > 99%, Cerradão < 1% |
| 10% | 6,554 | 10.10 | 99% | Cerrado 99%, Cerradão 1% |
| 25% | 16,384 | 15.46 | 85% | Cerrado 85%, Cerradão 11%, Open cerrado 3% |

### Cosine

| Closest | Pixels | Largest distance | Same class as the selected pixel | Largest classes |
| --- | --- | --- | --- | --- |
| 1% | 655 | 0.018 | > 99% | Cerrado > 99%, Cerradão < 1% |
| 5% | 3,277 | 0.048 | > 99% | Cerrado > 99%, Cerradão < 1% |
| 10% | 6,554 | 0.091 | 99% | Cerrado 99%, Cerradão 1% |
| 25% | 16,384 | 0.252 | 85% | Cerrado 85%, Cerradão 11%, Open cerrado 3% |

## What the page does not set

- The encoder and the window: the embeddings are the published ones of this window, and the
  search covers this window only.
- The processing arguments of the sits call (memory, cores, output folder), which do not
  change the embeddings.
- The class map: the one sits drew from the embeddings, which the page does not redraw.
  Similar to the encoder is not the same as the same class: the tables show how the two
  compare for one pixel.

## Sources

- `embeddings.bin`: the window cut from the published embeddings cube
  e-sensing/cerrado_emb_ssl_lejepa_tcnn_2018, one byte a value with a scale and an offset per band.
- `classification-embeddings-map.bin`: the land cover map journey 4 classified from the embeddings, with
  sits_classify(), sits_smooth() in windows of 9 × 9 pixels and
  sits_label_classification().
- `rgb-months.png`: the window's monthly images from the regular cube
  e-sensing/cerrado_cube_reg_2018, drawn with SWIR16, NIR08 and RED as red, green, blue.

## What to try

1. Drag the slider from 1% to 50%. For the centre pixel the set of similar pixels grows from 655 to 32,768 pixels, and the share of it with the class of that pixel goes from 100% to 47%.
2. Choose Cosine. At 5% the set keeps its 3,277 pixels. For the centre pixel the share with its class is 100% with either measure; find a pixel where switching the measure changes the panel.
3. Choose Map to see the land cover map sits drew under the similar pixels. The centre pixel is Cerrado; the panel lists the classes of the set, the largest first.
4. Click another pixel, or use the arrow keys on the picture: one pixel a press, 8 with Shift. Moving the pointer over the picture only previews a pixel; the kept one comes back when the pointer leaves.

## How much it loads

The page loads about 5.7 MB when its figure is first shown: `embeddings.bin` (4.2 MB), `classification-embeddings-map.bin` (66 kB) and `rgb-months.png` (1.4 MB).

In the sits book: [Foundational Models and Embeddings](https://e-sensing.github.io/sitsbook/embeddings.html).