# Regularization settings

**Task.** What does sits_regularize() make of the images when you change the period or the cell?

`sits_regularize()` turns the irregular Sentinel-2 images of two tiles into a cube with one image per period, on one grid of cells. Choose the period, the cell size and the tile to see every date of the cube it makes, from a recorded run of the pipeline. Open a date to see which images it was made from, in sits' order, and how each filled the cells.

Journey: [Building data cubes from image collections, step 4: What does sits_regularize() do?](/regular-cubes#regularize).

**Data.** One sits run per setting, recorded by the pipeline.

## Default: the book's call

```r
sits_regularize(
  cube,
  output_dir = dir,
  res = 40,
  period = "P16D"
)
```

Read in sits 2.0.0 (78d26eb), R/sits_regularize.R:179.

The book regularizes a copy of its images over whole tiles; its call, shortened here, also sets multicores. Each setting here is a recorded run of the same function on the remote images over a region, with the three bands of a natural-colour picture and the cloud band.

The page shows the cube sits_regularize() writes for one setting: a period (P8D, P16D, P1M), a cell size (20 m, 40 m, 80 m) and a tile (20LKP, 20LLP). Each setting is a recorded run: 9 runs over the same region, each with the size of the files it wrote. A row of dates shows every date of the cube as its image, under a time axis with the images of the tile, the intervals the period cuts and the images left outside. Choose a date to see how it was made. A date opens its composition in five steps: the order sits takes the images in, each image on the grid, the masked cells, the first value of each cell, and the files written.

Not set here: timeline (with a month period and a mid-month date it can make sits look for a file gdalcubes named otherwise); roi (another region needs runs not recorded); crs (it only says how to read the roi); grid_system (it needs runs not recorded); tiles (it does not change the timeline; the Tile buttons show each tile of the run); multicores (it changes no value); progress (it changes no value); output_dir (sits resumes from files of the same name there).

## The cube of the book's call

P16D at 40 m, the book's call. Start 3 Jul 2018: the latest first date of the tiles (20LKP). The timeline stops before passing 27 Aug 2018, the earliest last date of the tiles (20LKP). period = "P16D" cuts it into 4 intervals of 16 days, each from its date t up to t + 16 days, that day left out.

### 20LKP

| Date | Interval | Images | Cells with a value | Cells without | Files |
| --- | --- | --- | --- | --- | --- |
| 3 Jul 2018 | 3 Jul to 18 Jul 2018 | 6 | 74,400 of 74,400 | 0 | 3 |
| 19 Jul 2018 | 19 Jul to 3 Aug 2018 | 6 | 74,400 of 74,400 | 0 | 3 |
| 4 Aug 2018 | 4 Aug to 19 Aug 2018 | 8 | 74,400 of 74,400 | 0 | 3 |
| 20 Aug 2018 | 20 Aug to 4 Sep 2018 | 4 | 74,400 of 74,400 | 0 | 3 |

Every image of 20LKP falls in an interval.

### 20LLP

| Date | Interval | Images | Cells with a value | Cells without | Files |
| --- | --- | --- | --- | --- | --- |
| 3 Jul 2018 | 3 Jul to 18 Jul 2018 | 11 | 51,200 of 51,200 | 0 | 3 |
| 19 Jul 2018 | 19 Jul to 3 Aug 2018 | 9 | 51,200 of 51,200 | 0 | 3 |
| 4 Aug 2018 | 4 Aug to 19 Aug 2018 | 16 | 51,200 of 51,200 | 0 | 3 |
| 20 Aug 2018 | 20 Aug to 4 Sep 2018 | 7 | 51,200 of 51,200 | 0 | 3 |

Left outside: 2 images of 20LLP, of 30 Jun 2018.

The cube as files: 2 tiles × 3 bands × 4 dates = 24 files (CLOUD is not written).

## Where sits does each step

```r
# Read in sits 2.0.0 (78d26eb):
# Checks of the arguments: R/sits_regularize.R:193-206
# The roi, read in crs: R/sits_regularize.R:217
# Timeline: R/api_gdalcubes.R:318-361
# Images into dates: R/api_gdalcubes.R:21-44
# One file per tile, band and date: R/api_gdalcubes.R:757-765
# 1 Order: R/api_gdalcubes.R:29-44
# 2 On the grid: R/api_gdalcubes.R:71-112
# 3 Masked: R/api_gdalcubes.R:124-158
# 4 First value: R/api_gdalcubes.R:573
# 5 Written: R/api_gdalcubes.R:586-591
# Resampling, bilinear: R/api_gdalcubes.R:565-575
# Overviews, at a cell coarser than every band: gdalcubes 0.7.4 src/gdalcubes/src/warp.cpp:123-160 and R/zzz.R:18, R/api_gdalcubes.R:599-604
# Crop to the region: R/api_gdalcubes.R:699-706
```

## How 3 Jul 2018 was made (20LLP, the book's call)

3 Jul to 18 Jul 2018: 11 images of 20LLP, in the order sits takes them. sits takes the 11 images of 3 Jul to 18 Jul 2018 by their scene's cloud cover, least first: the cloud_cover the catalogue gives for the whole scene, not for the region. Equal covers keep the collection's order.

| Place | Date | Scene cloud cover | Masked | No data | Gave |
| --- | --- | --- | --- | --- | --- |
| 1 | 18 Jul 2018 | 0.00106% | 0 | 0 | 51,200 |
| 2 | 8 Jul 2018 | 0.00117% | 6 | 0 | 0 |
| 3 | 3 Jul 2018 | 0.00121% | 0 | 0 | 0 |
| 4 | 18 Jul 2018 | 0.00172% | 7 | 0 | 0 |
| 5 | 3 Jul 2018 | 0.00217% | 5 | 0 | 0 |
| 6 | 10 Jul 2018 | 0.00710% | 0 | all | 0 |
| 7 | 13 Jul 2018 | 0.04105% | 6 | 0 | 0 |
| 8 | 5 Jul 2018 | 1.38146% | 0 | all | 0 |
| 9 | 5 Jul 2018 | 1.88259% | 0 | all | 0 |
| 10 | 15 Jul 2018 | 2.13433% | 0 | all | 0 |
| 11 | 15 Jul 2018 | 3.35344% | 0 | all | 0 |

The first image gives every cell: 51,200 of 51,200.

Each image is warped onto the grid of 20LLP: 256 × 200 cells of 40 m in EPSG:32720, from x 300,000, y 8,848,000. sits reads B02, B03 and B04 at 10 m and resamples them bilinear to 40 m; the CLOUD band, file SCL.tif, at 20 m, nearest, as a mask. At 40 m gdalcubes has GDAL read each file from its overview closest to the cell size, never coarser, not at full resolution. The book's cube also has B8A and B11; these runs read only the bands of a natural-colour picture. In short: 10 m → 40 m bilinear · 20 m → 40 m nearest.

sits masks the cells where the CLOUD band holds 0, 1, 2, 3, 8, 9 and 10 (missing data, defective pixel, shadows, cloud shadows, cloud medium, cloud high and thin cirrus): a masked cell has no value in any band. It keeps 4, 5, 6, 7 and 11 (vegetation, non-vegetated, water, unclassified and snow or ice).

The files written for this date, one per band, in whole numbers. sits crops every file to the region: the files of 20LLP already lay inside the region and were copied as they are.

| File | Cells | Nodata | Size |
| --- | --- | --- | --- |
| SENTINEL-2_MSI_20LLP_B02_2018-07-03.tif | 256 × 200 | −9999 | 74 kB |
| SENTINEL-2_MSI_20LLP_B03_2018-07-03.tif | 256 × 200 | −9999 | 79 kB |
| SENTINEL-2_MSI_20LLP_B04_2018-07-03.tif | 256 × 200 | −9999 | 83 kB |

```r
# Read in sits 2.0.0 (78d26eb):
# 1 Order: R/api_gdalcubes.R lines 29-44
# 2 On the grid: R/api_gdalcubes.R lines 71-112
# 3 Masked: R/api_gdalcubes.R lines 124-158
# 4 First value: R/api_gdalcubes.R line 573
# 5 Written: R/api_gdalcubes.R lines 586-591
# Crop to the roi: R/api_gdalcubes.R lines 699-706
# Files written as Int16, nodata -9999.
```

## How 4 Aug 2018 was made at P8D, 40 m (20LKP)

4 Aug to 11 Aug 2018: 2 images of 20LKP, in the order sits takes them. sits takes the 2 images of 4 Aug to 11 Aug 2018 by their scene's cloud cover, least first: the cloud_cover the catalogue gives for the whole scene, not for the region. Equal covers keep the collection's order.

| Place | Date | Scene cloud cover | Masked | No data | Gave |
| --- | --- | --- | --- | --- | --- |
| 1 | 7 Aug 2018 | 74.1% | 18,056 | 0 | 56,344 |
| 2 | 7 Aug 2018 | 78.3% | 28,214 | 0 | 46 |

The first image gives 56,344 of 74,400 cells; the later images 46; 18,010 cells get no value and are written as −9999.

Each image is warped onto the grid of 20LKP: 372 × 200 cells of 40 m in EPSG:32720, from x 294,910, y 8,848,000. gdalcubes builds the view over the region widened to whole cells, 373 × 200, from x 294,870, y 8,848,000. sits reads B02, B03 and B04 at 10 m and resamples them bilinear to 40 m; the CLOUD band, file SCL.tif, at 20 m, nearest, as a mask. At 40 m gdalcubes has GDAL read each file from its overview closest to the cell size, never coarser, not at full resolution. The book's cube also has B8A and B11; these runs read only the bands of a natural-colour picture. In short: 10 m → 40 m bilinear · 20 m → 40 m nearest.

sits masks the cells where the CLOUD band holds 0, 1, 2, 3, 8, 9 and 10 (missing data, defective pixel, shadows, cloud shadows, cloud medium, cloud high and thin cirrus): a masked cell has no value in any band. It keeps 4, 5, 6, 7 and 11 (vegetation, non-vegetated, water, unclassified and snow or ice).

The files written for this date, one per band, in whole numbers. sits crops every file to the region: the 373 × 200 cells of the view become 372 × 200.

| File | Cells | Nodata | Size |
| --- | --- | --- | --- |
| SENTINEL-2_MSI_20LKP_B02_2018-08-04.tif | 372 × 200 | −9999 | 94 kB |
| SENTINEL-2_MSI_20LKP_B03_2018-08-04.tif | 372 × 200 | −9999 | 96 kB |
| SENTINEL-2_MSI_20LKP_B04_2018-08-04.tif | 372 × 200 | −9999 | 99 kB |

```r
# Read in sits 2.0.0 (78d26eb):
# 1 Order: R/api_gdalcubes.R lines 29-44
# 2 On the grid: R/api_gdalcubes.R lines 71-112
# 3 Masked: R/api_gdalcubes.R lines 124-158
# 4 First value: R/api_gdalcubes.R line 573
# 5 Written: R/api_gdalcubes.R lines 586-591
# Crop to the roi: R/api_gdalcubes.R lines 699-706
# Files written as Int16, nodata -9999.
```

## Every recorded run

9 runs of sits_regularize(), each over the region on 2 tiles, with the size of the files it wrote.

| Period | Cell | Dates | Cells per tile | Dates with cells without a value | Size |
| --- | --- | --- | --- | --- | --- |
| P8D | 20 m | 7 | 20LKP 745 × 400, 20LLP 512 × 400 | 20LKP 4, 20LLP 4 | 19.9 MB |
| P8D | 40 m | 7 | 20LKP 372 × 200, 20LLP 256 × 200 | 20LKP 1, 20LLP 1 | 3.8 MB |
| P8D | 80 m | 7 | 20LKP 186 × 100, 20LLP 128 × 100 | 20LKP 1, 20LLP 1 | 983 kB |
| P16D | 20 m | 4 | 20LKP 745 × 400, 20LLP 512 × 400 | 20LKP 1, 20LLP 1 | 11.5 MB |
| P16D | 40 m | 4 | 20LKP 372 × 200, 20LLP 256 × 200 | 20LKP 0, 20LLP 0 | 2.2 MB |
| P16D | 80 m | 4 | 20LKP 186 × 100, 20LLP 128 × 100 | 20LKP 0, 20LLP 0 | 556 kB |
| P1M | 20 m | 2 | 20LKP 745 × 400, 20LLP 512 × 400 | 20LKP 0, 20LLP 0 | 5.7 MB |
| P1M | 40 m | 2 | 20LKP 372 × 200, 20LLP 256 × 200 | 20LKP 0, 20LLP 0 | 1.1 MB |
| P1M | 80 m | 2 | 20LKP 186 × 100, 20LLP 128 × 100 | 20LKP 0, 20LLP 0 | 278 kB |

## What to try

1. Open 3 Jul on 20LLP, the date the book plots: the first image gives every cell, 51,200 of 51,200. Step through "First value" to see each image take its turn.
2. Press P8D and open 4 Aug on 20LLP: 6 images, 4 of them with no data over the region; 4,274 of 51,200 cells get no value, and the files are written all the same, −9999 there.
3. Press P1M: 2 dates from 1 Jul, not 3 Jul: a period of months starts on the 1st. The images of 30 Jun are still outside.
4. With a date open, press 80 m, then 20 m: the same images, on cells of another size (128 × 100 and 512 × 400). "On the grid" shows the files read at their native 10 and 20 m.
5. Switch the tile to 20LKP: its own grid, 372 × 200 cells, and its own files; the dates do not move, as one timeline serves every tile.

## How much it loads

Its first row of dates loads about 260 kB of pictures. Another setting loads its own row of dates, 28 kB to 3.8 MB, and a date you open loads the pictures of its images, 330 kB to 6.5 MB.

Also in the journeys: [Building data cubes from image collections, step 3: What must a regular cube agree on?](/regular-cubes#grid).

In the sits book: [Building regular data cubes](https://e-sensing.github.io/sitsbook/dc_regularize.html).