Building data cubes from image collections
How do the images a provider hands over become a regular data cube?
45 images as they come, on 25 datesRegular cube: 4 periods of 16 days
A provider hands over images cut into tiles and taken on uneven dates. Follow the book's example as sits turns them into a regular data cube, with one image per period.
Step 01 of 5
What does a provider hand over?
The book asks AWS for the Sentinel-2 images of 2 tiles from 30 Jun to 31 Aug 2018, in 3 bands plus the cloud band, CLOUD.
The book's code
s2_cube_rondonia <- sits_cube(
source = "AWS",
collection = "SENTINEL-2-L2A",
tiles = c("20LLP", "20LKP"),
bands = c("B02", "B8A", "B11", "CLOUD"),
start_date = as.Date("2018-06-30"),
end_date = as.Date("2018-08-31")
)Where the sits book writes this call: dc_regularize.qmd, lines 62–69
Sentinel-2 files its images by the tiles of a fixed grid. Each tile is 109.8 km a side and has a name.
The book’s call asks for 2 tiles, 20LKP and 20LLP, each drawn here with one of its pictures.
They overlap: a strip 9.8 km wide lies in both.
The page follows one small region, 15.4 by 8 km, across that strip. Each tile covers part of it: 20LKP 97% and 20LLP 67% of its width.
One tile: 20LLP has 45 items on 25 dates from 30 Jun to 31 Aug 2018. Each item is a picture of the whole tile on one date; the row shows its part over the region.
23 of them hold no data there. Under each, its date and the cloud cover of the whole scene.
sits_cube() returns this collection as a table: one row per tile, with the files of its images listed.
Switch the tile to see 20LKP's 24 items on 12 dates.
This figure draws the tiles' pictures and the row of images from cube-map-20lkp.png, cube-map-20llp.png, cube-ard-20lkp.png and cube-ard-20llp.png, with JavaScript.
Step 02 of 5
Why can't the images be used as they come?
The book asks sits for the dates of those images, tile by tile.
The book's code
sits_timeline(s2_cube_rondonia)Where the sits book writes this call: dc_regularize.qmd, line 71
Each tile has its own dates. From 30 Jun to 31 Aug 2018, 20LKP has 12 dates and 20LLP 25. They share 12: every date of 20LKP is one of 20LLP's, and 20LLP has 13 more.
The figure draws a recorded run of the book’s call: 69 images of 2 tiles.
Some images cover only part of the tile. Tile 20LLP on 30 Jun 2018: the image covers 52% of it. The hatched part has no data.
Bands come at different resolutions. The same 480 m square of 20LLP: B04 at 10 m is 48 pixels a side, B11 at 20 m only 24. B02, B03 and B04 come at 10 m; B11, B8A and CLOUD at 20 m.
Clouds and their shadows hide the ground. sits drops the pixels the CLOUD band marks: on 7 Aug 2018 that is 38% of the region in one of 20LKP's two images that day, most of it thin cirrus.
A cube needs one grid, one set of dates, and a value for each cell. Images as they come promise none of the three. The next steps build a cube that does.
This figure draws its pictures from regularize-coverage.png (339 kB), regularize-patch-b04.png (2 kB), regularize-patch-b11.png (1 kB), cube-ard-20lkp.png (2.2 MB) and cube-ard-20lkp-mask.png (12 kB), with JavaScript.
- 20LKP20LLP
Dates12 and 25 dates
Step 03 of 5
What must a regular cube agree on?
The book asks sits to make those images a regular cube: cells of 40 m, and one image for every 16 days.
The book's code
reg_cube_rondonia <- sits_regularize(
cube = s2_cube_local,
output_dir = tempdir_r_s2_reg,
res = 40,
period = "P16D",
multicores = 6)Where the sits book writes this call: dc_regularize.qmd, lines 204–209
The cube keeps the projection of each tile: WGS 84 / UTM zone 20S, EPSG:32720, for every tile of the call.
And it keeps the tile’s extent. 20LLP runs from x 300,000 to 409,800 m and from y 8,790,220 to 8,900,020 m: 109.8 km a side.
The figure draws 9 recorded runs of sits_regularize() on the 69 images of the book’s call: 3 periods at 3 cell sizes.
res = 40 sets the cell: a square of 40 m. Over the region, the cube of 20LLP is 256 by 200 cells; 20LKP has 372 by 200.
Each tile keeps a grid of its own, on its own pixels.
Every band is resampled to the cell. Under one 40 m cell lie 4 × 4 pixels of B02 (10 m) and 2 × 2 pixels of B8A and B11 (20 m).
In 20LLP the cells sit exactly on them; in 20LKP their edges lie off the 20 m pixels by 10 m in x.period = "P16D" cuts time into 4 intervals of 16 days, each from its date t up to t + 16 days, that day left out. Each image belongs to the one it falls in.
The timeline starts at the latest first date of the tiles, 3 Jul 2018, and stops at the earliest last date, 27 Aug 2018: the two images of 20LLP of 30 Jun 2018 fall outside.
This is what sits asks gdalcubes to build, one view per tile, band and date.
Change the period or the cell and see the grid and the dates change.
EPSG:32720 · WGS 84 / UTM zone 20S
20LKP 372 × 200 · 20LLP 256 × 200 cells of 40 m
This figure draws its pictures from regularize-run-p16d-20m-20lkp.png (2.2 MB), regularize-run-p16d-20m-20llp.png (1.5 MB), cube-regular-20lkp.png (314 kB), cube-regular-20llp.png (259 kB), regularize-run-p16d-80m-20lkp.png (66 kB) and regularize-run-p16d-80m-20llp.png (55 kB), with JavaScript.
Step 04 of 5
What does sits_regularize() do?
The book asks sits to give each 40 m cell one value per 16-day period, from the least cloudy image of the period that sees the ground there.
The book's code
reg_cube_rondonia <- sits_regularize(
cube = s2_cube_local,
output_dir = tempdir_r_s2_reg,
res = 40,
period = "P16D",
multicores = 6)Where the sits book writes this call: dc_regularize.qmd, lines 204–209
The images of one tile, 20LLP, as they come: 45 items on 25 days, each a card at its date.
23 of them hold no data over the region, drawn empty: the satellite’s swath passed beside it.
The figure draws one recorded run of sits_regularize(): the book’s call on 69 images of 2 tiles.
period = "P16D" cuts the axis into four bins of 16 days, each from its date t up to t + 16 days, that day left out.
Each image goes to the bin it falls in; the two items of 30 Jun 2018 fall before the first bin, and are left out.
In each bin sits takes the images by scene cloud cover, least first: the cloud_cover the provider’s catalogue gives for the whole scene. Equal covers keep the collection’s order.
The 21 items in the bins with no data here keep their place in the order, with nothing to give.
For each cell, sits keeps the first value, in that order, that the image has there and the CLOUD band does not mask: aggregation “first”. It writes one image per period at 40 m and crops each file to the region: on 20LKP the 373 × 200 cells of the view become 372 × 200; the files of 20LLP already lay inside the region and were copied as they are.
In these two dry months the first image fills nearly every cell of every period, on both tiles: where it fills least, 74,393 of 74,400 (20LKP, 3 Jul).
Open a period to see how its cells were chosen, image by image.
Switch to 20LKP: each tile is regularized on its own grid, apart from its neighbour.
In these two dry months the first image gives almost every cell. Over the year it does not: here are the 176 images of 20LLP in 2018, an extension of the book’s example, in 23 periods. The first image gives half the cells or more in 11 of them, and nothing in 10.
In 12 Apr to 27 Apr 2018, the first image has no data here and gives nothing; the 2nd and 3rd give 52% and 38%, and 10% of the cells stay empty. Open a period of the rainy season.
20LLP · 45 images · P16D · 40 m
This figure draws its pictures from cube-ard-20lkp.png (2.2 MB), cube-regular-20lkp.png (314 kB), cube-ard-20llp.png (1.5 MB) and cube-regular-20llp.png (259 kB), with JavaScript.
Try it: Where each cell of a regular image came from, or the tool Regularization settings
Step 05 of 5
What does a regular cube guarantee?
One image per period, every period the same grid. In the recorded run the figure draws, sits_regularize() made the 176 images of the tile into 23, one every 16 days, each 256 × 200 cells of 40 m.
This is the whole of 2018 on tile 20LLP: an extension of the book’s example, whose call covers 2 months (30 Jun 2018 to 31 Aug 2018).
A cell is the same ground on every date: its values are a time series.
The outlined cell goes through all 23 sheets; the chart draws its B02 reflectance, date by date.
And in every band, all at 40 m: B02, B03, B04, B8A and B11.
B03 and B04 were read in addition to the book’s bands, to draw the pictures.
The sits book lists 5 properties of a regular cube. Beside each, what this cube has.
Regular is not complete. sits_regularize() mosaics the images of each period; where none had a clear view of a cell, the cell stays empty.
On 16 of the 23 dates part of the region has no value; on 22 Jan 2018 none of it has. The bars give each date’s share.
Read the cube by date or by place.
Choose a date on the strip, a cell on the open sheet, or one band.
Loading cube-year.bin (5.9 MB).