Raster analysis in Julia#
In this lesson, we discuss cover the basics of Raster analysis data structures in Julia.
Context#
We will use the same example that is used in the xarray tutorial.
Data#
As example data we use the data in the data repository.
Here we’ll use air temperature from the National Center for Environmental Prediction.
Setup#
This episode uses the following main Python packages:
Please install these packages if not already available in your Julia environment. Also note that Julia standard libraries such as Statistics Julia standard Library are already installed by default with the julia kernel.
Packages#
In this episode, some Julia packages are imported when we start to use them. However, for best software practices, we recommend you to install and import all the necessary libraries at the top of your Jupyter notebook.
Before we can start we need to activate the current environment to install all the necessary dependencies (see Project.toml file in the current directory.
using Pkg
Pkg.activate(".")
Pkg.instantiate()
Activating project at `~/JuliaGeoTutorials/Raster-in-45-min`
using Rasters
using NCDatasets
path = "./data/air_temperature.nc"
# Now we can open the data as a RasterStack
ds = RasterStack(path)
╭────────────────────────╮
│ 53×25×2920 RasterStack │
├────────────────────────┴─────────────────────────────────────────────── dims ┐
↓ X Mapped{Float32} [200.0, 202.5, …, 327.5, 330.0] ForwardOrdered Regular Points,
→ Y Mapped{Float32} [75.0, 72.5, …, 17.5, 15.0] ReverseOrdered Regular Points,
↗ Ti Sampled{Dates.DateTime} [2013-01-01T00:00:00, …, 2014-12-31T18:00:00] ForwardOrdered Irregular Points
├────────────────────────────────────────────────────────────────────── layers ┤
:air eltype: Float64 dims: X, Y, Ti size: 53×25×2920
├──────────────────────────────────────────────────────────────────── metadata ┤
Metadata{Rasters.NCDsource} of Dict{String, Any} with 5 entries:
"references" => "http://www.esrl.noaa.gov/psd/data/gridded/data.ncep.reanaly…
"platform" => "Model"
"title" => "4x daily NMC reanalysis (1948)"
"description" => "Data is from NMC initialized reanalysis\n(4x/day). These a…
"Conventions" => "COARDS"
├────────────────────────────────────────────────────────────────────── raster ┤
extent: Extent(X = (200.0f0, 330.0f0), Y = (15.0f0, 75.0f0), Ti = (Dates.DateTime("2013-01-01T00:00:00"), Dates.DateTime("2014-12-31T18:00:00")))
missingval: missing
crs: EPSG:4326
mappedcrs: EPSG:4326
└──────────────────────────────────────────────────────────────────────────────┘
What’s in a DimStack?#
Many DimArrays!
DimStacks are dictionary-like containers of “DimArray”s. They are a mapping of variable name to DimArray. The RasterStack is a special case of a DimStack with some geospatial information. The DimArray s in the DimStack can share dimensions butdon’t have all the same dimensionality. If layers share a dimension name, this dimension is only stored once for the whole DimStack.
# pull out "air" dataarray with dictionary syntax
ds["air"]
╭──────────────────────────────────╮
│ 53×25×2920 Raster{Float64,3} air │
├──────────────────────────────────┴───────────────────────────────────── dims ┐
↓ X Mapped{Float32} [200.0, 202.5, …, 327.5, 330.0] ForwardOrdered Regular Points,
→ Y Mapped{Float32} [75.0, 72.5, …, 17.5, 15.0] ReverseOrdered Regular Points,
↗ Ti Sampled{Dates.DateTime} [2013-01-01T00:00:00, …, 2014-12-31T18:00:00] ForwardOrdered Irregular Points
├──────────────────────────────────────────────────────────────────── metadata ┤
Metadata{Rasters.NCDsource} of Dict{String, Any} with 12 entries:
"long_name" => "4xDaily Air temperature at sigma level 995"
"scale_factor" => 0.01
"GRIB_name" => "TMP"
"precision" => 2
"var_desc" => "Air temperature"
"actual_range" => Float32[185.16, 322.1]
"statistic" => "Individual Obs"
"GRIB_id" => 11
"dataset" => "NMC Reanalysis"
"units" => "degK"
"parent_stat" => "Other"
"level_desc" => "Surface"
├────────────────────────────────────────────────────────────────────── raster ┤
extent: Extent(X = (200.0f0, 330.0f0), Y = (15.0f0, 75.0f0), Ti = (Dates.DateTime("2013-01-01T00:00:00"), Dates.DateTime("2014-12-31T18:00:00")))
crs: EPSG:4326
mappedcrs: EPSG:4326
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
⋮ ⋱
You can save some typing by using the “attribute” or “dot” notation and using tab completion.
# pull out dataarray using dot notation
ds.air
╭──────────────────────────────────╮
│ 53×25×2920 Raster{Float64,3} air │
├──────────────────────────────────┴───────────────────────────────────── dims ┐
↓ X Mapped{Float32} [200.0, 202.5, …, 327.5, 330.0] ForwardOrdered Regular Points,
→ Y Mapped{Float32} [75.0, 72.5, …, 17.5, 15.0] ReverseOrdered Regular Points,
↗ Ti Sampled{Dates.DateTime} [2013-01-01T00:00:00, …, 2014-12-31T18:00:00] ForwardOrdered Irregular Points
├──────────────────────────────────────────────────────────────────── metadata ┤
Metadata{Rasters.NCDsource} of Dict{String, Any} with 12 entries:
"long_name" => "4xDaily Air temperature at sigma level 995"
"scale_factor" => 0.01
"GRIB_name" => "TMP"
"precision" => 2
"var_desc" => "Air temperature"
"actual_range" => Float32[185.16, 322.1]
"statistic" => "Individual Obs"
"GRIB_id" => 11
"dataset" => "NMC Reanalysis"
"units" => "degK"
"parent_stat" => "Other"
"level_desc" => "Surface"
├────────────────────────────────────────────────────────────────────── raster ┤
extent: Extent(X = (200.0f0, 330.0f0), Y = (15.0f0, 75.0f0), Ti = (Dates.DateTime("2013-01-01T00:00:00"), Dates.DateTime("2014-12-31T18:00:00")))
crs: EPSG:4326
mappedcrs: EPSG:4326
└──────────────────────────────────────────────────────────────────────────────┘
[:, :, 1]
⋮ ⋱
What’s in a DimArray?#
data + (a lot of) metadata
Name (optional)#
da = ds.air
Rasters.name(da)
:air
Named dimensions#
dims(da) correspond to the axes of your data.
In this case we have 2 spatial dimensions (X and Y) and one temporal dimension (Ti).
dims(da)
(↓ X Mapped{Float32} [200.0, 202.5, …, 327.5, 330.0] ForwardOrdered Regular Points,
→ Y Mapped{Float32} [75.0, 72.5, …, 17.5, 15.0] ReverseOrdered Regular Points,
↗ Ti Sampled{Dates.DateTime} [2013-01-01T00:00:00, …, 2014-12-31T18:00:00] ForwardOrdered Irregular Points)
You can also extract a single dimension.
# extracting coordinate variables
dims(da, X)
X Mapped{Float32} ForwardOrdered Regular DimensionalData.Dimensions.Lookups.Points
wrapping: 53-element Vector{Float32}:
200.0
202.5
205.0
207.5
210.0
212.5
215.0
217.5
220.0
222.5
225.0
227.5
230.0
⋮
302.5
305.0
307.5
310.0
312.5
315.0
317.5
320.0
322.5
325.0
327.5
330.0
It is useful to think of the values in the dimensions as axis “labels” such as “tick labels” in a figure. These are coordinate locations on a grid at which you have data.
Arbitrary attributes#
metadata is a metadata object that can hold arbitrary attributes that describe the underlying data.
metadata(da)
Metadata{Rasters.NCDsource} of Dict{String, Any} with 12 entries:
"long_name" => "4xDaily Air temperature at sigma level 995"
"scale_factor" => 0.01
"GRIB_name" => "TMP"
"precision" => 2
"var_desc" => "Air temperature"
"actual_range" => Float32[185.16, 322.1]
"statistic" => "Individual Obs"
"GRIB_id" => 11
"dataset" => "NMC Reanalysis"
"units" => "degK"
"parent_stat" => "Other"
"level_desc" => "Surface"
# assign your own attributes!
metadata(da)["myattrs"] = "Mine"
"Mine"
Underlying data#
A DimensionalData data structures wrap underlying simpler array-like data structures. These arrays have to fit into the Julia Array interface but can be either in memory arrays or DiskArray.jl arrays for lazy access or part of Xarray is quite extensible allowing for distributed array, GPU arrays, sparse arrays, arrays with units etc. We’ll briefly look at this later in this tutorial.
To access the underlying data use the parent function:
parent(da)
53×25×2920 Array{Float64, 3}:
[:, :, 1] =
241.2 243.8 250.0 266.5 274.5 … 297.0 296.6 295.9 296.29
242.5 244.5 249.8 267.1 274.29 296.7 296.2 296.2 296.79
243.5 244.7 248.89 267.1 274.1 296.1 296.4 296.79 297.1
244.0 244.2 247.5 266.7 274.0 295.9 296.5 296.5 297.0
244.1 243.39 246.0 265.9 273.79 294.9 295.5 296.1 296.9
243.89 242.39 244.39 264.7 273.0 … 293.4 294.0 295.1 296.4
243.6 241.7 243.1 263.1 271.79 292.79 293.6 294.6 296.0
243.1 241.3 242.39 261.6 270.6 292.4 293.4 294.4 295.7
242.5 241.1 242.2 260.6 270.0 292.2 292.79 293.79 295.5
241.89 241.1 242.5 260.1 269.9 292.2 292.9 293.79 295.4
241.2 241.0 243.1 260.0 270.0 … 291.5 292.6 293.79 295.2
240.3 240.5 243.5 259.9 269.9 291.1 292.2 293.29 295.2
239.5 239.7 243.5 259.29 269.2 290.9 292.2 292.79 295.0
⋮ ⋱
238.5 247.6 254.1 261.2 266.4 297.1 298.1 298.6 299.2
234.7 245.89 256.0 264.2 268.79 297.5 298.1 298.4 299.79
231.3 243.39 254.3 260.0 262.6 297.79 298.4 298.4 299.7
228.8 240.39 249.6 250.6 251.1 297.9 298.79 298.9 299.1
227.3 237.5 243.7 240.8 241.3 … 297.4 298.0 298.6 298.7
227.0 235.0 238.3 235.2 239.3 297.0 297.29 297.9 298.6
227.5 233.2 234.5 235.2 245.89 296.9 297.2 297.4 298.0
228.8 232.0 232.39 239.3 256.8 296.29 296.7 296.9 297.79
230.6 231.8 231.89 244.6 266.4 295.6 295.79 296.29 297.6
232.8 232.8 233.2 249.2 271.4 … 295.1 295.4 295.9 296.9
235.5 235.3 236.39 253.1 272.4 295.1 295.1 295.9 296.79
238.6 239.3 241.7 256.9 271.9 294.9 294.7 295.2 296.6
[:, :, 2] =
242.1 243.6 253.2 269.7 272.5 … 295.79 296.4 296.2 296.29
242.7 244.1 252.89 269.4 271.5 295.79 295.9 296.7 297.2
243.1 244.2 252.1 268.6 270.4 296.1 296.2 296.79 297.4
243.39 244.1 250.8 267.4 269.4 296.0 296.1 296.4 297.4
243.6 243.7 249.3 266.0 268.5 294.79 295.1 296.1 297.1
243.8 243.3 247.5 264.4 268.0 … 293.2 294.1 295.6 296.7
244.0 242.8 245.5 262.6 267.9 292.6 293.79 295.1 296.4
244.1 242.39 243.6 260.79 268.1 292.6 293.6 294.5 295.79
244.3 242.2 242.1 259.1 268.1 292.29 293.0 294.2 295.79
244.3 242.0 241.1 257.7 267.79 292.0 292.9 294.1 295.6
244.0 241.8 240.89 256.9 267.1 … 291.5 292.79 294.0 295.1
243.39 241.5 241.3 256.7 266.6 291.2 292.4 293.6 295.29
242.5 240.89 242.0 257.0 266.6 291.1 292.2 293.1 295.2
⋮ ⋱
238.1 247.6 254.8 262.1 266.2 296.79 297.7 298.4 298.79
234.7 245.6 256.0 264.4 268.4 297.2 297.9 298.2 299.29
231.7 242.89 253.6 259.79 262.5 297.6 298.4 298.29 299.29
229.6 239.89 248.6 250.6 252.0 297.6 298.5 298.5 298.6
228.5 237.3 242.8 241.5 243.1 … 297.5 298.0 298.29 298.2
228.39 235.1 237.7 236.3 241.3 297.29 297.4 297.7 298.1
228.89 233.3 234.0 236.1 247.2 297.0 297.0 297.2 297.5
229.7 231.89 231.6 239.39 257.1 296.5 296.7 297.1 297.1
230.7 231.0 230.39 243.5 266.0 295.9 296.1 296.4 296.9
232.0 231.0 230.8 247.2 270.9 … 295.2 295.4 295.6 296.4
233.6 232.5 233.39 250.5 272.29 294.9 295.1 295.5 296.4
235.8 235.7 238.5 254.39 272.29 294.7 294.79 295.1 296.6
[:, :, 3] =
242.3 244.6 256.2 269.29 271.9 … 295.0 295.6 296.2 296.4
242.2 244.39 255.5 268.5 271.79 295.0 295.4 296.5 296.29
242.3 244.0 254.2 267.0 271.6 294.9 295.4 296.29 296.4
242.5 243.39 252.3 265.0 270.7 295.0 295.2 295.5 296.29
242.89 242.8 250.2 262.7 269.4 294.0 294.1 295.1 296.29
243.6 242.39 248.1 260.29 267.79 … 292.4 293.1 294.5 295.79
244.5 242.3 246.3 258.5 266.6 292.1 293.1 294.1 295.2
245.7 242.3 244.89 257.29 266.2 292.2 292.9 293.5 295.1
246.8 242.39 243.7 256.8 266.2 292.0 292.29 293.0 295.0
247.7 242.5 242.6 256.39 265.9 291.4 292.2 293.2 294.5
248.2 242.3 241.7 255.8 265.1 … 290.6 291.7 293.2 294.29
248.0 241.8 240.8 255.2 264.0 290.7 291.4 292.79 294.7
247.2 241.1 240.3 254.89 263.4 290.9 291.79 292.7 294.9
⋮ ⋱
240.2 248.1 253.7 260.9 266.5 297.1 297.9 298.29 298.9
237.89 247.39 256.6 264.2 269.1 297.6 298.1 298.2 299.5
235.7 245.7 256.1 261.29 264.29 298.0 298.7 298.5 299.2
233.89 243.1 252.7 254.0 254.8 298.4 299.0 299.0 298.7
232.6 240.2 247.7 246.1 246.5 … 298.29 298.29 298.79 298.79
232.0 237.3 242.6 241.2 244.2 297.7 297.6 298.0 298.7
231.89 234.39 238.0 240.39 249.2 297.0 297.4 297.6 297.9
232.2 232.0 234.3 242.39 258.0 296.6 297.0 297.7 297.9
233.0 230.39 231.8 245.3 266.0 296.29 296.7 297.1 297.6
234.3 230.3 231.2 247.89 270.5 … 295.6 296.29 296.4 297.0
236.1 232.0 233.2 250.6 271.79 294.9 295.29 296.0 297.0
238.7 235.7 238.2 254.2 271.79 294.79 295.0 295.6 296.79
;;; …
[:, :, 2918] =
243.49 249.09 262.69 273.49 273.89 … 292.99 294.79 296.79 298.19
242.99 248.99 262.19 272.99 274.19 293.49 295.29 297.89 299.19
242.09 248.59 261.69 272.49 274.29 295.99 297.49 298.29 298.79
240.69 247.79 260.89 271.89 274.39 297.39 298.49 298.19 298.69
239.19 246.49 259.79 271.09 274.49 297.09 297.99 298.59 299.29
237.69 244.99 258.09 270.19 274.69 … 296.59 297.49 298.49 298.99
236.59 243.59 255.99 269.09 274.99 295.69 297.29 298.19 298.59
235.99 242.49 253.69 267.89 275.19 294.79 296.79 297.79 298.19
235.99 241.89 251.29 266.29 274.99 294.59 296.29 297.69 298.29
236.39 241.69 249.19 264.49 274.09 294.49 296.19 297.79 298.29
237.09 241.79 247.39 262.29 272.39 … 293.99 295.69 297.39 297.49
237.79 241.89 245.99 260.19 270.49 293.99 295.39 296.89 297.69
238.39 241.89 245.09 258.39 268.79 294.19 295.29 295.99 297.49
⋮ ⋱
254.49 255.29 258.79 263.99 267.29 297.99 298.59 298.59 299.19
252.39 256.09 262.19 267.39 269.59 297.89 298.59 298.29 299.69
249.99 254.99 261.69 263.69 262.69 297.09 298.29 297.59 298.79
247.79 252.49 257.49 254.89 250.99 296.49 297.89 297.69 297.89
245.89 249.09 251.39 245.59 241.79 … 295.89 296.99 297.39 297.79
244.59 245.79 245.39 240.09 241.09 295.79 296.89 296.79 297.69
243.89 243.09 241.09 240.09 249.19 295.59 296.79 296.79 297.29
243.69 241.39 238.79 243.89 261.29 295.09 296.19 297.09 297.19
243.89 240.59 238.39 248.99 271.29 295.09 295.69 296.39 297.09
244.19 240.59 239.39 253.49 276.49 … 294.69 295.49 295.49 296.09
244.49 241.29 241.69 257.39 277.89 294.69 295.39 295.49 295.79
244.89 242.69 245.19 261.39 278.29 294.79 294.69 294.79 295.79
[:, :, 2919] =
245.79 249.89 262.39 273.29 273.59 … 291.59 293.69 296.29 297.79
244.79 249.29 261.79 272.89 273.99 291.69 293.89 297.19 298.39
243.49 248.49 261.29 272.49 274.19 293.59 295.39 297.59 298.49
241.89 247.29 260.59 272.09 274.49 295.89 297.19 297.89 298.59
240.29 245.69 259.39 271.59 274.69 296.89 297.79 298.29 298.79
238.89 243.99 257.69 270.99 274.99 … 296.79 297.49 298.09 298.69
237.79 242.29 255.49 270.29 275.29 296.09 297.49 297.79 298.49
237.19 240.89 252.99 269.29 275.59 294.99 297.29 297.79 297.99
237.09 239.89 250.29 267.89 275.79 294.69 296.49 297.89 298.29
237.29 239.29 247.59 265.89 275.59 294.69 295.99 297.69 298.19
237.59 239.09 244.99 262.99 274.69 … 294.49 295.99 297.49 297.39
237.89 238.99 242.59 259.49 272.99 294.39 295.79 296.99 297.69
238.09 238.99 240.59 255.69 270.49 294.39 295.49 295.89 297.59
⋮ ⋱
255.49 256.19 258.69 262.69 267.19 298.09 298.39 298.49 298.69
253.19 256.59 262.09 265.99 269.59 297.49 297.89 297.89 299.09
250.69 255.39 261.79 263.19 263.99 296.69 297.89 297.69 298.89
248.19 252.79 257.99 255.69 253.99 296.19 297.59 297.69 297.79
246.09 249.39 252.09 247.59 246.29 … 295.69 296.59 297.59 297.99
244.39 245.89 246.29 242.79 245.99 295.69 296.39 296.99 297.99
243.39 243.09 241.89 242.79 253.39 295.29 296.29 296.49 296.79
242.89 241.39 239.59 246.29 264.19 294.59 295.49 296.79 296.89
242.89 240.79 239.19 250.79 272.99 294.49 294.89 295.89 296.59
243.29 241.29 240.49 254.89 277.29 … 294.49 295.09 295.29 295.69
243.99 242.49 243.09 258.29 277.99 294.29 294.69 295.09 295.49
244.79 244.29 246.89 261.79 277.59 294.49 294.29 294.39 295.19
[:, :, 2920] =
245.09 249.89 262.99 272.19 271.19 … 292.89 293.79 296.09 297.69
244.29 249.29 262.19 272.09 271.89 292.09 293.69 296.89 298.09
243.29 248.39 261.39 271.99 272.59 292.99 295.09 297.19 298.09
242.19 246.99 259.99 271.59 273.09 295.09 296.69 297.49 298.49
241.09 245.09 257.79 270.69 273.09 296.09 297.29 298.19 298.89
240.09 243.09 254.99 269.29 272.89 … 296.59 297.29 298.09 298.69
239.39 241.09 251.99 267.69 272.99 296.49 297.39 297.79 298.39
238.99 239.59 249.09 266.19 273.39 295.79 297.49 297.79 298.09
238.89 238.59 246.59 264.79 273.99 295.59 297.19 297.69 298.09
239.09 238.29 244.49 263.19 274.19 295.39 296.49 297.59 297.99
239.39 238.39 242.59 260.89 273.69 … 294.89 296.09 297.39 297.49
239.49 238.59 240.89 257.89 272.29 294.89 296.19 296.79 297.49
239.49 238.89 239.29 254.19 270.09 294.59 296.09 296.39 297.49
⋮ ⋱
256.09 255.49 257.29 262.39 266.69 298.59 299.39 299.99 299.89
254.29 255.99 260.79 266.49 269.39 298.39 299.09 299.69 300.49
252.49 255.19 261.29 264.99 264.79 297.49 299.09 298.99 299.99
250.59 253.29 258.49 258.89 255.89 296.49 298.69 298.69 299.09
248.69 250.29 253.39 251.69 248.59 … 296.09 297.59 298.69 299.29
246.79 246.99 247.69 246.79 247.69 296.19 297.09 298.19 299.29
245.09 243.79 242.89 245.69 253.69 295.79 296.69 297.39 297.89
243.59 241.39 239.89 247.89 263.19 294.99 295.79 297.29 297.39
242.39 239.99 238.99 251.59 271.49 294.59 295.19 296.49 297.19
241.69 239.59 239.89 255.39 276.09 … 294.59 295.29 295.69 296.49
241.49 240.29 242.59 258.99 277.39 294.19 295.09 295.69 296.19
241.79 241.69 246.29 262.49 277.29 294.29 294.69 295.19 295.69
# what is the type of the underlying data
typeof(parent(da))
Array{Float64, 3}
We can change the underlying data type by using the lazy keyword for opening the data. This is especially helpful for very large data or data that is hosted online where we would not want to download the whole dataset before starting the analysis.
dsl = RasterStack(path, lazy=true)
╭────────────────────────╮
│ 53×25×2920 RasterStack │
├────────────────────────┴─────────────────────────────────────────────── dims ┐
↓ X Mapped{Float32} [200.0, 202.5, …, 327.5, 330.0] ForwardOrdered Regular Points,
→ Y Mapped{Float32} [75.0, 72.5, …, 17.5, 15.0] ReverseOrdered Regular Points,
↗ Ti Sampled{Dates.DateTime} [2013-01-01T00:00:00, …, 2014-12-31T18:00:00] ForwardOrdered Irregular Points
├────────────────────────────────────────────────────────────────────── layers ┤
:air eltype: Union{Missing, Float64} dims: X, Y, Ti size: 53×25×2920
├──────────────────────────────────────────────────────────────────── metadata ┤
Metadata{Rasters.NCDsource} of Dict{String, Any} with 5 entries:
"references" => "http://www.esrl.noaa.gov/psd/data/gridded/data.ncep.reanaly…
"platform" => "Model"
"title" => "4x daily NMC reanalysis (1948)"
"description" => "Data is from NMC initialized reanalysis\n(4x/day). These a…
"Conventions" => "COARDS"
├────────────────────────────────────────────────────────────────────── raster ┤
extent: Extent(X = (200.0f0, 330.0f0), Y = (15.0f0, 75.0f0), Ti = (Dates.DateTime("2013-01-01T00:00:00"), Dates.DateTime("2014-12-31T18:00:00")))
missingval: missing
crs: EPSG:4326
mappedcrs: EPSG:4326
filename:
└──────────────────────────────────────────────────────────────────────────────┘
dal = dsl.air
╭──────────────────────────────────────────────────╮
│ 53×25×2920 Raster{Union{Missing, Float64},3} air │
├──────────────────────────────────────────────────┴───────────────────── dims ┐
↓ X Mapped{Float32} [200.0, 202.5, …, 327.5, 330.0] ForwardOrdered Regular Points,
→ Y Mapped{Float32} [75.0, 72.5, …, 17.5, 15.0] ReverseOrdered Regular Points,
↗ Ti Sampled{Dates.DateTime} [2013-01-01T00:00:00, …, 2014-12-31T18:00:00] ForwardOrdered Irregular Points
├──────────────────────────────────────────────────────────────────── metadata ┤
Metadata{Rasters.NCDsource} of Dict{String, Any} with 12 entries:
"long_name" => "4xDaily Air temperature at sigma level 995"
"scale_factor" => 0.01
"GRIB_name" => "TMP"
"precision" => 2
"var_desc" => "Air temperature"
"actual_range" => Float32[185.16, 322.1]
"statistic" => "Individual Obs"
"GRIB_id" => 11
"dataset" => "NMC Reanalysis"
"units" => "degK"
"parent_stat" => "Other"
"level_desc" => "Surface"
├────────────────────────────────────────────────────────────────────── raster ┤
extent: Extent(X = (200.0f0, 330.0f0), Y = (15.0f0, 75.0f0), Ti = (Dates.DateTime("2013-01-01T00:00:00"), Dates.DateTime("2014-12-31T18:00:00")))
missingval: missing
crs: EPSG:4326
mappedcrs: EPSG:4326
filename:
└──────────────────────────────────────────────────────────────────────────────┘
typeof(parent(dal))
Rasters.FileArray{Rasters.NCDsource, Union{Missing, Float64}, 3, Symbol, Nothing, DiskArrays.GridChunks{3, Tuple{DiskArrays.RegularChunks, DiskArrays.RegularChunks, DiskArrays.RegularChunks}}, DiskArrays.Unchunked}
Review#
DimensionalData provides two main data structures:
DimArraysthat wrap underlying data containers and contain associated metadata in theDimStacksthat are dictionary-like containers of DataArrays
DimArrays contain underlying arrays and associated metadata:
Name
Dimension names
Lookup values
Metadata
Why named dimensions and labeled dimensions?#
Metadata provides context and provides code that is more legible. This reduces the likelihood of errors from typos and makes analysis more intuitive and fun!
Analysis without named dimensions:#
# plot the first timestep
lon = ds.air.dims[1].val.data # Vector
lat = ds.air.dims[2].val.data # Vector
temp = parent(da) # vector
53×25×2920 Array{Float64, 3}:
[:, :, 1] =
241.2 243.8 250.0 266.5 274.5 … 297.0 296.6 295.9 296.29
242.5 244.5 249.8 267.1 274.29 296.7 296.2 296.2 296.79
243.5 244.7 248.89 267.1 274.1 296.1 296.4 296.79 297.1
244.0 244.2 247.5 266.7 274.0 295.9 296.5 296.5 297.0
244.1 243.39 246.0 265.9 273.79 294.9 295.5 296.1 296.9
243.89 242.39 244.39 264.7 273.0 … 293.4 294.0 295.1 296.4
243.6 241.7 243.1 263.1 271.79 292.79 293.6 294.6 296.0
243.1 241.3 242.39 261.6 270.6 292.4 293.4 294.4 295.7
242.5 241.1 242.2 260.6 270.0 292.2 292.79 293.79 295.5
241.89 241.1 242.5 260.1 269.9 292.2 292.9 293.79 295.4
241.2 241.0 243.1 260.0 270.0 … 291.5 292.6 293.79 295.2
240.3 240.5 243.5 259.9 269.9 291.1 292.2 293.29 295.2
239.5 239.7 243.5 259.29 269.2 290.9 292.2 292.79 295.0
⋮ ⋱
238.5 247.6 254.1 261.2 266.4 297.1 298.1 298.6 299.2
234.7 245.89 256.0 264.2 268.79 297.5 298.1 298.4 299.79
231.3 243.39 254.3 260.0 262.6 297.79 298.4 298.4 299.7
228.8 240.39 249.6 250.6 251.1 297.9 298.79 298.9 299.1
227.3 237.5 243.7 240.8 241.3 … 297.4 298.0 298.6 298.7
227.0 235.0 238.3 235.2 239.3 297.0 297.29 297.9 298.6
227.5 233.2 234.5 235.2 245.89 296.9 297.2 297.4 298.0
228.8 232.0 232.39 239.3 256.8 296.29 296.7 296.9 297.79
230.6 231.8 231.89 244.6 266.4 295.6 295.79 296.29 297.6
232.8 232.8 233.2 249.2 271.4 … 295.1 295.4 295.9 296.9
235.5 235.3 236.39 253.1 272.4 295.1 295.1 295.9 296.79
238.6 239.3 241.7 256.9 271.9 294.9 294.7 295.2 296.6
[:, :, 2] =
242.1 243.6 253.2 269.7 272.5 … 295.79 296.4 296.2 296.29
242.7 244.1 252.89 269.4 271.5 295.79 295.9 296.7 297.2
243.1 244.2 252.1 268.6 270.4 296.1 296.2 296.79 297.4
243.39 244.1 250.8 267.4 269.4 296.0 296.1 296.4 297.4
243.6 243.7 249.3 266.0 268.5 294.79 295.1 296.1 297.1
243.8 243.3 247.5 264.4 268.0 … 293.2 294.1 295.6 296.7
244.0 242.8 245.5 262.6 267.9 292.6 293.79 295.1 296.4
244.1 242.39 243.6 260.79 268.1 292.6 293.6 294.5 295.79
244.3 242.2 242.1 259.1 268.1 292.29 293.0 294.2 295.79
244.3 242.0 241.1 257.7 267.79 292.0 292.9 294.1 295.6
244.0 241.8 240.89 256.9 267.1 … 291.5 292.79 294.0 295.1
243.39 241.5 241.3 256.7 266.6 291.2 292.4 293.6 295.29
242.5 240.89 242.0 257.0 266.6 291.1 292.2 293.1 295.2
⋮ ⋱
238.1 247.6 254.8 262.1 266.2 296.79 297.7 298.4 298.79
234.7 245.6 256.0 264.4 268.4 297.2 297.9 298.2 299.29
231.7 242.89 253.6 259.79 262.5 297.6 298.4 298.29 299.29
229.6 239.89 248.6 250.6 252.0 297.6 298.5 298.5 298.6
228.5 237.3 242.8 241.5 243.1 … 297.5 298.0 298.29 298.2
228.39 235.1 237.7 236.3 241.3 297.29 297.4 297.7 298.1
228.89 233.3 234.0 236.1 247.2 297.0 297.0 297.2 297.5
229.7 231.89 231.6 239.39 257.1 296.5 296.7 297.1 297.1
230.7 231.0 230.39 243.5 266.0 295.9 296.1 296.4 296.9
232.0 231.0 230.8 247.2 270.9 … 295.2 295.4 295.6 296.4
233.6 232.5 233.39 250.5 272.29 294.9 295.1 295.5 296.4
235.8 235.7 238.5 254.39 272.29 294.7 294.79 295.1 296.6
[:, :, 3] =
242.3 244.6 256.2 269.29 271.9 … 295.0 295.6 296.2 296.4
242.2 244.39 255.5 268.5 271.79 295.0 295.4 296.5 296.29
242.3 244.0 254.2 267.0 271.6 294.9 295.4 296.29 296.4
242.5 243.39 252.3 265.0 270.7 295.0 295.2 295.5 296.29
242.89 242.8 250.2 262.7 269.4 294.0 294.1 295.1 296.29
243.6 242.39 248.1 260.29 267.79 … 292.4 293.1 294.5 295.79
244.5 242.3 246.3 258.5 266.6 292.1 293.1 294.1 295.2
245.7 242.3 244.89 257.29 266.2 292.2 292.9 293.5 295.1
246.8 242.39 243.7 256.8 266.2 292.0 292.29 293.0 295.0
247.7 242.5 242.6 256.39 265.9 291.4 292.2 293.2 294.5
248.2 242.3 241.7 255.8 265.1 … 290.6 291.7 293.2 294.29
248.0 241.8 240.8 255.2 264.0 290.7 291.4 292.79 294.7
247.2 241.1 240.3 254.89 263.4 290.9 291.79 292.7 294.9
⋮ ⋱
240.2 248.1 253.7 260.9 266.5 297.1 297.9 298.29 298.9
237.89 247.39 256.6 264.2 269.1 297.6 298.1 298.2 299.5
235.7 245.7 256.1 261.29 264.29 298.0 298.7 298.5 299.2
233.89 243.1 252.7 254.0 254.8 298.4 299.0 299.0 298.7
232.6 240.2 247.7 246.1 246.5 … 298.29 298.29 298.79 298.79
232.0 237.3 242.6 241.2 244.2 297.7 297.6 298.0 298.7
231.89 234.39 238.0 240.39 249.2 297.0 297.4 297.6 297.9
232.2 232.0 234.3 242.39 258.0 296.6 297.0 297.7 297.9
233.0 230.39 231.8 245.3 266.0 296.29 296.7 297.1 297.6
234.3 230.3 231.2 247.89 270.5 … 295.6 296.29 296.4 297.0
236.1 232.0 233.2 250.6 271.79 294.9 295.29 296.0 297.0
238.7 235.7 238.2 254.2 271.79 294.79 295.0 295.6 296.79
;;; …
[:, :, 2918] =
243.49 249.09 262.69 273.49 273.89 … 292.99 294.79 296.79 298.19
242.99 248.99 262.19 272.99 274.19 293.49 295.29 297.89 299.19
242.09 248.59 261.69 272.49 274.29 295.99 297.49 298.29 298.79
240.69 247.79 260.89 271.89 274.39 297.39 298.49 298.19 298.69
239.19 246.49 259.79 271.09 274.49 297.09 297.99 298.59 299.29
237.69 244.99 258.09 270.19 274.69 … 296.59 297.49 298.49 298.99
236.59 243.59 255.99 269.09 274.99 295.69 297.29 298.19 298.59
235.99 242.49 253.69 267.89 275.19 294.79 296.79 297.79 298.19
235.99 241.89 251.29 266.29 274.99 294.59 296.29 297.69 298.29
236.39 241.69 249.19 264.49 274.09 294.49 296.19 297.79 298.29
237.09 241.79 247.39 262.29 272.39 … 293.99 295.69 297.39 297.49
237.79 241.89 245.99 260.19 270.49 293.99 295.39 296.89 297.69
238.39 241.89 245.09 258.39 268.79 294.19 295.29 295.99 297.49
⋮ ⋱
254.49 255.29 258.79 263.99 267.29 297.99 298.59 298.59 299.19
252.39 256.09 262.19 267.39 269.59 297.89 298.59 298.29 299.69
249.99 254.99 261.69 263.69 262.69 297.09 298.29 297.59 298.79
247.79 252.49 257.49 254.89 250.99 296.49 297.89 297.69 297.89
245.89 249.09 251.39 245.59 241.79 … 295.89 296.99 297.39 297.79
244.59 245.79 245.39 240.09 241.09 295.79 296.89 296.79 297.69
243.89 243.09 241.09 240.09 249.19 295.59 296.79 296.79 297.29
243.69 241.39 238.79 243.89 261.29 295.09 296.19 297.09 297.19
243.89 240.59 238.39 248.99 271.29 295.09 295.69 296.39 297.09
244.19 240.59 239.39 253.49 276.49 … 294.69 295.49 295.49 296.09
244.49 241.29 241.69 257.39 277.89 294.69 295.39 295.49 295.79
244.89 242.69 245.19 261.39 278.29 294.79 294.69 294.79 295.79
[:, :, 2919] =
245.79 249.89 262.39 273.29 273.59 … 291.59 293.69 296.29 297.79
244.79 249.29 261.79 272.89 273.99 291.69 293.89 297.19 298.39
243.49 248.49 261.29 272.49 274.19 293.59 295.39 297.59 298.49
241.89 247.29 260.59 272.09 274.49 295.89 297.19 297.89 298.59
240.29 245.69 259.39 271.59 274.69 296.89 297.79 298.29 298.79
238.89 243.99 257.69 270.99 274.99 … 296.79 297.49 298.09 298.69
237.79 242.29 255.49 270.29 275.29 296.09 297.49 297.79 298.49
237.19 240.89 252.99 269.29 275.59 294.99 297.29 297.79 297.99
237.09 239.89 250.29 267.89 275.79 294.69 296.49 297.89 298.29
237.29 239.29 247.59 265.89 275.59 294.69 295.99 297.69 298.19
237.59 239.09 244.99 262.99 274.69 … 294.49 295.99 297.49 297.39
237.89 238.99 242.59 259.49 272.99 294.39 295.79 296.99 297.69
238.09 238.99 240.59 255.69 270.49 294.39 295.49 295.89 297.59
⋮ ⋱
255.49 256.19 258.69 262.69 267.19 298.09 298.39 298.49 298.69
253.19 256.59 262.09 265.99 269.59 297.49 297.89 297.89 299.09
250.69 255.39 261.79 263.19 263.99 296.69 297.89 297.69 298.89
248.19 252.79 257.99 255.69 253.99 296.19 297.59 297.69 297.79
246.09 249.39 252.09 247.59 246.29 … 295.69 296.59 297.59 297.99
244.39 245.89 246.29 242.79 245.99 295.69 296.39 296.99 297.99
243.39 243.09 241.89 242.79 253.39 295.29 296.29 296.49 296.79
242.89 241.39 239.59 246.29 264.19 294.59 295.49 296.79 296.89
242.89 240.79 239.19 250.79 272.99 294.49 294.89 295.89 296.59
243.29 241.29 240.49 254.89 277.29 … 294.49 295.09 295.29 295.69
243.99 242.49 243.09 258.29 277.99 294.29 294.69 295.09 295.49
244.79 244.29 246.89 261.79 277.59 294.49 294.29 294.39 295.19
[:, :, 2920] =
245.09 249.89 262.99 272.19 271.19 … 292.89 293.79 296.09 297.69
244.29 249.29 262.19 272.09 271.89 292.09 293.69 296.89 298.09
243.29 248.39 261.39 271.99 272.59 292.99 295.09 297.19 298.09
242.19 246.99 259.99 271.59 273.09 295.09 296.69 297.49 298.49
241.09 245.09 257.79 270.69 273.09 296.09 297.29 298.19 298.89
240.09 243.09 254.99 269.29 272.89 … 296.59 297.29 298.09 298.69
239.39 241.09 251.99 267.69 272.99 296.49 297.39 297.79 298.39
238.99 239.59 249.09 266.19 273.39 295.79 297.49 297.79 298.09
238.89 238.59 246.59 264.79 273.99 295.59 297.19 297.69 298.09
239.09 238.29 244.49 263.19 274.19 295.39 296.49 297.59 297.99
239.39 238.39 242.59 260.89 273.69 … 294.89 296.09 297.39 297.49
239.49 238.59 240.89 257.89 272.29 294.89 296.19 296.79 297.49
239.49 238.89 239.29 254.19 270.09 294.59 296.09 296.39 297.49
⋮ ⋱
256.09 255.49 257.29 262.39 266.69 298.59 299.39 299.99 299.89
254.29 255.99 260.79 266.49 269.39 298.39 299.09 299.69 300.49
252.49 255.19 261.29 264.99 264.79 297.49 299.09 298.99 299.99
250.59 253.29 258.49 258.89 255.89 296.49 298.69 298.69 299.09
248.69 250.29 253.39 251.69 248.59 … 296.09 297.59 298.69 299.29
246.79 246.99 247.69 246.79 247.69 296.19 297.09 298.19 299.29
245.09 243.79 242.89 245.69 253.69 295.79 296.69 297.39 297.89
243.59 241.39 239.89 247.89 263.19 294.99 295.79 297.29 297.39
242.39 239.99 238.99 251.59 271.49 294.59 295.19 296.49 297.19
241.69 239.59 239.89 255.39 276.09 … 294.59 295.29 295.69 296.49
241.49 240.29 242.59 258.99 277.39 294.19 295.09 295.69 296.19
241.79 241.69 246.29 262.49 277.29 294.29 294.69 295.19 295.69
using WGLMakie
heatmap(lon, lat, temp[1, :, :])
using Statistics
mean(temp, dims=3)# On what dimensions did we apply the reduction? I can't tell by looking at this line.
53×25×1 Array{Float64, 3}:
[:, :, 1] =
260.376 262.734 264.769 267.683 … 297.326 297.65 298.129 298.366
260.183 262.794 264.327 267.287 296.869 296.953 297.937 298.386
259.887 262.749 264.062 267.136 296.537 296.629 297.47 298.114
259.483 262.538 263.88 267.073 296.626 296.862 297.259 298.053
259.017 262.159 263.695 266.948 296.387 296.893 297.455 297.943
258.572 261.691 263.494 266.764 … 295.934 296.305 297.113 297.607
258.245 261.246 263.324 266.644 295.617 296.062 296.9 297.539
258.101 260.933 263.247 266.7 295.086 295.847 296.685 297.432
258.155 260.795 263.263 266.897 294.632 295.358 296.232 297.394
258.362 260.802 263.307 267.06 294.26 295.113 296.205 297.233
258.625 260.867 263.295 267.037 … 293.616 294.714 296.16 297.108
258.839 260.896 263.173 266.841 293.276 294.478 295.847 297.375
258.919 260.829 262.956 266.617 292.893 294.489 295.684 297.335
⋮ ⋱
260.526 264.603 266.153 268.031 298.577 299.104 299.383 299.594
258.087 263.368 267.242 270.214 298.446 298.789 298.897 299.656
255.691 261.447 266.561 269.183 298.091 298.659 298.679 299.268
253.582 258.987 263.948 265.031 297.81 298.463 298.744 298.672
251.907 256.238 259.97 259.666 … 297.692 297.918 298.563 298.651
250.731 253.527 255.641 255.585 297.603 297.736 298.097 298.693
250.056 251.213 251.99 254.337 297.24 297.695 297.752 297.994
249.86 249.644 249.745 255.839 296.826 297.258 297.796 297.891
250.118 249.103 249.263 258.904 296.666 296.857 297.307 297.818
250.816 249.756 250.608 262.341 … 296.421 296.811 296.86 297.338
251.938 251.586 253.584 265.62 295.877 296.288 296.777 297.281
253.438 254.359 257.716 268.701 295.71 295.816 296.444 297.305
Analysis with DimensionalData#
How readable is this code?
plot(ds.air[Ti=1])