xsar: efficient level 1 sar reader for xarray/dask¶
xsar is a distributed level 1 SAR file reader designed to write efficient distributed processing algorhitm with xarray and dask.
It currently handles Level-1 Sentinel-1 data in SAFE format, as found on scihub or PEPS.
xsar is as simple to use as the well known xarray.open_dataset : simply give the dataset path, and xsar.open_dataset()
will return an xarray.Dataset:
import xsar
import os
filename = xsar.get_test_file('S1A_IW_GRDH_1SDV_20170907T103020_20170907T103045_018268_01EB76_Z010.SAFE')
sar_ds = xsar.open_dataset(filename)
sar_ds
<xarray.Dataset> Dimensions: (atrack: 16778, pol: 2, xtrack: 25187) Coordinates: * pol (pol) object 'VV' 'VH' * atrack (atrack) float64 0.5 1.5 2.5 ... 1.678e+04 1.678e+04 * xtrack (xtrack) float64 0.5 1.5 2.5 ... 2.519e+04 2.519e+04 Data variables: digital_number (pol, atrack, xtrack) uint16 dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray> time (atrack) datetime64[ns] 2017-09-07T10:30:20.936409088 ...... longitude (atrack, xtrack) float32 dask.array<chunksize=(5000, 5000), meta=np.ndarray> latitude (atrack, xtrack) float32 dask.array<chunksize=(5000, 5000), meta=np.ndarray> elevation (atrack, xtrack) float32 dask.array<chunksize=(5000, 5000), meta=np.ndarray> incidence (atrack, xtrack) float32 dask.array<chunksize=(5000, 5000), meta=np.ndarray> sigma0_raw (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray> nesz (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray> gamma0_raw (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray> negz (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray> sigma0 (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray> gamma0 (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray> Attributes: (12/15) ipf: 2.84 platform: SENTINEL-1A swath: IW product: GRDH pols: VV VH name: SENTINEL1_DS:/tmp/S1A_IW_GRDH_1SDV_20170907T103020_201... ... ... coverage: 170km * 251km (atrack * xtrack ) pixel_atrack_m: 10.152619433217325 pixel_xtrack_m: 9.986179379582328 orbit_pass: Descending platform_heading: -167.7668824808032 Conventions: CF-1.7
- atrack: 16778
- pol: 2
- xtrack: 25187
- pol(pol)object'VV' 'VH'
array(['VV', 'VH'], dtype=object)
- atrack(atrack)float640.5 1.5 2.5 ... 1.678e+04 1.678e+04
- units :
- 1
array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 1.67755e+04, 1.67765e+04, 1.67775e+04])
- xtrack(xtrack)float640.5 1.5 2.5 ... 2.519e+04 2.519e+04
- units :
- 1
array([5.00000e-01, 1.50000e+00, 2.50000e+00, ..., 2.51845e+04, 2.51855e+04, 2.51865e+04])
- digital_number(pol, atrack, xtrack)uint16dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray>
Array Chunk Bytes 1.57 GiB 47.68 MiB Shape (2, 16778, 25187) (1, 5000, 5000) Count 98 Tasks 48 Chunks Type uint16 numpy.ndarray - time(atrack)datetime64[ns]2017-09-07T10:30:20.936409088 .....
array(['2017-09-07T10:30:20.936409088', '2017-09-07T10:30:20.937899264', '2017-09-07T10:30:20.939389184', ..., '2017-09-07T10:30:45.932283904', '2017-09-07T10:30:45.933773824', '2017-09-07T10:30:45.935264000'], dtype='datetime64[ns]')
- longitude(atrack, xtrack)float32dask.array<chunksize=(5000, 5000), meta=np.ndarray>
- standard_name :
- longitude
- units :
- degrees_east
Array Chunk Bytes 1.57 GiB 95.37 MiB Shape (16778, 25187) (5000, 5000) Count 144 Tasks 24 Chunks Type float32 numpy.ndarray - latitude(atrack, xtrack)float32dask.array<chunksize=(5000, 5000), meta=np.ndarray>
- standard_name :
- latitude
- units :
- degrees_north
Array Chunk Bytes 1.57 GiB 95.37 MiB Shape (16778, 25187) (5000, 5000) Count 144 Tasks 24 Chunks Type float32 numpy.ndarray - elevation(atrack, xtrack)float32dask.array<chunksize=(5000, 5000), meta=np.ndarray>
Array Chunk Bytes 1.57 GiB 95.37 MiB Shape (16778, 25187) (5000, 5000) Count 97 Tasks 24 Chunks Type float32 numpy.ndarray - incidence(atrack, xtrack)float32dask.array<chunksize=(5000, 5000), meta=np.ndarray>
Array Chunk Bytes 1.57 GiB 95.37 MiB Shape (16778, 25187) (5000, 5000) Count 97 Tasks 24 Chunks Type float32 numpy.ndarray - sigma0_raw(pol, atrack, xtrack)float64dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray>
- units :
- m2/m2
Array Chunk Bytes 6.30 GiB 190.73 MiB Shape (2, 16778, 25187) (1, 5000, 5000) Count 628 Tasks 48 Chunks Type float64 numpy.ndarray - nesz(pol, atrack, xtrack)float64dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray>
Array Chunk Bytes 6.30 GiB 190.73 MiB Shape (2, 16778, 25187) (1, 5000, 5000) Count 798 Tasks 48 Chunks Type float64 numpy.ndarray - gamma0_raw(pol, atrack, xtrack)float64dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray>
- units :
- m2/m2
Array Chunk Bytes 6.30 GiB 190.73 MiB Shape (2, 16778, 25187) (1, 5000, 5000) Count 628 Tasks 48 Chunks Type float64 numpy.ndarray - negz(pol, atrack, xtrack)float64dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray>
Array Chunk Bytes 6.30 GiB 190.73 MiB Shape (2, 16778, 25187) (1, 5000, 5000) Count 798 Tasks 48 Chunks Type float64 numpy.ndarray - sigma0(pol, atrack, xtrack)float64dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray>
Array Chunk Bytes 6.30 GiB 190.73 MiB Shape (2, 16778, 25187) (1, 5000, 5000) Count 1232 Tasks 48 Chunks Type float64 numpy.ndarray - gamma0(pol, atrack, xtrack)float64dask.array<chunksize=(1, 5000, 5000), meta=np.ndarray>
Array Chunk Bytes 6.30 GiB 190.73 MiB Shape (2, 16778, 25187) (1, 5000, 5000) Count 1232 Tasks 48 Chunks Type float64 numpy.ndarray
- ipf :
- 2.84
- platform :
- SENTINEL-1A
- swath :
- IW
- product :
- GRDH
- pols :
- VV VH
- name :
- SENTINEL1_DS:/tmp/S1A_IW_GRDH_1SDV_20170907T103020_20170907T103045_018268_01EB76_Z010.SAFE:IW
- start_date :
- 2017-09-07 10:30:20.936409
- stop_date :
- 2017-09-07 10:30:45.935264
- footprint :
- POLYGON ((-67.84221143971432 20.72564283093837, -70.22162571215458 21.14758369556577, -70.51434171237564 19.64044158236074, -68.15836401927538 19.21519321274318, -67.84221143971432 20.72564283093837))
- coverage :
- 170km * 251km (atrack * xtrack )
- pixel_atrack_m :
- 10.152619433217325
- pixel_xtrack_m :
- 9.986179379582328
- orbit_pass :
- Descending
- platform_heading :
- -167.7668824808032
- Conventions :
- CF-1.7
Documentation¶
Getting Started¶
Installation¶
Quick install:
conda create -n xsar conda activate xsar # FIXME 'docutils>=0.16' with nbsphinx give "WARNING: Line 710 exceeds the line-length-limit." when converting ipynb to html conda install -c conda-forge rasterio gdal dask[array] dask[distributed] jupyterlab geoviews cartopy holoviews datashader nbsphinx pandoc jq 'docutils<0.16' pip fiona git lxml pip install git+https://github.com/oarcher/xsar.git
Overview¶
xsar rely on xarray.open_rasterio, rasterio and GDAL to read digital_number from SAFE product to return an xarray.Dataset object with dask chunks.
Luts are decoded from xml files and applied to digital_number, following official ESA thermal denoising document and ESA Sentinel-1 Product Specification.
So end user can directly use for example sigma0 variable, thas is the denoised sigma0 computed from digital_number and by applying relevants luts.
Because xsar rely on dask, it have a small memory footprint: variables are read from file and computed only if needed.
xsar.open_dataset()
is very close to xarray.open_dataset, but in the followings examples, you will find some aditionnal keywords and classes that allow to:
Examples¶
Note
With recommended installation you will be able to download and execute those examples in jupyter notebook.
Those examples will automatically download test data from https://cyclobs.ifremer.fr/static/sarwing_datarmor/xsardata/
Those file are not official ones: they are resampled to a lower resoltion and compressed to avoid big network transfert and disk usage.
Don’t use them for real science !