Download this notebook from github.

Open multiples datasets¶

Some safe files have multiples subdatasets, like WV or IW_SLC. This notebook show how to handle them, or how to handle multiple datasets.

[1]:

import xsar
import glob
import geoviews as gv
gv.extension('bokeh')

SAFES containings multiples subdatasets like WV_SLC cannot be openned with xsar.open_dataset given the safe path.

[2]:

try:
    wv_slc_ds = xsar.open_dataset(xsar.get_test_file('S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE'))
except IndexError as e:
    print(str(e))

Can't open an multi-dataset. Use `xsar.SentinelMeta('/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE').subdatasets` to show availables ones

An IndexError is raised, with an advice to use xsar.SentinelMeta to get the available subdatasets list.

[3]:

wv_slc_meta = xsar.SentinelMeta(xsar.get_test_file('S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE'))
wv_slc_meta

[3]:

Multi (24) dataset

SENTINEL1_DS:S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:

ipf	2.7
platform	SENTINEL-1A
swath	WV
product	SLC
pols	VV
start_date	2016-05-10 10:16:03.843410
stop_date	2016-05-10 10:21:43.669704
subdatasets	list of 24 subdatasets

Note that the abose wv_slc_meta object is a multi dataset

Subdatasets list is available with xsar.SentinelMeta.subdatasets property

[4]:

wv_slc_meta.subdatasets

[4]:

['SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_001',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_002',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_003',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_004',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_005',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_006',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_007',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_008',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_009',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_010',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_011',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_012',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_013',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_014',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_015',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_016',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_017',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_018',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_019',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_020',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_021',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_022',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_023',
 'SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_024']

A subdataset can be directly be opened by xsar.open_dataset

[5]:

wv_slc_ds1 = xsar.open_dataset(wv_slc_meta.subdatasets[0])
wv_slc_ds1

[5]:

<xarray.Dataset>
Dimensions:         (atrack: 4923, pol: 1, xtrack: 5681)
Coordinates:
  * pol             (pol) object 'VV'
  * atrack          (atrack) float64 0.5 1.5 2.5 ... 4.922e+03 4.922e+03
  * xtrack          (xtrack) float64 0.5 1.5 2.5 ... 5.678e+03 5.68e+03 5.68e+03
Data variables:
    digital_number  (pol, atrack, xtrack) complex128 dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>
    time            (atrack) datetime64[ns] 2016-05-10T10:16:03.843409920 ......
    longitude       (atrack, xtrack) float32 dask.array<chunksize=(4923, 5000), meta=np.ndarray>
    latitude        (atrack, xtrack) float32 dask.array<chunksize=(4923, 5000), meta=np.ndarray>
    elevation       (atrack, xtrack) float32 dask.array<chunksize=(4923, 5000), meta=np.ndarray>
    incidence       (atrack, xtrack) float32 dask.array<chunksize=(4923, 5000), meta=np.ndarray>
    sigma0_raw      (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>
    nesz            (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>
    gamma0_raw      (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>
    negz            (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>
    sigma0          (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>
    gamma0          (pol, atrack, xtrack) float64 dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>
Attributes: (12/15)
    ipf:               2.7
    platform:          SENTINEL-1A
    swath:             WV
    product:           SLC
    pols:              VV
    name:              SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_201...
    ...                ...
    coverage:          20km * 20km (atrack * xtrack )
    pixel_atrack_m:    4.127512286013487
    pixel_xtrack_m:    3.6571523143890947
    orbit_pass:        Descending
    platform_heading:  -166.3012322109123
    Conventions:       CF-1.7

xarray.Dataset

Dimensions:
- atrack: 4923
- pol: 1
- xtrack: 5681

Coordinates: (3)

pol
(pol)
object
'VV'
```
array(['VV'], dtype=object)
```

atrack

(atrack)

float64

0.5 1.5 2.5 ... 4.922e+03 4.922e+03

units :: 1

array([5.0000e-01, 1.5000e+00, 2.5000e+00, ..., 4.9205e+03, 4.9215e+03,
       4.9225e+03])

xtrack

(xtrack)

float64

0.5 1.5 2.5 ... 5.68e+03 5.68e+03

units :: 1

array([5.0000e-01, 1.5000e+00, 2.5000e+00, ..., 5.6785e+03, 5.6795e+03,
       5.6805e+03])

Data variables: (12)

digital_number

(pol, atrack, xtrack)

complex128

dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>

	Array	Chunk
Bytes	426.75 MiB	375.60 MiB
Shape	(1, 4923, 5681)	(1, 4923, 5000)
Count	3 Tasks	2 Chunks
Type	complex128	numpy.ndarray

time

(atrack)

datetime64[ns]

2016-05-10T10:16:03.843409920 .....

array(['2016-05-10T10:16:03.843409920', '2016-05-10T10:16:03.844015872',
       '2016-05-10T10:16:03.844621824', ..., '2016-05-10T10:16:06.824925184',
       '2016-05-10T10:16:06.825531136', '2016-05-10T10:16:06.826137088'],
      dtype='datetime64[ns]')

longitude

(atrack, xtrack)

float32

dask.array<chunksize=(4923, 5000), meta=np.ndarray>

standard_name :: longitude
units :: degrees_east

	Array	Chunk
Bytes	106.69 MiB	93.90 MiB
Shape	(4923, 5681)	(4923, 5000)
Count	12 Tasks	2 Chunks
Type	float32	numpy.ndarray

latitude

(atrack, xtrack)

float32

dask.array<chunksize=(4923, 5000), meta=np.ndarray>

standard_name :: latitude
units :: degrees_north

	Array	Chunk
Bytes	106.69 MiB	93.90 MiB
Shape	(4923, 5681)	(4923, 5000)
Count	12 Tasks	2 Chunks
Type	float32	numpy.ndarray

elevation

(atrack, xtrack)

float32

dask.array<chunksize=(4923, 5000), meta=np.ndarray>

	Array	Chunk
Bytes	106.69 MiB	93.90 MiB
Shape	(4923, 5681)	(4923, 5000)
Count	9 Tasks	2 Chunks
Type	float32	numpy.ndarray

incidence

(atrack, xtrack)

float32

dask.array<chunksize=(4923, 5000), meta=np.ndarray>

	Array	Chunk
Bytes	106.69 MiB	93.90 MiB
Shape	(4923, 5681)	(4923, 5000)
Count	9 Tasks	2 Chunks
Type	float32	numpy.ndarray

sigma0_raw

(pol, atrack, xtrack)

float64

dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>

units :: m2/m2

	Array	Chunk
Bytes	213.38 MiB	187.80 MiB
Shape	(1, 4923, 5681)	(1, 4923, 5000)
Count	26 Tasks	2 Chunks
Type	float64	numpy.ndarray

nesz

(pol, atrack, xtrack)

float64

dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>

	Array	Chunk
Bytes	213.38 MiB	187.80 MiB
Shape	(1, 4923, 5681)	(1, 4923, 5000)
Count	33 Tasks	2 Chunks
Type	float64	numpy.ndarray

gamma0_raw

(pol, atrack, xtrack)

float64

dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>

units :: m2/m2

	Array	Chunk
Bytes	213.38 MiB	187.80 MiB
Shape	(1, 4923, 5681)	(1, 4923, 5000)
Count	26 Tasks	2 Chunks
Type	float64	numpy.ndarray

negz

(pol, atrack, xtrack)

float64

dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>

	Array	Chunk
Bytes	213.38 MiB	187.80 MiB
Shape	(1, 4923, 5681)	(1, 4923, 5000)
Count	33 Tasks	2 Chunks
Type	float64	numpy.ndarray

sigma0

(pol, atrack, xtrack)

float64

dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>

	Array	Chunk
Bytes	213.38 MiB	187.80 MiB
Shape	(1, 4923, 5681)	(1, 4923, 5000)
Count	50 Tasks	2 Chunks
Type	float64	numpy.ndarray

gamma0

(pol, atrack, xtrack)

float64

dask.array<chunksize=(1, 4923, 5000), meta=np.ndarray>

	Array	Chunk
Bytes	213.38 MiB	187.80 MiB
Shape	(1, 4923, 5681)	(1, 4923, 5000)
Count	50 Tasks	2 Chunks
Type	float64	numpy.ndarray

Attributes: (15)
ipf :
2.7
platform :
SENTINEL-1A
swath :
WV
product :
SLC
pols :
VV
name :
SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_001
start_date :
2016-05-10 10:16:03.843410
stop_date :
2016-05-10 10:16:06.826137
footprint :
POLYGON ((-59.80263505017974 41.73435353976264, -60.04794367802916 41.77204176347002, -60.09752876528052 41.59309352477252, -59.85296208623707 41.55544353659125, -59.80263505017974 41.73435353976264))
coverage :
20km * 20km (atrack * xtrack )
pixel_atrack_m :
4.127512286013487
pixel_xtrack_m :
3.6571523143890947
orbit_pass :
Descending
platform_heading :
-166.3012322109123
Conventions :
CF-1.7

Or we can get a single dataset by using xsar.SentinelMeta

[6]:

wv_slc_meta1 = xsar.SentinelMeta(wv_slc_meta.subdatasets[0])
wv_slc_meta1

[6]:

Single dataset

SENTINEL1_DS:S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_001

ipf	2.7
platform	SENTINEL-1A
swath	WV
product	SLC
pols	VV
orbit_pass	Descending
pixel size	3.7 * 4.1 meters (xtrack * atrack)
coverage	20km * 20km (atrack * xtrack )
start_date	2016-05-10 10:16:03.843410
stop_date	2016-05-10 10:16:06.826137
dsid	WV_001

And use this wv_slc_meta1 object with xsar.open_dataset

[7]:

wv_slc_ds1 = xsar.open_dataset(wv_slc_meta1)

Getting all xsar.SentinelMeta objects from a list of path¶

xsar.SentinelMeta can only handle one path at a time.

With xsar.product_info, you can provide a list of path, and get a pandas.DataFrame with all meta objects.

[8]:

df_info = xsar.product_info([
    xsar.get_test_file('S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE'),
    xsar.get_test_file('S1A_IW_SLC__1SDV_20170907T102951_20170907T103021_018268_01EB76_Z010.SAFE')
])
df_info

[8]:

		name	ipf	platform	swath	product	pols	meta
path	dsid
/tmp/S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE	WV_001	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_002	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_003	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_004	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_005	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_006	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_007	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_008	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_009	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_010	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_011	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_012	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_013	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_014	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_015	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_016	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_017	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_018	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_019	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_020	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_021	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_022	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_023	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
	WV_024	SENTINEL1_DS:/tmp/S1A_WV_SLC__1SSV_20160510T10...	2.7	SENTINEL-1A	WV	SLC	VV	single SentinelMeta object
/tmp/S1A_IW_SLC__1SDV_20170907T102951_20170907T103021_018268_01EB76_Z010.SAFE	IW1	SENTINEL1_DS:/tmp/S1A_IW_SLC__1SDV_20170907T10...	2.84	SENTINEL-1A	IW	SLC	VV VH	single SentinelMeta object
	IW2	SENTINEL1_DS:/tmp/S1A_IW_SLC__1SDV_20170907T10...	2.84	SENTINEL-1A	IW	SLC	VV VH	single SentinelMeta object
	IW3	SENTINEL1_DS:/tmp/S1A_IW_SLC__1SDV_20170907T10...	2.84	SENTINEL-1A	IW	SLC	VV VH	single SentinelMeta object

By default, xsar.product_info only return a minimal set of columns.

Columns list can be given by the columns= keyword.

Columns name can be any attributes from xsar.SentinelMeta, with the addition of the ‘meta’ column name, who refer to the xsar.SentinelMeta objet itself.

If the columns list include ‘geometry’, the result will by a geopandas.GeoDataframe, who can be displayed whith holoviews.

[9]:

gdf_info = xsar.product_info(
    [
        xsar.get_test_file('S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE'),
        xsar.get_test_file('S1A_IW_SLC__1SDV_20170907T102951_20170907T103021_018268_01EB76_Z010.SAFE')

    ],
    columns=[ 'safe', 'dsid', 'meta', 'geometry' ]
)

# 'meta' column is dropped for display
gv.feature.land * gv.Polygons(gdf_info.drop(['meta'], axis=1)).opts(tools=['hover'])

[9]:

[10]:

gdf_info.iloc[0]['meta']

[10]:

Single dataset

SENTINEL1_DS:S1A_WV_SLC__1SSV_20160510T101603_20160510T102143_011195_010EA1_Z010.SAFE:WV_001

ipf	2.7
platform	SENTINEL-1A
swath	WV
product	SLC
pols	VV
orbit_pass	Descending
pixel size	3.7 * 4.1 meters (xtrack * atrack)
coverage	20km * 20km (atrack * xtrack )
start_date	2016-05-10 10:16:03.843410
stop_date	2016-05-10 10:16:06.826137
dsid	WV_001

[ ]: