2. Command line tools

The most straightforward way of using sen2mosaic it to call its various stages from the Linux command line. Here the functionality of each of the four commands is explained. In the next section we show how it can be used by example.

Note

Remember, each command line script has a help flag, which can be examined when in doubt.

2.1. Getting L1C data

Data from Sentinel-2 are available from the Copernicus Open Access Data Hub, which has a graphical interface to download scenes from selected areas. Whilst useful for smaller areas, generating mosaics at national scales requires a volume of data which makes this extremely labour intensive.

Note

If you already have access to Sentinel-2 data, you can skip straight to preprocess.py. This may be the case if you’re using a cloud platform where Sentinel-2 data archives are stored at the same location as servers.

The alternative is to download data using the API Hub. This system allows users to search for files using conditions on the command line, and automatically download files. To interface with the API hub, we use an excellent open source utility called Sentinelsat. This operates both as a command line tool, and as a Python API, which we use here. You will need to sign up for an account at Scihub.

download.py is a program to interface with Sentinelsat to download Sentinel-2 files, specifying a particular tile, date ranges and degrees of cloud cover. It will also decompress and tidy up .zip files, ready for use with preprocess.py.

Help for download.py can be viewed by typing s2m download --help:

usage: download.py [-h] -u USER -p PASS -t [TILES [TILES ...]] [-l LEVEL]
                [-s START] [-e END] [-c %] [-m MB] [-o PATH] [-r]

Download Sentinel-2 data from the Copernicus Open Access Hub, specifying a
particular tile, date ranges and degrees of cloud cover.

Required arguments:
-u USER, --user USER  Scihub username
-p PASS, --password PASS
                        Scihub password
-t [TILES [TILES ...]], --tiles [TILES [TILES ...]]
                        Sentinel 2 tile name, in format ##XXX

Optional arguments:
-l LEVEL, --level LEVEL
                        Set to search and download level '1C' (default) or
                        '2A' data. Note that L2A data may not be available at
                        all locations.
-s START, --start START
                        Start date for search in format YYYYMMDD. Defaults to
                        20150523.
-e END, --end END     End date for search in format YYYYMMDD. Defaults to
                        today's date.
-c %, --cloud %       Maximum percentage of cloud cover to download.
                        Defaults to 100 % (download all images, regardless of
                        cloud cover).
-m MB, --minsize MB   Minimum file size to download in MB. Defaults to 25
                        MB.
-o PATH, --output_dir PATH
                        Specify an output directory. Defaults to the present
                        working directory.
-r, --remove          Remove level 1C .zip files after decompression.

For example, to download all data for tile 36KWA between for May and June 2017, with a maximum cloud cover percentage of 30 %, specifying an output location and removing decompressed .zip files, use the following command:

s2m download -u user.name -p supersecret -t 36KWA -s 20170501 -e 20170630 -c 30 -r -o /path/to/DATA_dir/

Note

What if I want data before 6th December 2016?.

The format in which Sentinel-2 data is distributed was modified in December 2016, and the earlier format is not well supported. As there is a limited volume of data from before this date, we recommend downloading the data from Scihub.

A nice tool to help out with this is aria2. After adding products to your basket at Sentinelhub, you’ll download a metadata file called products.meta4. Use aria2 to download the file’s contents as follows:

aria2c --http-user=username --http-passwd=supersecret --check-certificate=false --max-concurrent-downloads=2 -M products.meta4

2.2. Processing to L2A

Once you have Sentinel-2 (Level 1C) data, the next step is to perform atmospheric correction and identify clouds and cloud shadows. This step is based on sen2cor.

Note

If you already have access to Sentinel-2 L2A, skip straight to mosaic.py. This may be the case if you’re using a cloud platform where Sentinel-2 data archives are stored at the same location as servers. You can also skip this step if you’re happy to build a mosaic using L1C data, but be aware that the output will be of lower quality.

preprocess.py takes a list of level 1C .SAFE files as input, initiates sen2cor, and performs simple modifications to improve the quality of it’s cloud and cloud shadow mask.

Help for preprocess.py can be viewed by typing s2m preprocess --help:

usage: preprocess.py [-h] [-t TILE] [-g GIPP] [-o DIR] [-res 10/20/60]
                    [-s PATH] [-s255 PATH] [-p N] [-v]
                    [L1C_FILES [L1C_FILES ...]]

Process level 1C Sentinel-2 data from the Copernicus Open Access Hub to level
2A. This script initiates sen2cor, which performs atmospheric correction and
generate a cloud mask. This script also performs simple improvements to the
cloud mask.

Positional arguments:
L1C_FILES             Sentinel 2 input files (level 1C) in .SAFE format.
                        Specify one or more valid Sentinel-2 .SAFE, a
                        directory containing .SAFE files, a Sentinel-2 tile or
                        multiple granules through wildcards (e.g.
                        *.SAFE/GRANULE/*), or a file containing a list of
                        input files. Leave blank to process files in current
                        working directoy. All granules that match input
                        conditions will be atmospherically corrected.

Optional arguments:
-t TILE, --tile TILE  Specify a specific Sentinel-2 tile to process. If
                        omitted, all tiles in L1C_FILES will be processed.
-g GIPP, --gipp GIPP  optionally specify a custom L2A_Process settings file.
-o DIR, --output_dir DIR
                        Specify a directory to output level 2A files. If not
                        specified, atmospherically corrected images will be
                        written to the same directory as input files.
-res 10/20/60, --resolution 10/20/60
                        Process only one of the Sentinel-2 resolutions, with
                        options of 10, 20, or 60 m. Defaults to processing all
                        three. N.B It is not currently possible to only the 10
                        m resolution, an input of 10 m will instead process
                        all resolutions.
-s PATH, --sen2cor PATH
                        Path to sen2cor (v2.8), if not callable with the
                        default 'L2A_Process'.
-s255 PATH, --sen2cor255 PATH
                        Path to sen2cor (v2.5.5), required if processing
                        Sentinel-2 data with the old file format.
-p N, --n_processes N
                        Specify a maximum number of tiles to process in
                        paralell. Bear in mind that more processes will
                        require more memory. Defaults to 1.
-v, --verbose         Make script verbose.

For example, to run preprocess.py on a set of level 1C Sentinel-2 files in a directory, processing only 20 m resolution data, use the following command:

s2m preprocess -res 20 /path/to/DATA_dir/

The pre-processing script supports parallel processing of L1C files. Be aware that this will entail greater processing and memory requirements than are available on most standard desktop PCs. To parallel process 3 tiles for the 20 m resolution, input:

s2m preprocess -res 20 -n 3 /path/to/DATA_dir/

2.3. Processing to a mosaic

The final sen2mosaic processing step creates a composite image of multiple Sentinel-2 level 2A images in user-specified tiling grid. This script takes L2A data as input, identifies the tiles that fall within the specified spatial extent, and builds a composite image using available data to produce single-band GeoTiff files for easy use in classification systems.

Note

You can also build a mosaic using L1C data, but be aware that the output will be of lower quality.

mosaic.py takes a directory containing Sentinel-2 .SAFE files, an output image extent (xmin, ymin, xmax, ymax) and projection EPSG code as inputs, along with a series of options to modify the compositing approach.

Help for mosaic.py can be viewed by typing s2m mosaic --help:

usage: mosaic.py [-h] -te XMIN YMIN XMAX YMAX -e EPSG [-res m] [-l 1C/2A]
                [-st START] [-en END] [-pc PC] [-m [N [N ...]]] [-b] [-i]
                [-t DIR] [-o DIR] [-n NAME] [-p N] [-v]
                [PATH [PATH ...]]

Process Sentinel-2 data to a composite mosaic product to a customisable grid
square, based on specified UTM coordinate bounds. Data are output as GeoTiff
files for each spectral band, with .vrt files for ease of visualisation.

positional arguments:
PATH                  Sentinel 2 input files (level 1C/2A) in .SAFE format.
                        Specify one or more valid Sentinel-2 .SAFE, a
                        directory containing .SAFE files, a Sentinel-2 tile or
                        multiple granules through wildcards (e.g.
                        *.SAFE/GRANULE/*), or a file containing a list of
                        input files. Leave blank to process files in current
                        working directoy. All granules that match input
                        conditions will be included.

required arguments:
-te XMIN YMIN XMAX YMAX, --target_extent XMIN YMIN XMAX YMAX
                        Extent of output image tile, in format <xmin, ymin,
                        xmax, ymax>.
-e EPSG, --epsg EPSG  EPSG code for output image tile CRS. This must be UTM.
                        Find the EPSG code of your output CRS as
                        https://www.epsg-registry.org/.
-res m, --resolution m
                        Specify a resolution in metres.

optional arguments:
-l 1C/2A, --level 1C/2A
                        Input image processing level, '1C' or '2A'. Defaults
                        to '2A'.
-st START, --start START
                        Start date for tiles to include in format YYYYMMDD.
                        Defaults to processing all dates.
-en END, --end END    End date for tiles to include in format YYYYMMDD.
                        Defaults to processing all dates.
-pc PC, --percentile PC
                        Specify a percentile of reflectance to output.
                        Defaults to 25 percent, which tends to produce good
                        results.
-m [N [N ...]], --masked_vals [N [N ...]]
                        Specify SLC values to not include in the mosaic (e.g.
                        -m 7 8 9). See http://step.esa.int/main/third-party-
                        plugins-2/sen2cor/ for description of sen2cor mask
                        values. Defaults to 'auto', which masks values 0 and
                        9. Also accepts 'none', to include all values.
-b, --colour_balance  Perform colour balancing between tiles. Not generally
                        recommended, particularly where working over large
                        areas. Defaults to False.
-i, --improve_mask    Apply improvements to Sentinel-2 cloud mask. Not
                        generally recommended, except where a very
                        conservative mask is desired. Defaults to no
                        improvement.
-t DIR, --temp_dir DIR
                        Directory to write temporary files, only required for
                        L1C data. Defaults to '/tmp'.
-o DIR, --output_dir DIR
                        Specify an output directory. Defaults to the present
                        working directory.
-n NAME, --output_name NAME
                        Specify a string to precede output filename. Defaults
                        to 'mosaic'.
-p N, --n_processes N
                        Specify a maximum number of tiles to process in
                        paralell. Bear in mind that more processes will
                        require more memory. Defaults to 1.
-v, --verbose         Make script verbose.

For example, to run mosaic.py in the directory /path/to/DATA_dir/ which contains level 2A files to create a 200 x 200 km output tile in the UTM36S projection at 20 m resoluton, input:

s2m mosaic -te 700000 7900000 900000 8100000 -e 32736 -res 20 /path/to/DATA_dir/

To do the same operation, but specifying an output directory, a name to prepend to outputs from this tile, and performing inter-scene colour balancing and corrections to the sen2cor mask, input:

s2m mosaic -te 700000 7900000 900000 8100000 -e 32736 -res 20 -o /path/to/output/ -n my_output -b -c /path/to/DATA_dir/