Auxiliary file generation

Overview

The FarEarth Archive supports the generation of optional auxiliary metadata files that serves to augment any products being published. A list of currently supported auxiliary file generators are provided below.

note

More than one generator can be added in the auxFileGenerators list

The Auxiliary file generation can either be configured against the entire Archive, or conditionally by Workflow steps.

Configure in Workflow

Additional parameters can be added to the archiving step in the workflow, as shown in the example below. In this example, the PRODUCT_FILE_JSON and TILE_INFO_JSON file generators have been added. In addition, the file-naming convention for the tile-info JSON file is specified.

        {
            "id": "archive-step",
            "executorId": "farearth.eks-archive-0",
            "displayName": "Archiving",
            "multi": true,
            "optional": true,
            "continueOnFail": true,
            "parameters": {
                "auxFileGenerators": [
                    {
                        "id": "PRODUCT_FILE_JSON"
                    },
                    {
                        "id": "TILE_INFO_JSON",
                        "tileIdTemplate": "{id}_tileInfo.json"
                    }
                ]
            }
        }

note

The sequence of the generators defined in the auxFileGenerators list is not important. The sequence in which they are generated are determined by an assigned priority, where a lower number is given precedence and is generated first

Configure in Archive

An example of an archive configuration is shown below

{
    "id": "farearth.azure-archive-0",
    "protocol": "azure",
    "auxFileGenerators": [
        {
            "id": "TILE_INFO_JSON"
        }
    ],
    "properties": {
        "containerName": "archive",
        "baseUrl": "http...ows.net/",
        "connectionString": "DefaultEndpo...indows.net" 
    }
}

Auxiliary file types

Tile info JSON

ID: TILE_INFO_JSON
Priority: 25

Properties for tile info JSON generator:

Property	Example	Description
`enabled`	`true` `false`	Default: `true`. Determines whether the generator is enabled
`relativePathTemplate`	`tileInfo.json`	Templated field to modify the location of the JSON file within the product folder
`tileIdTemplate`	`{id}_tileInfo.json`	Templated field to modify the name of the generated file (defaults to the product ID)
`pathTemplate`		Path to the folder containing the required data for the generator. The default path should be correct by default

XML L1C Tile metadata

ID: META_DATA_L1C_TILE_XML
Priority: 35

Properties for XML L1C tile metadata generator:

Property	Example	Description
`enabled`	`true` `false`	Default: `true`. Determines whether the generator is enabled
`relativePathTemplate`	`tileInfo.xml`	Templated field to modify the location of the XML file within the product folder
`tileIdTemplate`	`{id}_tileInfo.xml`	Templated field to modify the name of the generated file (defaults to the product ID)

Below is an example of a configuration for the META_DATA_L1C_TILE_XML configuration.

{
    "id": "farearth.azure-archive-0",
    "protocol": "azure",
    "auxFileGenerators": [
        {
            "id": "META_DATA_L1C_TILE_XML",
            "relativePathTemplate": "AUX_DATA/{productType}_{start_datetime.isoSafe}_{end_datetime.isoSafe}_{processed_datetime.isoSafe}_{utmZone}{latitudeBand}{gridSquare}_R{sceneRow}C{sceneCol}_{version}_tile.xml",
            "tileIdTemplate": "TRUBIT_{productType}_{correlationId}_{processed_datetime.isoSafe}_{utmZone}{latitudeBand}{gridSquare}_R{sceneRow}C{sceneCol}_{version}"
        }
    ],
    "properties": {
        "containerName": "archive",
        "baseUrl": "http...ows.net/",
        "connectionString": "DefaultEndpo...indows.net" 
    }
}

XML L1C product metadata

ID: META_DATA_L1C_PRODUCT_XML
Priority: 65

Properties for the XML L1C product metadata generator:

Property	Example	Description
`enabled`	`true` `false`	Default: `true`. Determines whether the generator is enabled
`relativePathTemplate`	`MTD_L1C.xml`	Templated field to modify the location of the XML file within the product folder
`processingLevel`	`Level-1C`	The processing level. Defaults to `Level-1C`
`productType`	`L1C`	The product type. Defaults to `L1C`
`spacecraft`	`TruBit-1`	The spacecraft ID to use. Defaults to the `spacecraft` field of the product
`referenceBandName`	`RED`	The name of the band to use as a reference. Defaults to `RED`
`redBandName`	`RED`	The name of the red band. Defaults to `RED`
`greenBandName`	`GREEN`	The name of the green band. Defaults to `GREEN`
`blueBandName`	`BLUE`	The name of the blue band. Defaults to `BLUE`
`productUriTemplate`		A template for the product URI. Defaults to the product ID
`datatakeIdentifierTemplate`		ID template for the datatake. Defaults to the value of `productUriTemplate`
`datastripIdentifierTemplate`		ID template for the datastrip. Defaults to the value of `productUriTemplate`
`tileInfoAssetId`	`TILE_INFO`	The STAC asset ID of the tile info file. Defaults to `TILE_INFO`
`flatNamespace`	`true` `false`	Whether to omit the top-level XML namespace. Defaults to `false`

Below is an example of a configuration for the META_DATA_L1C_PRODUCT_XML configuration.

{
    "id": "farearth.azure-archive-0",
    "protocol": "azure",
    "auxFileGenerators": [
        {
            "id": "META_DATA_L1C_PRODUCT_XML",
            "relativePathTemplate": "AUX_DATA/MTD_L1C.xml",
            "processingLevel": "Level-1C",
            "productType": "TRUBITL1C"
        }
    ],
    "properties": {
        "containerName": "archive",
        "baseUrl": "http...ows.net/",
        "connectionString": "DefaultEndpo...indows.net" 
    }
}

STAC product file

ID: PRODUCT_FILE_JSON
Priority: 75

note

The STAC product file generator will be enabled if not other auxFileGenerators are specified. If any other generators are specified, the PRODUCT_FILE_JSON generator must be specified explicitely

note

Disabling the PRODUCT_FILE_JSON auxiliary file generator is not advised, as other FarEarth *services rely on this file

Properties for STAC product file generator:

Property	Example	Description
`KEY_RELATIVE_PATH_TEMPLATE`	`{id}_product.json`	Template to modify the file location of the JSON product. Defaults to `{id}_product.json`

Below is an example of a configuration for the PRODUCT_FILE_JSON configuration.

{
    "id": "farearth.azure-archive-0",
    "protocol": "azure",
    "auxFileGenerators": [
        {
            "id": "PRODUCT_FILE_JSON"
        }
    ],
    "properties": {
        "containerName": "archive",
        "baseUrl": "http...ows.net/",
        "connectionString": "DefaultEndpo...indows.net" 
    }
}

Overview​

Configure in Workflow​

Configure in Archive​

Auxiliary file types​

Tile info JSON​

XML L1C Tile metadata​

XML L1C product metadata​

STAC product file​

Overview

Configure in Workflow

Configure in Archive

Auxiliary file types

Tile info JSON

XML L1C Tile metadata

XML L1C product metadata

STAC product file