Skip to main content

Google Cloud Storage in FarEarth

Google Cloud Storage (GCS) is a scalable and secure object storage service designed for reliably storing and accessing unstructured data across distributed systems.

You can integrate GCS into FarEarth, both as a pickup and an archive.

This document is an overview of FarEarth requirements when integrating your own GCS solution.

Helpful tips​

When configuring the buckets, you do not have to provide a hierarchical namespace on the bucket.

Service Account​

FarEarth requires that you configure a Service Account to connect to your bucket and Pub/Sub queues.

  • You will need to provide FarEarth with a Service Account key (JSON file)

  • The IAM role for the Service Account must provide the permissions in the table below

GCS buckets​

When configuring a bucket, you must enable Cross-Origin Resource Sharing (CORS) for the following origin:

  • gateway.farearth.space
  • access method: GET

You need to provide FarEarth with a:

  • project ID
  • region where bucket is configured
  • name of the bucket

Google Pub/Sub triggers​

FarEarth uses Pub/Sub messages to be notified when new data is available on a pickup.

When configuring Pub/Sub, you will have to configure a:

  • subscription
  • topic
  • GCS notifications to trigger on Object Creation events, and to publish to the Pub/Sub topic

You need to provide FarEarth with a:

  • subscription ID
  • topic

IAM Service Account permissions and policies​

The table below lists the permissions and policies required for the IAM Service Account that you need to configure for use by FarEarth.

Permissions
pubsub.subscriptions.consume
pubsub.subscriptions.create
pubsub.subscriptions.delete
pubsub.subscriptions.get
pubsub.subscriptions.getIamPolicy
pubsub.subscriptions.list
pubsub.subscriptions.setIamPolicy
pubsub.subscriptions.update
pubsub.subscriptions.consume
pubsub.subscriptions.create
pubsub.subscriptions.delete
pubsub.subscriptions.get
pubsub.subscriptions.getIamPolicy
pubsub.subscriptions.list
pubsub.subscriptions.setIamPolicy
pubsub.subscriptions.update
pubsub.topics.attachSubscription
pubsub.topics.create
pubsub.topics.delete
pubsub.topics.detachSubscription
pubsub.topics.get
pubsub.topics.getIamPolicy
pubsub.topics.list
pubsub.topics.publish
pubsub.topics.setIamPolicy
pubsub.topics.update
pubsub.topics.updateTag
resourcemanager.hierarchyNodes.listEffectiveTags
resourcemanager.projects.get
storage.bucketOperations.cancel
storage.bucketOperations.get
storage.bucketOperations.list
storage.hmacKeys.list
storage.buckets.create
storage.buckets.createTagBinding
storage.buckets.delete
storage.buckets.deleteTagBinding
storage.buckets.enableObjectRetention
storage.buckets.get
storage.buckets.getIamPolicy
storage.buckets.getObjectInsights
storage.buckets.list
storage.buckets.listEffectiveTags
storage.buckets.listTagBindings
storage.buckets.relocate
storage.buckets.restore
storage.buckets.setIamPolicy
storage.buckets.update
storage.folders.create
storage.folders.delete
storage.folders.get
storage.folders.list
storage.folders.rename
storage.managedFolders.create
storage.managedFolders.delete
storage.managedFolders.get
storage.managedFolders.getIamPolicy
storage.managedFolders.list
storage.managedFolders.setIamPolicy
storage.multipartUploads.abort
storage.multipartUploads.create
storage.multipartUploads.list
storage.multipartUploads.listParts
storage.objects.create
storage.objects.delete
storage.objects.get
storage.objects.getIamPolicy
storage.objects.list
storage.objects.move
storage.objects.overrideUnlockedRetention
storage.objects.restore
storage.objects.setIamPolicy
storage.objects.setRetention
storage.objects.update
storagetransfer.jobs.create
storagetransfer.jobs.delete
storagetransfer.jobs.list
storagetransfer.jobs.update
storagetransfer.operations.list