Skip to main content

Copy data from Aiven for OpenSearch® to AWS S3 using elasticsearch-dump

Backup your OpenSearch® data into an AWS S3 bucket.

To copy the index data, we will be using elasticsearch-dump tool. You can read the instructions on GitHub on how to install it. From this library, we will use elasticdump command to copy the input index data to an specific output.

Prerequisites​

  • elasticsearch-dump tool installed
  • Aiven for OpenSearch cluster as the input
  • AWS S3 bucket as the output

You need to collect the following information about your Aiven for OpenSearch cluster and your AWS service:

Aiven for OpenSearch:

  • SERVICE_URI: OpenSearch service URI, which you can find in Aiven's dashboard.
  • INPUT_INDEX_NAME: the index that you aim to copy from your input source.

AWS S3:

  • AWS credentials (ACCESS_KEY_ID and SECRET_ACCESS_KEY).
  • S3 file path. This includes bucket name and file name, for for example, s3://${BUCKET_NAME}/${FILE_NAME}.json

You can find more information about AWS credentials in the AWS documentation.

Export OpenSearch index data to S3​

Use elasticsearch-dump command to copy the data from your Aiven for OpenSearch cluster to your AWS S3 bucket. Use your Aiven for OpenSearch SERVICE_URI for the input . For the output, choose an AWS S3 file path including the file name that you want for your document.

elasticdump \
--s3AccessKeyId "${ACCESS_KEY_ID}" \
--s3SecretAccessKey "${SECRET_ACCESS_KEY}" \
--input=SERVICE_URI/INPUT_INDEX_NAME --output "s3://${BUCKET_NAME}/${FILE_NAME}.json"

Resources​

Aiven for OpenSearch databases are automatically backed up, so you can check more information about how the Backup process works.