Send logs to Elasticsearch®
You can store logs from one of your Aiven services in an external Elasticsearch service.
Collect these values for the connection:
Variable | Description |
---|---|
ELASTICSEARCH_USER | User name to access the Elasticsearch service. |
ELASTICSEARCH_PASSWORD | Password to access the Elasticsearch service. |
ELASTICSEARCH_HOST | HTTPS service host of your external Elasticsearch service. |
ELASTICSEARCH_PORT | Port to use for the connection. |
CA_CERTIFICATE | CA certificate in PEM structure, if necessary. |
CONNECTION_NAME | Name of this external connection to be used with Aiven services. |
Create external Elasticsearch integration
Start by setting up an external service integration for Elasticsearch.
- Log in to the Aiven Console.
- Navigate to Service Integration from the menu on the left.
- You'll see a list of external services you can integrate with Aiven.
- Select External Elasticsearch from the list.
- Select Add new endpoint.
- Set a preferred endpoint name, we'll call it
CONNECTION_NAME
later. - In the connection URL field set the connection string in a format
https://ELASTICSEARCH_USER:ELASTICSEARCH_PASSWORD@ELASTICSEARCH_HOST:ELASTICSEARCH_PORT
, using your own values for those parameters. - Set desired index prefix, that doesn't overlap with any of already existing indexes in your Elasticsearch service.
- If you need a certificate to access the endpoint, add the body of your CA certificate in PEM format. This field is optional.
- Set other fields based on your requirements, or leave the default values there.
- Select Create.
A new service integration will be added. You can now reference your
service by the CONNECTION_NAME
you chose.
Send logs to an external service
- Navigate to Services from the menu on the left.
- Select the service which logs you want to send to the external Elasticsearch service.
- On the Overview page of your service, navigate to the Service integrations section.
- Select Manage integrations.
- Select Elasticsearch Logs from the list.
- In the newly appeared modal window, select the endpoint with name
CONNECTION_NAME
from the list and select ENABLE. Close the modal window. - Observe the status change for newly added integration in the Service integrations section on the Overview page of your service.
- Verify that the logs are flowing into your Elasticsearch.
note
Logs are split per day with index name consisting of your desired index
prefix and a date in a format year-month-day, for example
logs-2022-08-30
.
note
You can also set up the integration using Aiven CLI and the following commands:
warning
Integration are not available on Hobbyist plans. To enable integrations, select at least a startup plan.