Connect Aiven for Apache Kafka® with Klaw#
Klaw is an open-source, web-based data governance toolkit for managing Apache Kafka® Topics, ACLs, Schemas, and Connectors. Klaw provides a self-service user interface where teams of Apache Kafka service users can request changes to the Apache Kafka configuration without the intervention of administrators.
This article provides you with the steps to connect Aiven for Apache Kafka® service with Klaw.
To connect Aiven for Apache Kafka® and Klaw, you need to have the following setup:
A running Aiven for Apache Kafka® service. See Getting started with Aiven for Apache Kafka for more information.
A running Klaw cluster. See Run Klaw from the source for more information.
Connect Aiven for Apache Kafka® to Klaw#
Follow the below steps to configure and connect Aiven for Apache Kafka® with Klaw:
Log in to the Klaw web interface.
Click Environments, and then click Clusters.
On the Clusters page, click Add Cluster to add an Aiven for Apache Kafka® cluster.
Enter the following details:
Cluster Type: Select Kafka from the drop-down list
Cluster Name: Provide a name for the cluster
Protocol: Select the protocol for your cluster
Based on the protocol selected, you need to configure Klaw application.properties file to enable connection between Aiven for Apache Kafka® and Klaw clusters.
Kafka Flavor: Select Aiven for Apache Kafka® as the flavor
Project Name: Select the project name defined in the Aiven Console
Bootstrap server: Enter the Service URI for your Apache Kafka service. You can find the service URI in the Connection information page of your service in Aiven Console
Service Name: Enter the name of the service as defined in the Aiven Console for your Apache Kafka service
Next, add the cluster to the preferred environment. Select Environments, and then select Environments from the drop-down menu.
Click Add Environment and enter the details to add Kafka environment:
Environment Name: Select environment from the drop-down list
To learn more, see Clusters and environments in Klaw documentation.
Select Cluster: Select the cluster you added from the drop-down list. The bootstrap servers and protocol details are automatically populated
Default Partitions: Enter the number of partitions based on your requirements. The default value is set to 2
Maximum Partitions: Enter the maximum number of partitions. The default value is set to 2
Default Replication factor: Enter the required replication factor. The default value is set to 1
Max Replication factor: Enter the required max replication factor. The default value is set to 1
Topic prefix (optional): Enter a topic prefix
Tenant: The value is set to default Tenant
Klaw is multi-tenant by default. Each tenant manages topics with their own teams in isolation. Every tenant has its own set of Apache Kafka® environments, and users of one tenant cannot view/access topics, or ACLS from other tenants. It provides isolation avoiding any security breach. For this topic, I have used the default tenant configuration. For more information, see Klaw documentation.
If the connection based on the cluster and environment configurations are successful, the connection status displays a blue thumbs-up icon on the Environments page in the Klaw web interface. If it is unsuccessful, a red thumbs-down icon is displayed.
For unsuccessful connections, check the authentication protocol configurations. See the Configure Klaw application.properties file section for further details. After completing these configurations, check the connection status on the Environments page by clicking the thumbs-down icon to see if the setup was successful.
Configure Klaw application.properties file#
Klaw uses the
application.properties file to configure any application-related properties, such as secret key and authentication protocol configurations. Connecting Aiven for Apache Kafka® with Klaw requires you to perform the following additional configurations in the
Secret key configuration#
Set the value of
klaw.clusterapi.access.base64.secret with a secret key in the form of a Base64 encoded string in the
application.properties file located in the following paths:
Configure authentication protocol#
You can connect Aiven for Apache Kafka® using either of the following authentication protocols:
SASL SSL (GSSAPI / Kerberos),
SASL_SSL (SCRAM SHA 256/512)
If you are using
PLAINTEXT, you do not need to perform any additional configuration.
Connect using SSL protocol#
To use SSL as the authentication protocol to connect the Apache Kafka® cluster to Klaw, you need to perform the following steps:
Retrieve SSL certificate files#
You need to retrieve the Aiven for Apache Kafka SSL certificate files. Aiven for Apache Kafka® by default enables TLS security. Download the certificates from the service overview page in the Aiven console or via the dedicated page.
Considering you have already configured the Java SSL keystore and truststore files, move the keystore named
client.keystore.p12 and truststore named
client.truststore.jks into a directory that can be easily accessed and configured with Klaw.
Configure SSL properties#
After retrieving the SSL certificate files and configuring the SSL keystore and truststore files, you need to configure these SSL values in the
Get the Cluster ID by clicking the copy icon on the Clusters page in the Klaw web interface.
Next, open the
application.propertiesfile located in the
Configure the SSL properties to connect to Apache Kafka® clusters by editing the following lines:
klawssl.kafkassl.keystore.location=client.keystore.p12 klawssl.kafkassl.keystore.pwd=klaw1234 klawssl.kafkassl.key.pwd=klaw1234 klawssl.kafkassl.truststore.location=client.truststore.jks klawssl.kafkassl.truststore.pwd=klaw1234 klawssl.kafkassl.keystore.type=pkcs12 klawssl.kafkassl.truststore.type=JKS
For the lines starting with
klawsslwith the Klaw Cluster ID.
client.keystore.p12with the path for the keystore and
klaw1234with the password configured for the keystore file.
client.truststore.jkswith the path for the truststore and
klaw1234with the password configured for the truststore file.
The following is an example of an
application.propertiesfile configured with Klaw Cluster ID, keystore, and truststore paths and passwords.
demo_cluster.kafkassl.keystore.location=/Users/demo.user/Documents/Klaw/demo-certs/client.keystore.p12 demo_cluster.kafkassl.keystore.pwd=Aiventest123! demo_cluster.kafkassl.key.pwd=Aiventest123! demo_cluster.kafkassl.truststore.location=/Users/demo.user/Documents/Klaw/demo-certs/client.truststore.jks demo_cluster.kafkassl.truststore.pwd=Aiventest123! demo_cluster.kafkassl.keystore.type=pkcs12 demo_cluster.kafkassl.truststore.type=JKS
To add multiple SSL configurations, copy and paste the above lines by prefixing them with the required cluster identification and relevant certificates.
Connect using SASL protocols#
To use protocols, such as
SASL_SSL/GSSAPI, in the
application.properties file, look for the lines starting with
acc1.kafkasasl.jaasconfig.<>, uncomment the line and enter the required values. Save the