kcat is a generic command line non-JVM producer and consumer for Apache Kafka. You can connect to your Upstash Kafka cluster using kcat.

If you do not have a Kafka cluster and/or topic already, follow these steps to create one.

In the cluster details section of the Upstash Console copy bootstrap endpoint, username and password. Then replace following parameters in the code snippets below with the actual values you copied earlier.

  • $BOOTSTRAP_ENDPOINT
  • $UPSTASH_KAFKA_USERNAME
  • $UPSTASH_KAFKA_PASSWORD
  • $GROUP_ID
  • $TOPIC_NAME

Query cluster metadata:

kcat -b $BOOTSTRAP_ENDPOINT -X security.protocol=SASL_SSL \
    -X sasl.mechanisms=SCRAM-SHA-512 \
    -X sasl.username=$UPSTASH_KAFKA_USERNAME \
    -X sasl.password=$UPSTASH_KAFKA_PASSWORD \
    -L

Produce a message:

echo "Hello Upstash!" | kcat -b $BOOTSTRAP_ENDPOINT
    -X security.protocol=SASL_SSL \
    -X sasl.mechanisms=SCRAM-SHA-512 \
    -X sasl.username=$UPSTASH_KAFKA_USERNAME \
    -X sasl.password=$UPSTASH_KAFKA_PASSWORD \
    -P -t $TOPIC_NAME

Fetch messages:

kcat -b $BOOTSTRAP_ENDPOINT -X security.protocol=SASL_SSL \
    -X sasl.mechanisms=SCRAM-SHA-512 \
    -X sasl.username=$UPSTASH_KAFKA_USERNAME \
    -X sasl.password=$UPSTASH_KAFKA_PASSWORD \
    -C -t $TOPIC_NAME

Consume messages using consumer groups:

kcat -b $BOOTSTRAP_ENDPOINT -X security.protocol=SASL_SSL \
    -X sasl.mechanisms=SCRAM-SHA-512 \
    -X sasl.username=$UPSTASH_KAFKA_USERNAME \
    -X sasl.password=$UPSTASH_KAFKA_PASSWORD \
    -o beginning -G $GROUP_ID $TOPIC_NAME

For more information see kcat repository.

Was this page helpful?

Use Serverless Kafka to Produce Events in Cloudflare WorkersConnect Using kaf CLI