Kafka plugin

Purpose

This plugin will post alarms and metrics to a Kafka node.

Different topics for alarms and metrics can be created to distribute the events and to facilitate the integration with third party consummers.

The Kafka plugin will flush received events every 30 seconds to the Kafka node.

If the node cannot be reached, events will be queued until connectivity is back or until queue is full.

Configuration

From the plugin menu of Pro.Monitor, select Kafka in the plugin drop down and press Add

The plugin has the following parameters:

Parameter	Description	Mandatory
Name	Give a name to the plugin	Yes
Boostrap servers	The list of nodes	Yes
Client Id	The id to use for the promonitor producer	Yes
Alarms topic	The topic to use for alarms	Yes
Metrics topic	The topic to use for metrics	Yes
User	The user for authentication	No
Password	The password for authentication	No
Properties	Semicolon separated list of properties	No
Max queue size	The size of the queue holding the alarms and metrics	Yes
Send alarms	If active, alarms received by the plugin will be sent	Yes
Send metrics	If active, metrics received by the plugin will be sent	Yes

Note: If the alarm or metric queue is full, incoming events will be discarded until space is made in the queue.

Topics

You can use variable names in topic to dynamically redirect the events into specific topics. You can use metric or alarm tag property names as variables:

Example: Let's consider following metric:

{"timestamp":1594742051,"value":4.0,"key":"sap.hana.HANA_THREADS.THREADS_COUNT","tags":{"instance":"aglgesrvhana02b","sid":"HA5","group":"Agentil"}}

If you set “metrics-%group%-%sid%“ in metric topic, this metric will be put in topic: metrics-Agentil-HA5

Properties

Properties are key/value pairs separated by semicolon: val1=12;val2=no

The list of exhaustive properties can be found here

Default settings:

Batch size is set to 500KB

Linger time is set to 10ms (Time to wait for new events before sending an incomplete batch)

Compression is OFF (see property compression.type=gzip)

Example