====== Kafka plugin ====== ===== Purpose ===== * This plugin will post alarms and metrics to a Kafka node. * Different topics for alarms and metrics can be created to distribute the events and to facilitate the integration with third party consummers. * The Kafka plugin will flush received events every 30 seconds to the Kafka node. * If the node cannot be reached, events will be queued until connectivity is back or until queue is full. ===== Configuration ===== * From the plugin menu of Redpeaks, select ''Kafka'' in the plugin drop down and press ''Add'' * The plugin has the following parameters: ^ Parameter ^ Description ^ Mandatory ^ | Name | Give a name to the plugin | Yes | | Boostrap servers | The list of nodes | Yes | | Client Id | The id to use for the promonitor producer | Yes | | Alarms topic | The topic to use for alarms | Yes | | Metrics topic | The topic to use for metrics | Yes | | User | The user for authentication | No | | Password | The password for authentication | No | | Properties| Semicolon separated list of [[https://kafka.apache.org/25/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html|properties]] | No | | Max queue size | The size of the queue holding the alarms and metrics | Yes | | Send alarms | If active, alarms received by the plugin will be sent | Yes | | Send metrics | If active, metrics received by the plugin will be sent | Yes | **Note:** If the alarm or metric queue is full, incoming events will be discarded until space is made in the queue. ==== Topics ==== You can use variable names in topic to dynamically redirect the events into specific topics. You can use metric or alarm tag property names as variables: **Example:** Let's consider following metric: {"timestamp":1594742051,"value":4.0,"key":"sap.hana.HANA_THREADS.THREADS_COUNT","tags":{"instance":"aglgesrvhana02b","sid":"HA5","group":"Agentil"}} If you set **"metrics-%group%-%sid%"** in metric topic, this metric will be put in topic: **metrics-Agentil-HA5** ==== Properties ==== * Properties are key/value pairs separated by semicolon: ''val1=12;val2=no'' * The list of exhaustive properties can be found [[https://kafka.apache.org/25/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html|here]] * **Default settings:** * Batch size is set to 500KB * Linger time is set to 10ms (Time to wait for new events before sending an incomplete batch) * Compression is OFF (see property //compression.type=gzip//) ===== Example ===== {{..:plugins:pasted:plugin_kafka.png}}