====== Kafka plugin ======

===== Purpose =====

  * This plugin will post alarms and metrics to a Kafka node.
  * Different topics for alarms and metrics can be created to distribute the events and to facilitate the integration with third party consummers.
  * The Kafka plugin will flush received events every 30 seconds to the Kafka node.
  * If the node cannot be reached, events will be queued until connectivity is back or until queue is full.

===== Configuration =====

  * From the plugin menu of Redpeaks, select ''Kafka'' in the plugin drop down and press ''Add''
  * The plugin has the following parameters:

^ Parameter ^ Description ^ Mandatory ^
| Name | Give a name to the plugin | Yes |
| Boostrap servers | The list of nodes | Yes |
| Client Id | The id to use for the promonitor producer | Yes |
| Alarms topic | The topic to use for alarms | Yes |
| Metrics topic | The topic to use for metrics | Yes |
| User | The user for authentication | No |
| Password | The password for authentication | No |
| Properties| Semicolon separated list of [[https://kafka.apache.org/25/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html|properties]] | No |
| Max queue size | The size of the queue holding the alarms and metrics | Yes |
| Send alarms | If active, alarms received by the plugin will be sent | Yes |
| Send metrics | If active, metrics received by the plugin will be sent | Yes |

**Note:** If the alarm or metric queue is full, incoming events will be discarded until space is made in the queue.

==== Topics ====

You can use variable names in topic to dynamically redirect the events into specific topics. You can use metric or alarm tag property names as variables:

**Example:** Let's consider following metric:
<file json>
{"timestamp":1594742051,"value":4.0,"key":"sap.hana.HANA_THREADS.THREADS_COUNT","tags":{"instance":"aglgesrvhana02b","sid":"HA5","group":"Agentil"}}
</file>

If you set **"metrics-%group%-%sid%"** in metric topic, this metric will be put in topic: **metrics-Agentil-HA5**

==== Properties ====

  * Properties are key/value pairs separated by semicolon: ''val1=12;val2=no''
  * The list of exhaustive properties can be found [[https://kafka.apache.org/25/javadoc/index.html?org/apache/kafka/clients/producer/KafkaProducer.html|here]]
  * **Default settings:**
    * Batch size is set to 500KB
    * Linger time is set to 10ms (Time to wait for new events before sending an incomplete batch)
    * Compression is OFF (see property //compression.type=gzip//)

===== Example =====

{{..:plugins:pasted:plugin_kafka.png}}