In this lesson we will:

  • Learn how to emit metrics from Apache Druid to Kafka.

Druid Operational Metrics

Apache Druid captures detailed metrics about how it is operating and performing, including query performance, memory usage, CPU utilisation, ingestion speed and more.

We may need to capture and monitor these metrics in order to investigate problems or identify proactive optimisations which can improve the performance of the cluster.

In this article we explain how this process is turned on and managed using configuration files. As many people are also using Kafka alongside Druid, we also explain how to push the metrics to Kafka where they can be routed and processed more freely than they can in log files.

Configuring Metrics Emitting

By default, Druid doesn't emit any metrics until this process is explicitly turned on.

As you may know, the Druid configuration files are organised based on the type of cluster you are starting. For the purposes of this demo, we can work with a nano single server on a local laptop, so we would start with the configuration files stored here:


Though we can configure components individually to capture different metrics in different ways, we can begin by editing properties which are common to all servers in the common properties file.

vim conf/druid/single-server/nano-quickstart/_common/

Checking in this file, you can see that we are setup to log metrics to a "noop" emitter i.e. "no operation" or do nothing with the metrics.


Though our ultimate destination is Kafka, let's briefly log to a file to see what happens. This can be achieved by changing the noop to logging to request that metrics are logged to the log file.


If you restart Druid, your metrics will start logging to component specific log files such as the following broker log file every minute by default.


As an example, you should fine JVM related metrics related to memory usage and garbage collection:

2021-03-22T11:05:08,343 INFO [MonitorScheduler-0] - {"feed":"metrics","timestamp":"2021-03-22T11:05:08.343Z","service":"druid/broker","host":"localhost:8082","version":"0.20.1","metric":"jvm/gc/mem/max","value":536870920,"gcGen":["old"],"gcGenSpaceName":" space  string [internal]","gcName":["g1"]}
2021-03-22T11:05:08,343 INFO [MonitorScheduler-0] - {"feed":"metrics","timestamp":"2021-03-22T11:05:08.343Z","service":"druid/broker","host":"localhost:8082","version":"0.20.1","metric":"jvm/gc/mem/capacity","value":198180872,"gcGen":["old"],"gcGenSpaceName":" space  string [internal]","gcName":["g1"]}
2021-03-22T11:05:08,343 INFO [MonitorScheduler-0] - {"feed":"metrics","timestamp":"2021-03-22T11:05:08.343Z","service":"druid/broker","host":"localhost:8082","version":"0.20.1","metric":"jvm/gc/mem/used","value":0,"gcGen":["old"],"gcGenSpaceName":" space  string [internal]","gcName":["g1"]}
2021-03-22T11:05:08,344 INFO [MonitorScheduler-0] - {"feed":"metrics","timestamp":"2021-03-22T11:05:08.343Z","service":"druid/broker","host":"localhost:8082","version":"0.20.1","metric":"jvm/gc/mem/init","value":508559368,"gcGen":["old"],"gcGenSpaceName":" space  string [internal]","gcName":["g1"]}
2021-03-22T11:05:08,345 INFO [MonitorScheduler-0] - {"feed":"metrics","timestamp":"2021-03-22T11:05:08.345Z","service":"druid/broker","host":"localhost:8082","version":"0.20.1","metric":"jvm/heapAlloc/bytes","value":887472}

In files such as broker.log, you will also see metrics specific and relevant to the broker such as these on query performance and data processed.

2021-03-22T12:11:02,837 INFO [HttpClient-Netty-Worker-3] - {"feed":"metrics","timestamp":"2021-03-22T12:11:02.836Z","service":"druid/broker","host":"localhost:8082","version":"0.20.1"
2021-03-22T12:11:02,848 INFO [DruidSchema-Cache-0] - {"feed":"metrics","timestamp":"2021-03-22T12:11:02.848Z","service":"druid/broker","host":"localhost:8082","version":"0.20.1","metr
2021-03-22T12:11:02,849 INFO [DruidSchema-Cache-0] - {"feed":"metrics","timestamp":"2021-03-22T12:11:02.849Z","service":"druid/broker","host":"localhost:8082","version":"0.20.1","metr


Druid has the concept of monitors which extends the metric system to capture additional types of metrics. These also have to be explicitly turned on in the configuration file in order to capture additional metrics

By default, the JVM Monitor is configured in this way in the above common properties, but we can add another to the historical process.

vim conf/druid/single-server/nano-quickstart/historical/

Restart the server, and you will see that we have new metrics being logged including query/count which is captured from the new monitor.

historical.log:2021-03-22T11:45:18,128 INFO [MonitorScheduler-0] - {"feed":"metrics","timestamp":"2021-03-22T11:45:18.128Z","service":"druid/historical","host":"localhost:8083","version":"0.20.1","metric":"query/count","value":0}

Connecting HTTP Emits to Kafka

Next up, we can emit our metrics into Kafka to break free of the Druid log files.

The first thing we need to do is to pull down a Druid extension which allows us to do this which is not distributed with the Druid binary.

java -classpath "/Users/benjaminwootton/development/_tools/apache-druid-0.20.1/lib/*" org.apache.druid.cli.Main tools pull-deps --clean -c org.apache.druid.extensions.contrib:kafka-emitter:0.20.1

Next, we can edit the to load the extension:

druid.extensions.loadList=[ "druid-kafka-indexing-service", "kafka-emitter"]

Finally, we can set some parameters in the common properties file, or indeed one of the individual processes, to point towards our Kafka broker.


When you restart Druid, you should then see metrics published to your Kafka broker on the configured topic.

To check this and analyse the data, you could of course re-ingest these metrics back into Druid.


This lesson demonstrated key concept such as metrics, monitors and how these are configured within Druid. We also demonstrated how we can emit these metrics through Kafka where they can then be processed by a monitoring tool or indeed within an OLAP database such as Druid.

Next Lesson:

Anomaly Detection Using Data Stored In Apache Druid

How to emit operational metrics from Druid into Apache Kafka for monitoring purposes.

0h 15m

Continuous Delivery For Data Engineers

This site has been developed by the team behind Timeflow, an Open Source CI/CD platform designed for Data Engineers who use dbt as part of the Modern Data Stack. Our platform helps Data Engineers improve the quality, reliability and speed of their data transformation pipelines.

Join our mailing list for our latest insights on Data Engineering:

Timeflow Academy is the leading online, hands-on platform for learning about Data Engineering using the Modern Data Stack. Bought to you by Timeflow CI

© 2023 Timeflow Academy. All rights reserved