Metrics

Taskforce.sh provides some useful metrics for your queues.

The metrics view provides you a glimpse on how your queues are performing, this will allow you to quickly discover if some queue is running slower than what you expect, so that you can take actions to resolve the problems.

The first section of the metrics view shows the global statistics for the queue, the total number of jobs "In Queue", that is, jobs that are currently either waiting for being processed or being processed right now.

It also shows you how many jobs in total have been completed and failed, note that for this metric to be accurate you must either enable metrics on your queues or keep all the completed and failed jobs (usually not recommended since it will consume more and more space from your Redis instance).

The second second section shows real-time metrics for up to the last 2 weeks with a minimum granularity of 1 minute, in other words, you can see how many jobs per minute your queue is completing (or failing).

The third section displays performance statistics. Currently these statistics requires you to keep at least 1k completed jobs on the queue.

Enabling Queue Metrics

In order to enable metrics collection on the queues you need to add the metrics options to your queue.

Bull

In Bull you should enable the metrics on every queue instance that defines a processor:

const Queue = require('bull');

new Queue("queue-with-metrics", {
  metrics: {
    maxDataPoints: Queue.utils.MetricsTime.TWO_WEEKS
  }
})

queue.process('myprocessor');

We recommend to keep two weeks of metrics, which would require around 120Kb of RAM from your Redis instance.

BullMQ

In BullMQ you should enable the metrics on every worker:

import { Worker, MetricsTime } from 'bullmq'

const worker = new Worker('queue-with-metrics', {
  connection,
  metrics: {
    maxDataPoints: MetricsTime.TWO_WEEKS,
  },
});