ScyllaDB Documentation Logo Documentation
  • Server
    • ScyllaDB Open Source
    • ScyllaDB Enterprise
    • ScyllaDB Alternator
  • Cloud
  • Tools
    • ScyllaDB Manager
    • ScyllaDB Monitoring Stack
    • ScyllaDB Operator
  • Drivers
    • CQL Drivers
    • DynamoDB Drivers
Download
Menu

Caution

You're viewing documentation for a previous version of ScyllaDB Monitoring. Switch to the latest stable version.

ScyllaDB Monitoring Scylla Monitoring Stack

Scylla Monitoring Stack¶

Scylla Monitoring Stack is a full stack for Scylla monitoring and alerting. The stack contains open source tools including Prometheus and Grafana, as well as custom Scylla dashboards and tooling.

_images/monitor.png

The Scylla Monitoring Stack consists of multiple components, wrapped in Docker containers:

  • prometheus - Collects and stores metrics

  • grafan-loki - Parses logs and generates metrics and alerts

  • alertmanager - Handles alerts

  • grafana - Dashboards server

A few optional components are used for additional services

  • grafana-image-renderer - Allows you to download a dashboard as an image.

  • Thanos sidecar - Allows a centralized Thanos server to read from the local Prometheus server.

High Level Architecture¶

_images/monitoring_stack.png

We use Prometheus for metrics collection and storage, and to generate alerts. Prometheus collects Scylla’s metrics from Scylla and the host metrics from the node_exporter agent that runs on the Scylla server.

We use Loki for metrics and alerts generation based on logs, Loki gets the logs from rsyslog agents that run on each of the Scylla servers.

The alertmanager, receives alerts from Prometheus and Loki and distributes them to other systems like email and slack.

We use Grafana to display the dashboards. Grafana gets its data from Prometheus, the alertmanager and directly from Scylla using CQL.

Choose a topic to get started:

  • User Guide

  • Download and Install

  • Procedures

  • Troubleshooting

  • Reference

  • Scylla Monitoring Stack lesson on Scylla University

For older versions of Scylla Monitoring Stack Documentation see here.

PREVIOUS
Scylla Monitoring Stack
NEXT
Download and Install Scylla Monitoring Stack
  • 3.10
    • 4.2
    • 4.1
    • 4.0
    • 3.10
    • 3.9
    • 3.8
    • 3.7
    • 3.6
    • 3.5
  • Introduction
  • Download and Install
    • Install
    • The start-all.sh script
    • Deploy without Docker
    • Docker Compose
    • System Recommendations
    • Using Thanos
  • User Guide
    • CQL Optimization Dashboard
    • Advisor
      • Some queries use ALLOW FILTERING
      • Some queries use Consistency Level: ALL
      • Some queries use Consistency Level: ANY
      • Some queries are not token-aware
      • Some SELECT queries are non-paged
      • Some queries are non-prepared
      • Some queries use reverse order
      • Compaction takes lots of memory and CPU
      • Some operation failed due to unsatisfied consistency level
      • I/O Errors can indicate a node with a faulty disk
      • Some operations failed on the replica side
      • CQL queries are not balanced among shards
      • Prepared statements cache eviction
      • System Overload
  • Procedures
    • Alert Manager
      • Alerting
    • Adding and Modifying Dashboards
    • Upgrade Guide
  • Upgrade
    • Monitoring 3.x to 3.y
    • Monitoring 2.x to 3.y
    • Monitoring 2.x to 2.y
    • Monitoring 1.x to 2.x
  • Troubleshooting
    • Troubleshooting
    • Troubleshooting Guide for Scylla Manager and Scylla Monitor Integration
  • Reference
    • Support Matrix
    • Interfaces
  • GitHub Project
  • Create an issue
  • Edit this page

On this page

  • Scylla Monitoring Stack
    • High Level Architecture
Logo
Docs Contact Us About Us
Mail List Icon Slack Icon Forum Icon
© 2023, ScyllaDB. All rights reserved.
Last updated on 29 January 2023.
Powered by Sphinx 4.3.2 & ScyllaDB Theme 1.3.4