Adding and Modifying Dashboards¶

consistency between restarts¶

By default, Grafana internal storage is inside the container. That means whenever you restart the Scylla Monitoring Stack (explicitly when restarting Grafana) any local changes will be gone.  If you are doing and saving changes from the GUI make sure to configure an external directory for Grafana.

consistency between upgrades¶

As mentioned earlier, the dashboards are provisioned from files, this means that when the files are changed, any changes stored locally will be overridden.For this reason, do not make permanent changes to a dashboard, or your changes eventually will be lost.

At large, we suggest maintaining your dashboards as files, as Scylla Monitor does.

The template class system¶

Scylla Monitoring Stack dashboard templates use class attribute that can be added to any JSON object in a template file. The different classes are defined in a file.

The class system resembles CSS classes. It is hierarchical, so a class type definition can have a class attribute and it would inherit that class attributes, the inherit class can add or override inherited attributes.

In the template file, you can also add or override attributes.

The Scylla Monitor generation script, uses the types.json file and a template file and creates a dashboard.

When generating dashboards, each class will be replaced by its definition.

For example row in the type.json defined

{
 "base_row": {
     "collapse": false,
     "editable": true
 },
 "row": {
     "class": "base_row",
     "height": "250px"
 }
 }

In a Template it will be used like

{
     "class": "row",
     "height": "150px",
     "panels": [
     ]
}

The output will be

{
     "class": "row",
     "collapse": false,
     "editable": true,
     "height": "150px",
     "panels": [

     ]
}

We can see that the template added the panels attribute and override the height attribute.

Panel example¶

Look at the following example that defines a row inside a dashboard with a graph panel for the available disk size.

{
     "class": "row",
     "panels": [
         {
             "class": "bytes_panel",
             "span": 3,
             "targets": [
                 {
                     "expr": "sum(node_filesystem_avail) by (instance)",
                     "intervalFactor": 1,
                     "legendFormat": "",
                     "metric": "",
                     "refId": "A",
                     "step": 1
                 }
             ],
             "title": "Available Disk Size"
         }
     ]
}

In the example, we used the bytes_panel class that generate a graph with bytes as units (that would mean that your Y axis unit would adjust themselves to make the graph readable (i.e. GB, MB, bytes, etc’)

You can also see that we override the span attribute to set the panel size.

To get a grasp of the difference, take a look at the Grafana panel example and see how it looks originally.

Grafana Formats and Layouts¶

The Grafana layout used to be based on rows, where each row contained multiple panels. Each row would have a total of 12 panels and if the total span of the panels was larger than 12, it would break them into multiple lines. This is no longer the case.

Starting from Grafana version 5.0 and later, rows are no longer supported, it was replaced with a layout that uses absolute positions (ie. X,Y, height, width).

The server should be backward compatible, but we’ve found it had issues with parsing it correctly. More so, absolute positions are impossible to get right when done by hand.

To overcome these issues, the dashboard generation script will generate the dashboards in the Grafana version 5.0 format. In the transition, rows will be replaced with a calculated absolute position.

The panel’s height will be taken from their row. The span attribute is still supported as is row height.

you can use the gridPos attribute which is a Grafana 5.0 format, but unlike Grafana, you can use partial attributes.

gridPos has the following attributes:

{
   "x": 0,
   "y": 0,
   "w": 24,
   "h": 4
 }

When using Scylla’s template you don’t need to supply all of the attributes, so for example to specify that a row is 2 units high you can use:

{
   "gridPos": {
      "h": 2
    }
}

Generating the dashboards from templates (generate-dashboards.sh)¶

Validation¶

After making changes to a template, run the generate_generate-dashboards.sh you should see that it run without any errors.

Refresh your browser for changes to take effect. Make sure that your panels contain data, if not, maybe there is something wrong with your expr attribute.