Overview
Grafana has an official docker image. Though permission problems occurred in previous versions, it can running flawlessly. There is also an example in OKD[1]. We will start with the examples, but also do further configuration with data sources and dashboards.
Deploying Prometheus
Prometheus is supported officially by OpenShift and hence could be quicky deployed using openshift-ansible
. Simply run
$ ansible-playbook -vvv -i ${INVENTORY_FILE} playbooks/openshift-prometheus/config.yml
would automatically deploy Prometheus.
Alternatively, Prometheus installation could be customized by adding more options into inventory file. Details could be found at enabling cluster metrics chapter in documents.
Deploying Grafana
We could create a template named grafana.yaml
---
kind: Template
apiVersion: v1
metadata:
name: grafana
annotations:
"openshift.io/display-name": Grafana
description: |
Grafana server with patched Prometheus datasource.
iconClass: fa fa-cogs
tags: "metrics,monitoring,grafana,prometheus"
parameters:
- description: The location of the grafana image
name: IMAGE_GRAFANA
value: docker.io/grafana/grafana:master
- description: The location of the proxy image
name: IMAGE_PROXY
value: openshift/oauth-proxy:v1.0.0
- description: External URL for the grafana route
name: ROUTE_URL
value: ""
- description: The namespace to instantiate heapster under. Defaults to 'grafana'.
name: NAMESPACE
value: grafana
- description: The session secret for the proxy
name: SESSION_SECRET
generate: expression
from: "[a-zA-Z0-9]{43}"
objects:
- apiVersion: v1
kind: ServiceAccount
metadata:
name: grafana
namespace: "${NAMESPACE}"
annotations:
serviceaccounts.openshift.io/oauth-redirectreference.primary: '{"kind":"OAuthRedirectReference","apiVersion":"v1","reference":{"kind":"Route","name":"grafana"}}'
- apiVersion: authorization.openshift.io/v1
kind: ClusterRoleBinding
metadata:
name: grafana-cluster-reader
roleRef:
name: cluster-reader
subjects:
- kind: ServiceAccount
name: grafana
namespace: "${NAMESPACE}"
- apiVersion: route.openshift.io/v1
kind: Route
metadata:
name: grafana
namespace: "${NAMESPACE}"
spec:
host: "${ROUTE_URL}"
to:
name: grafana
tls:
termination: Reencrypt
- apiVersion: v1
kind: Service
metadata:
name: grafana
annotations:
prometheus.io/scrape: "true"
prometheus.io/scheme: https
service.alpha.openshift.io/serving-cert-secret-name: grafana-tls
namespace: "${NAMESPACE}"
labels:
metrics-infra: grafana
name: grafana
spec:
ports:
- name: grafana
port: 443
protocol: TCP
targetPort: 8443
selector:
app: grafana
- apiVersion: v1
kind: Secret
metadata:
name: grafana-proxy
namespace: "${NAMESPACE}"
stringData:
session_secret: "${SESSION_SECRET}="
# Deploy Grafana behind an oauth proxy
- apiVersion: extensions/v1beta1
kind: Deployment
metadata:
labels:
app: grafana
name: grafana
namespace: "${NAMESPACE}"
spec:
replicas: 1
selector:
matchLabels:
app: grafana
template:
metadata:
labels:
app: grafana
name: grafana
spec:
serviceAccountName: grafana
containers:
- name: oauth-proxy
image: ${IMAGE_PROXY}
imagePullPolicy: IfNotPresent
ports:
- containerPort: 8443
name: web
args:
- -https-address=:8443
- -http-address=
- -email-domain=*
- -client-id=system:serviceaccount:${NAMESPACE}:grafana
- -upstream=http://localhost:3000
- -provider=openshift
# - '-openshift-delegate-urls={"/api/datasources": {"resource": "namespace", "verb": "get", "resourceName": "grafana", "namespace": "${NAMESPACE}"}}'
- '-openshift-sar={"namespace": "${NAMESPACE}", "verb": "list", "resource": "services"}'
- -tls-cert=/etc/tls/private/tls.crt
- -tls-key=/etc/tls/private/tls.key
- -client-secret-file=/var/run/secrets/kubernetes.io/serviceaccount/token
- -cookie-secret-file=/etc/proxy/secrets/session_secret
- -skip-auth-regex=^/metrics,/api/datasources,/api/dashboards
volumeMounts:
- mountPath: /etc/tls/private
name: grafana-tls
- mountPath: /etc/proxy/secrets
name: secrets
- name: grafana
image: ${IMAGE_GRAFANA}
ports:
- name: grafana-http
containerPort: 3000
volumeMounts:
- mountPath: "/root/go/src/github.com/grafana/grafana/data"
name: grafana-data
- mountPath: "/usr/share/grafana/conf"
name: grafanaconfig
- mountPath: /etc/tls/private
name: grafana-tls
- mountPath: /etc/proxy/secrets
name: secrets
volumes:
- name: grafanaconfig
configMap:
name: grafana-config
- name: secrets
secret:
secretName: grafana-proxy
- name: grafana-tls
secret:
secretName: grafana-tls
- emptyDir: {}
name: grafana-data
In this template, we defined 2 containers in a pod, which are grafana itself and a oauth-proxy for access authorization. We also defined several volumes for grafana
container, including a config maps named grafana-config
for configuration persistence.
By now, the real configuration is still missing in the template. We could follow the default configuration in Grafana to minimize installation difficulty. Add the following content to the template:
- apiVersion: v1
kind: ConfigMap
metadata:
name: grafana-config
namespace: "${NAMESPACE}"
data:
defaults.ini: |-
[...content of the configuration...]
Then, create a new project and upload the template to it.
$ oc create project grafana
$ NAMESPACE=grafana oc process -f grafana.yml | oc create -f -
$ oc rollout status deployment/grafana
Now, the grafana pod should be up and running. We still needs to setup OAuth by adding proper permission to users.
$ oc adm policy add-cluster-role-to-user cluster-reader grafana
Now point your browser to your apps route, and OAuth should be working in front of grafana.
Datasource Provisioning
From Grafana 5.0, datasource could be provisioned by placing YAML files in given directory. This directory could be specified in configuration. With default configration, we could add a config maps named grafana-datasources
, and mount it to grafana-datasources → /usr/share/grafana/datasources
. In the config map, create a pair whose key is datasources.yaml
, and insert datasource parameters. For example, to bind a Prometheues behind an OAuth proxy:
apiVersion: 1
datasources:
- name: "OCP Prometheus"
type: prometheus
access: proxy
url: https://route.to.prometheues.app
basicAuth: false
withCredentials: false
isDefault: true
jsonData:
tlsSkipVerify: true
"httpHeaderName1": "Authorization"
secureJsonData:
"httpHeaderValue1": "Bearer [Prometheues Secret]"
editable: true
where [Prometheues Secret]
could be gained by oc sa get-token prometheus -n openshift-metrics
.
Dashboard Provisioning
Dashboard could be provisioned in Grafana 5 as well. Slightly different from datasource provisioning, we need to create a config map to tell grafana where we place our dashboards, and then another config map which contains real dashboard configrations.
First we create and mount a config map grafana-provisioning-dashboards → /usr/share/grafana/dashboards
, and create a pair named dashboards.yaml
with value:
apiVersion: 1
providers:
- name: 'default'
orgId: 1
folder: ''
type: file
disableDeletion: false
updateIntervalSeconds: 3
options:
path: /var/lib/grafana/dashboards
Then create and mount grafana-dashboards → /var/lib/grafana/dashboards
, where the path of the mountpoint should be changed upon the configuration in grafana-provisioning-dashboards
config map. Finally we could create pairs whose key is any-filename.json
and value is the JSON of the dashboard. Grafana should load it as it is mounted and changed.
Previously OpenShift Origin ↩︎
Preview:
Nice article! Tnx a lot! Just one remark. It is better to attach some kind of real configuration in ConfigMap for newbies.