Exporting Telemetry
Connecting to Section’s /federate Endpoint
Section provides a /federate endpoint from which you can gather live metrics pertaining to your account. The systems underlying this endpoint are Prometheus systems. But the metrics returned are OpenMetrics compatible, and therefore you can use any compatible tooling in order to process them.
In the examples below we will give details that help you process the metrics with Grafana Cloud and Datadog.
To setup remote collection for any of the following 3rd party systems you will need to collect the following information from Section Console:
SECTION_ACCOUNT_ID
is 4 or more digits, as in “1234”. Learn about accounts here.- URL: console.section.io/prometheus/account/
SECTION_ACCOUNT_ID
/federate - Authentication type: Bearer
SECTION_API_TOKEN
: get an API token using instructions here.
Useful Links
- Learn more about Prometheus and OpenMetrics at this primer.
Monitoring with Grafana Cloud
Here we provide a simple example of how to scrape Section metrics into Grafana Cloud. The basic idea is that you need to run a Prometheus agent that regularly scrapes Section’s /federate endpoint to fetch metrics, and writes the results into Grafana Cloud using “remote write”. (At the time of this writing Grafana Cloud has no way to do the scraping itself.)
You may run the Prometheus agent on any infrastructure of your choice. It could be in a docker container on your PC, hosted on a cloud provider, or even hosted at Section. A single agent can be used to scrape all of the metrics for your account.
Obtain the following information from your instance of Grafana Cloud:
GRAFANA_METRICS_INSTANCE_ID
: obtain from Grafana CloudGRAFANA_API_KEY
: obtain from Grafana CloudGRAFANA_METRICS_INSTANCE_REMOTE_WRITE_ENDPOINT
:- Is of the form: https://…/api/prom/push
Useful Links
- Get started with Grafana Cloud
- Learn about how to setup your Prometheus Agent
Example YAML Configuration for a Prometheus Docker Container
The following YAML file configures a docker container that will run Prometheus, scrape Section’s /federate endpoint, and write the results into Grafana Cloud. Replace the following accordingly: SECTION_ACCOUNT_ID
, SECTION_API_TOKEN
, GRAFANA_METRICS_INSTANCE_REMOTE_WRITE_ENDPOINT
, GRAFANA_METRICS_INSTANCE_ID
, GRAFANA_API_KEY
.
# my global config
global:
scrape_interval: 30s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
evaluation_interval: 30s # Evaluate rules every 15 seconds. The default is every 1 minute.
# scrape_timeout is set to the global default (10s).
# Attach these labels to any time series or alerts when communicating with
# external systems (federation, remote storage, Alertmanager).
external_labels:
monitor: 'section-monitor'
# Load rules once and periodically evaluate them according to the global 'evaluation_interval'.
rule_files:
# - "first.rules"
# - "second.rules"
# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
# The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.
- job_name: 'section-federation'
metrics_path: '/prometheus/account/SECTION_ACCOUNT_ID/federate'
params:
'match[]':
- '{__name__=~".+"}'
scheme: 'https'
authorization:
type: Bearer
credentials: SECTION_API_TOKEN
static_configs:
- targets: ['console.section.io']
remote_write:
- url: GRAFANA_METRICS_INSTANCE_REMOTE_WRITE_ENDPOINT
basic_auth:
username: GRAFANA_METRICS_INSTANCE_ID
Password: GRAFANA_API_KEY
The command to run the container is as follows. Replace C:/myfiles
accordingly.
docker run \
-p 9090:9090 \
-v C:/myfiles/prometheus.yml:/etc/prometheus/prometheus.yml \
prom/prometheus
Monitoring with Datadog
Here we provide a simple example of how to scrape Section metrics into Datadog. The basic idea is that you need to run a Datadog Agent that regularly scrapes Section’s /federate endpoint to fetch metrics, and writes the results into your Datadog cloud instance. You may run this agent on any infrastructure of your choice. It could be in a docker container on your PC, hosted on a cloud provider, or even hosted at Section. A single agent can be used to scrape all of the metrics for your account.
Obtain the following information from your instance of Datadog:
DATADOG_API_KEY
: this will appear during the Datadog setup wizard. Or generate one by visiting the API Keys area of Organization Settings.
Useful Links
- Get started with Datadog.
- Specifically, you are using the Datadog Agent, so you will want to learn about that.
Example YAML Configuration for a Datadog Agent Docker Container
The following YAML file defines a docker container that will run the Datadog agent, scrape Section’s /federate endpoint, and write the results into your Datadog cloud instance.
- Replace
SECTION_ACCOUNT_ID
with your Section Account ID, typically of the form1234
. - Note that the
Bearer TOKEN
is not something that you are supposed to replace, just leave it alone.
init_config:
instances:
- namespace: section
openmetrics_endpoint: https://console.section.io/prometheus/account/SECTION_ACCOUNT_ID/federate
auth_token:
reader:
type: file
path: /etc/datadog-agent/conf.d/openmetrics.d/token
pattern: ^(.+)$
writer:
type: header
name: Authorization
value: "Bearer <TOKEN>"
metrics:
- .+:
type: gauge
The token
file should be filled with one line containing your SECTION_API_TOKEN
. Shown below is an obfuscated Section API token.
*********************************************************************4c57
The command to run the container above follows. Note the following:
- You'll need to properly specify the location of the files
datadog/openmetrics.d/conf.yaml
anddatadog/openmetrics.d/token
. In the example below they are specified as being inC:/myfiles
. - And be sure to replace
DATADOG_API_KEY
.
docker run -d --name dd-agent \
-v /var/run/docker.sock:/var/run/docker.sock:ro \
-v /proc/:/host/proc/:ro \
-v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
-v C:/myfiles/datadog/openmetrics.d/conf.yaml:/etc/datadog-agent/conf.d/openmetrics.d/conf.yaml \
-v C:/myfiles/datadog/openmetrics.d/token:/etc/datadog-agent/conf.d/openmetrics.d/token \
-e DD_API_KEY=DATADOG_API_KEY \
-e DD_SITE="datadoghq.com" \
gcr.io/datadoghq/agent:7
If you are having trouble, try adding debugging to the list of command line arguments.
-e DD_LOG_LEVEL=debug \