> ## Documentation Index
> Fetch the complete documentation index at: https://datum.net/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Metrics Export

> The metrics Datum publishes for your project, and how to export them to Grafana Cloud or any Prometheus remote-write endpoint.

> [!NOTE]
>
> **Working with this resource via an AI agent?** Datum publishes a [Metrics Export skill](https://datum.net/docs/agents/skills/metrics-export.md) that teaches agents the canonical patterns for this resource.

Datum collects metrics from the services in your project and keeps them in your project's metrics store. The [Cloud Portal](https://cloud.datum.net) charts them on each resource's pages. To see them next to your other metrics, export them to [Grafana Cloud](https://grafana.com/) or any other endpoint that accepts Prometheus remote write.

## What Datum measures

| Product | What you can measure | Reference |
| - | - | - |
| Application Load Balancer | Requests per second by location and response code, origin response time, and firewall events by outcome and rule | [ALB metrics](https://datum.net/docs/alb/observability.md#metrics) |
| Compute | CPU and memory for each Instance, ready and desired Instances for each workload and placement, and traffic to published workloads | [Compute metrics](https://datum.net/docs/compute/observability.md#metrics) |
| gVPC | Whether each network is ready, and each Instance's private address | [gVPC metrics](https://datum.net/docs/galactic-vpc/observability.md#metrics) |
| Domains and DNS | Whether each zone and record set is serving, and whether each domain is verified | [Domains and DNS metrics](https://datum.net/docs/domain-dns/observability.md#metrics) |
| Connectors | Traffic through the load balancers that route to each Connector | [Connector metrics](https://datum.net/docs/connectors/observability.md#metrics) |

Metrics come in two kinds:

* **Traffic and usage metrics** measure what your resources are doing, such as requests per second or CPU time. Their names start with the component that records them, such as `envoy_` for load balancer traffic, `coraza_` for the firewall, and `datum_compute_instance_` for Instance CPU and memory.
* **Status metrics** report the state of your resources, such as whether a load balancer is programmed or a domain is verified. They follow a consistent pattern, so once you know one, you know them all:

| Pattern | What it reports |
| - | - |
| `<resource>_info` | One series for each resource, with descriptive labels. Use it to count resources or join in labels. |
| `<resource>_status_condition` | `1` when a condition is true and `0` when it's false, with `condition`, `reason`, and `status` labels. |
| `<resource>_status_condition_last_transition_time` | When each condition last changed, as a Unix timestamp. |
| `<resource>_created` | When the resource was created, as a Unix timestamp. |

### Labels on every series

| Label | Meaning |
| - | - |
| `resourcemanager_datumapis_com_project_name` | Your project ID. Include it in every query, because resource names are only unique within a project. |
| `resource_name` | On status metrics, the name of the resource. |
| `label_topology_kubernetes_io_region` or `region` | On traffic and usage metrics, the Datum location, such as `us-west-1`. |

Queries use [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/), which also accepts standard PromQL.

## Export with the Cloud Portal

You can set up an export policy in the Datum Cloud portal. Go to the project of your choice, find the **Metrics** tab, and follow the instructions.

![Grafana Access](https://mintcdn.com/datum-4926dda5/9pdr37W0ZlTfIFim/images/grafana-access.png?fit=max\&auto=format\&n=9pdr37W0ZlTfIFim\&q=85\&s=dd3607d7505fb2a29720e4cb9399d4d6)

Step 1: Provide the Grafana instance URL

![Grafana Finish Setup](https://mintcdn.com/datum-4926dda5/9pdr37W0ZlTfIFim/images/grafana-finish-setup.png?fit=max\&auto=format\&n=9pdr37W0ZlTfIFim\&q=85\&s=f8e5ce9b6a19d6f8897568275ea04556)

Step 2: Configure your export policy

## Export with datumctl

An `ExportPolicy` defines:

* **Sources**, which select the metrics to export. A source with no `metrics` selects everything in your project. A source with a MetricsQL selector exports only the matching series.
* **Sinks**, which send the selected sources to a Prometheus remote-write endpoint.

**1. Store your credentials in a Secret**

Create a Secret with the credentials for your endpoint. For [Grafana Cloud](https://grafana.com/products/cloud/), the username is your instance ID and the password is an access token with permission to write metrics.

```yaml theme={null}
apiVersion: v1
kind: Secret
metadata:
  name: grafana-cloud-credentials
stringData:
  username: "123456"
  password: "glc_..."
```

```bash theme={null}
datumctl apply -f grafana-secret.yaml
```

**2. Create the export policy**

This policy exports only load balancer traffic and firewall metrics. To export everything, remove the `metrics` field from the source.

```yaml theme={null}
apiVersion: telemetry.miloapis.com/v1alpha1
kind: ExportPolicy
metadata:
  name: grafana-cloud-export
spec:
  sources:
    - name: alb-metrics
      metrics:
        metricsql: '{__name__=~"envoy_vhost_vcluster_upstream_rq.*|coraza_envoy_filter_request_events_total"}'
  sinks:
    - name: grafana-cloud
      sources:
        - alb-metrics
      target:
        prometheusRemoteWrite:
          endpoint: https://prometheus-prod-01-prod-us-east-0.grafana.net/api/prom/push
          authentication:
            basicAuth:
              secretRef:
                name: grafana-cloud-credentials
          batch:
            maxSize: 500
            timeout: "5s"
          retry:
            maxAttempts: 3
            backoffDuration: "1s"
```

Replace `endpoint` with your own remote-write URL. Every field under `batch` and `retry` is required.

```bash theme={null}
datumctl diff -f exportpolicy.yaml
datumctl apply -f exportpolicy.yaml
```

**3. Check that it's exporting**

```bash theme={null}
datumctl describe exportpolicy grafana-cloud-export
```

Check each sink's conditions in `status.sinks`. The policy can report `Programmed` while one of its sinks is failing, for example because of wrong credentials.

## Datum dashboards in Grafana Cloud

To add prebuilt Datum dashboards in Grafana Cloud, go to **Dashboards**, select **New dashboard**, then select **Import dashboard**. Load the Datum dashboards using the following Grafana IDs.

### Datum Overview

Grafana ID: **23939**

Provides real-time visibility into your load balancers, including:

* Total HTTP proxies, hostnames, and domains
* Upstream requests per second (RPS) by region
* Global response codes and latency percentiles
* Domain and proxy health status

### Datum Proxy

Grafana ID: **24261**

Offers a real-time view of one load balancer's hostnames and regional traffic, including:

* Regional RPS and upstream response codes
* Latency percentiles (p50, p90, p99)
* Domain and proxy status

For additional help, consult the [Grafana Cloud documentation](https://grafana.com/docs/grafana-cloud/).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.