# Datum Cloud Docs > Learn about Datum's edge cloud: Compute, gVPC, the Application Load Balancer, Connectors, and Domains and DNS. > ## Agent Instructions > Agent home for Datum, with what it is, pricing, and how to act: https://agents.datum.net > If you can only fetch URLs: read https://www.datum.net/docs/llms.txt in full and open the most specific page for the task. Commands, limits, and field paths are on task, operations, and limits pages, not on section overviews. Pages described as "Documentation is coming soon" are placeholders. > If you have a shell or can save files: download https://www.datum.net/docs/llms-full.txt (every page, about 740 KB) once and search it with grep, instead of fetching pages one at a time. > datumctl describes itself: `datumctl api-resources` lists resource types and `datumctl explain ` shows a schema. To act on live resources, use Datum MCP or datumctl. To search the docs, use the Docs MCP server at https://www.datum.net/docs/mcp. # Create an account Source: https://datum.net/docs/account/create-account Create a Datum Cloud account with your Google or GitHub account, or with your email address and a passkey. ## Sign up with Google or GitHub To sign up with Google or GitHub, follow these steps: 1. Go to [https://cloud.datum.net](https://cloud.datum.net). On **Welcome**, click **Create account**. 2. On **Get started**, click **Google** or **GitHub**, and sign in to that account. Your Datum Cloud account uses the email address from your Google or GitHub account. You don't need a passkey, but you can add one later as another way to sign in. For the steps, see [Add a passkey](/docs/account/passkeys#add-a-passkey). ## Sign up with your email and a passkey A passkey lets you sign in with your fingerprint, face, or device PIN. Datum doesn't use passwords, so if you sign up with your email address, your passkey is how you sign in. ### Before you begin To sign up with your email address, you need the following: * A device, browser, or password manager that supports passkeys, such as a recent phone or laptop. * Access to the inbox for your email address. ### Create your account To sign up with your email address and a passkey, follow these steps: **1. Start sign up** Go to [https://cloud.datum.net](https://cloud.datum.net). On **Welcome**, click **Create account**. **2. Enter your email address** On **Get started**, click **Email**, enter your email address, and click **Continue**. The **Check your email** screen opens. **3. Verify your email address** Open the email with the subject "Verify your email address for Datum". It contains a **Verify email** link and a code. Do one of the following: * To keep going on the device you're using, enter the code in **Or enter the code from that email** and click **Continue**. * To create your passkey on the device where you read the email, such as your phone, click **Verify email** on that device. Datum creates your passkey on the device where you finish signing up, so choose the device you want to sign in with. The link and code expire after 60 minutes. **4. Create your passkey** Click **Register passkey** and confirm with your device. Stay on this page until you finish, because the passkey is the only way to sign in to your new account. On **Name your passkey**, keep the suggested name or enter your own, and click **Save**. You can't rename a passkey later, so pick a name that tells you which device it's on. **5. Sign in** On **Your account is ready**, click **Go to sign in**. On **Welcome**, click **Passkey** and confirm with your device. The first time, your device can ask you to confirm twice: once to find your account and once to sign you in. ## Troubleshooting sign up | Problem | What to do | | - | - | | The verification email doesn't arrive. | Check your spam folder. Wait a few minutes, then go back to **Get started** and submit your email address again to get a new email. | | "That code is invalid or has expired. Start over to get a new one." | Check the code for typos. If it has expired, click **Wrong address? Start over**, wait a few minutes, and submit your email address again. | | **Link expired** | The link was already used or is more than 60 minutes old. Click **Start over**. | | "An account with this email already exists. Try signing in instead." | Sign in instead. If you can't use your passkey, click **Recover your account**, or see [Recover your account](/docs/account/recover-account). | | "That didn't go through. Please try again." | Try again. If it keeps happening, contact [support@datum.net](mailto:support@datum.net). | ## What's next * To sign in to your new account, see [Sign in](/docs/account/sign-in). * To add a passkey on another device as a backup, see [Manage passkeys](/docs/account/passkeys). * To learn how organizations and the Owner, Editor and Viewer roles work, see [Account setup](/docs/getting-started/account-setup). # Manage passkeys Source: https://datum.net/docs/account/passkeys Add, name, and remove the passkeys you use to sign in to Datum Cloud, and keep a backup sign-in method. A passkey lets you sign in to Datum Cloud with your fingerprint, face, or device PIN instead of a password. You can add passkeys on several devices and remove the ones you no longer use. You do both on the **Passkeys** page. > [!NOTE] > > If you last signed in more than 10 minutes ago, Datum asks you to confirm it's you before it shows the **Passkeys** page. ## Open the Passkeys page To open the **Passkeys** page, follow these steps: 1. In the Cloud Portal, click your avatar or initials in the top right, and click **Account Settings**. 2. Click the **Security** tab. 3. In **Sign-in Methods**, next to **Passkeys**, click **Add** if you have no passkeys, or **Manage** if you have at least one. 4. If **Confirm it's you** opens, choose one of your sign-in methods, such as **Passkey**, and confirm. The **Passkeys** page opens in a new browser tab. It lists each passkey with its name and the date you added it. Close the tab to return to the portal. ## Add a passkey Each device or password manager holds one passkey for your account. To add a passkey on the device you're using, follow these steps: **1. Start setup** On the **Passkeys** page, click **Add passkey**. **2. Register the passkey** On **Set up passkey**, click **Register passkey**, and confirm with your device. If the device already has a passkey for your account, Datum shows "You already have a passkey for this account on this device." Use a different device instead. **3. Name the passkey** On **Name your passkey**, keep the suggested name or enter your own in **Passkey name**, and click **Save**. The name is optional and can be up to 200 characters. You can't rename a passkey later, so pick a name that tells you which device it's on, such as `Work laptop`. Datum sends you an email with the subject "New passkey added to your Datum account". To add a passkey on another device, such as your phone, sign in to the Cloud Portal on that device and repeat these steps. ## Remove a passkey To remove a passkey, follow these steps: 1. On the **Passkeys** page, click the trash icon next to the passkey. 2. In **Remove this passkey?**, click **Remove passkey**. The passkey stops working for sign-in, and you can't undo this. 3. In **Passkey removed**, choose one of the following: * To sign out every other device that's signed in to your account, click **Sign out other sessions**. Do this if you removed the passkey because you lost the device. * To stay signed in on your other devices, click **Not now**. Datum sends you an email with the subject "A passkey was removed from your Datum account". The passkey stays in your device or password manager until you delete it there. ## Keep a backup sign-in method You can't remove your only sign-in method. If you try, the **Passkeys** page shows "You can't remove your only sign-in method. Add another method first." To keep a way in if you lose a device, do one or both of the following: * Add passkeys on more than one device. * Link your Google or GitHub account: on the **Security** tab, in **Sign-in Methods**, click **Connect** next to Google or GitHub. On **Linked accounts**, under **Available accounts to link**, click the provider and sign in to that account. If you lose access to every sign-in method, see [Recover your account](/docs/account/recover-account). ## If you get an email about a change you didn't make If you get a passkey email that you don't expect, do the following: 1. Open the **Passkeys** page and remove any passkey you don't recognize. 2. In the Cloud Portal, go to **Account Settings** > **Active Sessions** and click the trash icon (**Revoke**) next to any session you don't recognize. 3. Contact [support@datum.net](mailto:support@datum.net). ## What's next * To sign in with a passkey, including one on your phone, see [Sign in](/docs/account/sign-in). * To get back into your account when you can't use any of your passkeys, see [Recover your account](/docs/account/recover-account). # Recover your account Source: https://datum.net/docs/account/recover-account Get back into your Datum Cloud account when you can't use your passkey, by setting up a new passkey from a recovery email. If you can't use your passkey, for example because you lost or reset the device it was on, you can recover your account from your email. Recovery adds a new passkey on the device you choose. Your other passkeys stay on your account until you remove them. ## Recover your account To recover your account, follow these steps: **1. Open the recovery page** In your browser, go to [https://auth.datum.net/id/recover](https://auth.datum.net/id/recover). You also reach this page from **Recover your account** on the sign up screen when your address already has an account, and on the **No sign-in method available** screen. **2. Request a recovery link** On **Recover your account**, enter your email address and click **Send recovery link**. If an account exists for that address, Datum sends you an email with a recovery link and a code. The **Check your email** screen stays open in your browser. **3. Open the link or enter the code** Do one of the following: * To set up the new passkey on another device, such as your phone, open the link in the email on that device. On **Set up a new passkey**, click **Continue**. * To set up the new passkey on the device you're using, enter the code from the email in the **Code** field on **Check your email**, and click **Continue**. The code works only in the browser where you requested it. **4. Register your new passkey** Click **Register passkey** and confirm with your fingerprint, face, or device PIN. On **Name your passkey**, keep the suggested name or enter your own, and click **Save**. You can't rename a passkey later, so pick a name that tells you which device it's on. **5. Sign in** Datum returns you to the sign-in screen with the notice "New passkey added. Sign in with it below." Sign in with your new passkey. ## Limits The link and the code each work once and expire after 1 hour. Use the most recent email, and wait a while before you request another link if you've asked for several in a row. An expired or used link shows **This link is invalid or has expired**. Click **Request a new link** to start again. A wrong or expired code shows "That code is invalid or has expired. Check the email, or start over." Check that you used the code from the most recent email. If it has expired, click **Wrong address? Start over** and request a new link. If you never finished creating your account, you get an email with the subject "Verify your email address for Datum" instead of a recovery email. Follow it to create your passkey. ## After you recover Remove the passkeys from any device you lost, so no one can use them to sign in to your account. For the steps, see [Remove a passkey](/docs/account/passkeys#remove-a-passkey). ## If you still can't get in If the email doesn't arrive, or recovery doesn't work for your account, contact [support@datum.net](mailto:support@datum.net). Support can send you a recovery link. That email contains a link only, with no code, so open it on the device where you want your new passkey. ## What's next * To add a passkey on another device as a backup, see [Manage passkeys](/docs/account/passkeys). * To sign in with your new passkey, see [Sign in](/docs/account/sign-in). # Sign in Source: https://datum.net/docs/account/sign-in Sign in to Datum Cloud with a passkey, or with your Google or GitHub account. To sign in, go to [https://cloud.datum.net](https://cloud.datum.net). The **Welcome** screen lists the ways you can sign in: **Google**, **GitHub**, **Passkey**, and **Email**. A **Last used** badge marks the method you used last in this browser. ## Sign in with a passkey To sign in with a passkey, follow these steps: 1. On **Welcome**, click **Passkey**. 2. Your browser shows its passkey dialog. Choose your passkey and confirm with your fingerprint, face, or device PIN. Datum finds your account from the passkey, so you don't type your email address. In a browser you've signed in with before, the browser can offer your passkey when the page opens, and one confirmation signs you in. In a browser you haven't used with Datum, your device can ask you to confirm twice: the first confirmation finds your account and the second signs you in. ## Use a passkey from your phone If your passkey is on your phone and you're signing in on another computer, follow these steps: 1. On **Welcome**, click **Passkey**. 2. In your browser's passkey dialog, choose the option to use a phone, tablet, or another device. The browser shows a QR code. 3. Scan the QR code with your phone's camera, and confirm on your phone. Turn on Bluetooth on both devices. Your phone uses it to check that it's near the computer, then approves this one sign-in. The passkey stays on your phone. > [!TIP] > > To sign in on this computer without your phone next time, add a passkey on the computer. For the steps, see [Add a passkey](/docs/account/passkeys#add-a-passkey). ## Sign in with your email address To sign in with your email address, follow these steps: 1. On **Welcome**, click **Email**, enter your email address, and click **Continue**. 2. On **Choose how to sign in**, click the method you want to use, such as **Passkey**. If a passkey is your only sign-in method, your browser's passkey dialog can open on its own, and you confirm with your device. Click **Passkey** if it doesn't open. If your only sign-in method is a linked Google or GitHub account, Datum skips **Choose how to sign in** and sends you to Google or GitHub. Datum doesn't email you a sign-in link or code. ## Sign in with Google or GitHub On **Welcome**, click **Google** or **GitHub**, and sign in to that account if it asks you to. Datum signs you in to the Datum Cloud account that's linked to it. ## Sign out To sign out of the Cloud Portal, click your avatar or initials in the top right, and click **Log Out**. To sign out on other devices, go to **Account Settings** > **Active Sessions**, and click the trash icon (**Revoke**) next to each session you want to end. You can't revoke the session you're using. ## Troubleshooting sign in If passkey sign-in fails, Datum shows one of the following messages on the sign-in screen: | Message | What to do | | - | - | | "Passkey sign-in was cancelled, or no passkey for this account is available on this device. Make sure you're using a device where you set up your passkey, then try again." | Try again, and choose the passkey for your Datum account. If your passkey is on another device, see [Use a passkey from your phone](#use-a-passkey-from-your-phone). | | "This device can't use a passkey to sign in. Try another device or a different sign-in method." | Use a device or browser that supports passkeys, or sign in with Google or GitHub if you linked one. | | "Passkey sign-in couldn't be completed for security reasons. Please contact support if this continues." | Try again. If it keeps happening, contact [support@datum.net](mailto:support@datum.net). | | "Incorrect credentials. Please try again." | The passkey you chose is no longer on your Datum account, for example because you removed it. Choose another passkey. If you don't have one, see [Recover your account](/docs/account/recover-account). | | "The passkey verification failed. Please try again." | Try again. | If you can't use any of your passkeys, for example because you lost the device they were on, go to [https://auth.datum.net/id/recover](https://auth.datum.net/id/recover) to set up a new passkey. For the steps, see [Recover your account](/docs/account/recover-account). ## What's next * To add a passkey on another device or link Google or GitHub as a backup, see [Manage passkeys](/docs/account/passkeys). * To get back into your account when you can't use your passkey, see [Recover your account](/docs/account/recover-account). # Datum MCP Source: https://datum.net/docs/agents/datum-mcp Connect Claude, Cursor, and other AI tools to your Datum projects with Datum's official MCP server: manage resources and run Datum's own diagnostic tools. Datum offers two [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) servers for AI tools: | Server | What it does | Where it runs | Sign-in | | - | - | - | - | | [Datum MCP](#install) | Reads and changes your resources, and diagnoses problems with them. | On your machine, as the `datum-mcp` binary | Your Datum account | | [Datum Docs MCP](#datum-docs-mcp) | Searches and reads Datum's documentation. | Hosted by Datum at `https://www.datum.net/docs/mcp` | None | Most agents benefit from both: Docs MCP to learn how Datum works, and Datum MCP to act on your projects. Datum's official [MCP server](https://github.com/datum-cloud/datum-mcp) connects AI tools to your Datum projects. It gives an agent tools to read and change your resources, and to diagnose problems with them, with the same permissions as your own account. Datum MCP covers: * **Organizations, projects, and users.** * **Networking:** domains, load balancers and their routes, gateways, traffic protection, DNS zones and records, and gVPC networks. * **Compute** workloads and Instances. * **IP address management, IAM, billing, and services.** * **Activity**, to search your project's audit history. * **Any other Datum resource**, through a generic escape hatch. For load balancers, networks, Compute, DNS, and activity, Datum MCP also gives the agent each service's own diagnostic tools and guides. These are the same tools that [Patch](/docs/datumctl/ai/assistant), Datum's own assistant, uses. > [!NOTE] > > Prefer the terminal? `datumctl assistant` answers questions in natural language without a separate MCP client. See [AI assistant](/docs/datumctl/ai/assistant). For how the CLI docs frame this server, see [Connecting an AI client with the MCP server](/docs/datumctl/ai/mcp-server). ## Install > [tab] Homebrew (macOS / Linux) > > ```bash theme={null} > brew tap datum-cloud/tap > brew install datum-mcp > ``` > > To upgrade later, run `brew upgrade datum-mcp`. > [tab] Install script (macOS / Linux) > > ```bash theme={null} > curl -fsSL https://github.com/datum-cloud/datum-mcp/releases/latest/download/install.sh | sh > ``` > [tab] Download > > Download a binary for macOS, Linux, or Windows, on amd64 or arm64, from the [latest release](https://github.com/datum-cloud/datum-mcp/releases/latest). Rename it to `datum-mcp` (`datum-mcp.exe` on Windows), and put it on your `PATH`. ## Add it to your AI client > [tab] Claude Code > > Install the `datum-cloud` plugin. It starts Datum MCP and the [Datum Docs MCP](#datum-docs-mcp) server for you, and adds skills for signing in, CI service accounts and handing off to Patch: > > ```bash theme={null} > claude plugin marketplace add datum-cloud/claude-code-plugins > claude plugin install datum-cloud@datum-claude-code-plugins > ``` > > To add only the MCP server instead, run `claude mcp add --scope user datum-mcp -- datum-mcp`. `--scope user` makes it available in every project. Leave it off to add it to the current project only. > [tab] Claude Desktop > > Add this to your MCP configuration: > > ```json theme={null} > { > "datum-mcp": { > "command": "datum-mcp", > "args": [] > } > } > ``` > [tab] Cursor > > [Add Datum MCP to Cursor](https://cursor.com/en-US/install-mcp?name=datum-mcp\&config=eyJ0eXBlIjoic3RkaW8iLCJlbnYiOnt9LCJjb21tYW5kIjoiL3Vzci9sb2NhbC9iaW4vZGF0dW0tbWNwICJ9) with one click. On Windows, point your client's configuration at the full path, such as `"command": "C:\\tools\\datum-mcp.exe"`. ## Sign in The first time an agent uses Datum MCP, the server opens your browser to sign in with OAuth 2.1 (PKCE), and stores the credentials in your system keychain. After that, it refreshes the token automatically. To run without a browser or keychain, such as in CI or an agent-hosted deployment, set both `DATUM_TOKEN` and `DATUM_API_HOSTNAME` (for example, `api.datum.net`). With a [service account](/docs/projects/service-accounts)'s token, the server works in that service account's own project automatically. ## Get started in a conversation Ask your agent to set up its context first. It checks what's already selected, then picks an organization and project: 1. `context` shows the current organization and project. 2. `organizations` lists your organizations and selects one. 3. `projects` lists the organization's projects and selects one. The `onboard-to-project` prompt does all three for you. After that, ask for what you need, such as "why is my load balancer not serving?" or "create a DNS zone for example.com". ## Toolsets To keep the agent's tool list short, Datum MCP loads only its core tools and the `alb`, `dns`, and `network` toolsets when it starts. The agent loads others, such as `compute`, `vpc`, `activity`, `ipam`, `iam`, `billing`, `services`, `search`, or `resource`, with the `toolsets` tool when a task needs them, without a restart. To see every toolset, run `datum-mcp --list-toolsets`. To change what loads at startup, set `DATUM_MCP_ENABLE_TOOLSETS`, such as `dns,compute` or `all`. To rule a toolset out entirely, set `DATUM_MCP_DISABLE_TOOLSETS`, such as `billing,iam`. ## Prompts Datum MCP includes ready-made prompts for common tasks, such as: * `deploy-http-proxy` — put a load balancer in front of a backend. * `configure-dns` — create a DNS zone for a domain. * `alb-not-serving`, `alb-certificate-not-issued`, `alb-hostname-not-working`, and other diagnostic prompts for load balancers, networks, Compute, DNS, and activity. For every tool and prompt, see the [tool reference](https://github.com/datum-cloud/datum-mcp/blob/main/docs/TOOLS.md). ## Hand off to Patch For an open-ended diagnostic question, or a risky change that touches several resources, Patch can be a better fit than the MCP server's resource tools. The `datum-cloud` Claude Code plugin includes a skill that teaches an agent when to hand a task to Patch through `datumctl assistant`. For other agents, copy the [skill](https://github.com/datum-cloud/datum-mcp/tree/main/.claude/skills/datum-cloud-assistant) from the Datum MCP repository into the agent's skills directory. ## Run over HTTP By default, your AI client starts Datum MCP and talks to it over standard input and output. To serve it over streamable HTTP instead, so a client connects by URL, run the following command: ```bash theme={null} datum-mcp --mode http --host localhost --port 9000 ``` > [!WARNING] > > The HTTP mode has no authentication of its own. Every request runs with this process's Datum credentials, so it listens only on localhost unless you add `--allow-non-loopback`. Only do that behind your own access control. ## Datum Docs MCP Datum also hosts a read-only MCP server for its documentation, at `https://www.datum.net/docs/mcp`. It needs no installation and no sign-in, and it only reads published docs. Add it to an agent so that it can answer questions about Datum from the current documentation instead of from what it remembers. It provides three tools: * `search_datum_cloud_docs` — search the documentation, including code examples, API references, and guides. * `query_docs_filesystem_datum_cloud_docs` — read pages and Datum's OpenAPI specs through a read-only, virtual filesystem. * `submit_feedback` — report a page that's wrong, outdated, or confusing to the docs team. > [tab] Claude Code > > ```bash theme={null} > claude mcp add --transport http datum-cloud-docs https://www.datum.net/docs/mcp > ``` > [tab] Claude Desktop > > In **Settings** > **Connectors**, add a custom connector with the URL `https://www.datum.net/docs/mcp`. > [tab] Cursor > > Add this to `.cursor/mcp.json`: > > ```json theme={null} > { > "mcpServers": { > "datum-cloud-docs": { > "url": "https://www.datum.net/docs/mcp" > } > } > } > ``` > [tab] VS Code > > Add this to `.vscode/mcp.json`: > > ```json theme={null} > { > "servers": { > "datum-cloud-docs": { > "type": "http", > "url": "https://www.datum.net/docs/mcp" > } > } > } > ``` > [tab] Codex > > Add this to `~/.codex/config.toml`: > > ```toml theme={null} > [mcp_servers.datum-cloud-docs] > url = "https://www.datum.net/docs/mcp" > ``` For other clients, use the streamable HTTP transport with the URL `https://www.datum.net/docs/mcp`. To fetch the docs without MCP, see [llms.txt](/docs/agents/llms-txt). # llms.txt Source: https://datum.net/docs/agents/llms-txt Give an AI agent Datum's documentation in one fetch: llms.txt for an index of every page, llms-full.txt for the full text, and a Markdown version of every page. [`llms.txt`](https://llmstxt.org/) is a convention for giving AI agents and LLMs a map of a site in plain Markdown. Instead of crawling the docs, an agent fetches one file and gets a list of every page, with a one-line description of each, linked to a clean Markdown version of the page. Datum publishes three forms of its documentation for agents: | URL | What it is | Use it when | | - | - | - | | [`www.datum.net/docs/llms.txt`](https://www.datum.net/docs/llms.txt) | An index of every docs page, with its title, a one-line description, and a link to its Markdown version. | An agent needs to find the right page for a task. This is the best place to start. | | [`www.datum.net/docs/llms-full.txt`](https://www.datum.net/docs/llms-full.txt) | The full text of every docs page in one file. | An agent can save and search files, such as a coding agent with a shell. Download it once and search it. At about 740 KB, it's too large for a web fetch tool, so other agents should use `llms.txt`. | | Any page URL with `.md` added | The Markdown version of one page, such as [`www.datum.net/docs/alb/overview.md`](https://www.datum.net/docs/alb/overview.md). | An agent needs one page without the site's navigation and layout. | All three are generated from the docs on every deploy, so they always match the live site. Every Markdown page, `llms.txt`, and `llms-full.txt` starts with a short **Agent Instructions** block that tells an agent where to start and which of these to use. For an agent-first overview of Datum, start at [agents.datum.net](https://agents.datum.net/): what Datum is, when to use it, pricing, and how to act. Datum also publishes [`www.datum.net/llms.txt`](https://www.datum.net/llms.txt), an index of the Datum website. ## Use it from an agent Many agents look for `llms.txt` on their own. To point an agent at it explicitly, add an instruction like the following to your prompt, project instructions, or agent configuration: ```text theme={null} Before answering questions about Datum, fetch https://www.datum.net/docs/llms.txt and read it in full, not as a summary. Use it to find the most specific page for the task, then fetch that page's .md URL. ``` To give an agent a single page, use the **Copy page** or **Open in Claude** menu at the top of any docs page, or add `.md` to the page's URL. To let an agent search the docs instead of fetching files, add the hosted [Datum Docs MCP](/docs/agents/datum-mcp#datum-docs-mcp) server, at `https://www.datum.net/docs/mcp`. ## Choose the right tool `llms.txt` gives an agent knowledge about Datum. To let an agent work with your resources, use one of these as well: * [Skills](/docs/agents/skills/index) — procedural knowledge an agent loads on demand, such as how to create a load balancer or a DNS record correctly. * [Datum MCP](/docs/agents/datum-mcp) — live, authenticated access to your Datum resources from an MCP client. * [`datumctl`](/docs/datumctl/overview) — the CLI, which agents can run directly, with [machine-readable output](/docs/datumctl/output-and-scripting). # Agents Overview Source: https://datum.net/docs/agents/overview How AI agents work with Datum Cloud: Skills, Datum MCP, llms.txt, datumctl, and Patch, Datum's own assistant. Datum is built for agents as first-class users. Rather than wrapping an API in a chat interface, Datum exposes the same primitives people use, such as YAML manifests, `datumctl` commands, and [Kubernetes-style resources](/docs/platform/kubernetes), in ways that agents can reason about reliably. This section covers the surfaces agents use to work with Datum. ## Set up Claude Code Install `datumctl` and `datum-mcp`, then add Datum's Claude Code plugins in one command: ```bash theme={null} brew install datum-cloud/tap/datumctl datum-cloud/tap/datum-mcp claude plugin marketplace add datum-cloud/claude-code-plugins && for p in datum-cloud datum-networking datum-dns datum-compute milo-observe; do claude plugin install "$p@datum-claude-code-plugins" || break; done ``` Then run `claude` and ask it to sign you in to Datum Cloud. The plugins start [Datum MCP](/docs/agents/datum-mcp) and the [Datum Docs MCP](/docs/agents/datum-mcp#datum-docs-mcp) server, and add the [skills](/docs/agents/skills/index) for load balancers, networks, DNS, Compute, and metrics. Running the command again is safe. Without Homebrew, see [Install](/docs/agents/datum-mcp#install) for other ways to get `datum-mcp`. ## Agent surfaces > [card] [Skills](/docs/agents/skills/index) > > Self-contained instructions agents load on demand. Install once, and agents know the canonical way to work with each Datum resource. > [card] [Datum MCP](/docs/agents/datum-mcp) > > Model Context Protocol servers: one for live, authenticated access to your resources, and a hosted one for searching the docs. > [card] [llms.txt](/docs/agents/llms-txt) > > An index of every docs page, plus the full text and a Markdown version of each page, for agents to fetch. > [card] [datumctl](/docs/datumctl/overview) > > The CLI. Agents run it directly for any platform action, with machine-readable output. > [card] [Patch](/docs/datumctl/ai/assistant) > > Datum's own assistant. Ask it in plain language, or hand it a diagnostic question, with `datumctl assistant`. ## Which one should I use? | If your agent needs to… | Use | | - | - | | Know *how* to do a Datum task, such as attaching a domain to a load balancer | [Skills](/docs/agents/skills/index) | | Read or change live Datum resources during a task | [Datum MCP](/docs/agents/datum-mcp) | | Find the right documentation | [Datum Docs MCP](/docs/agents/datum-mcp#datum-docs-mcp) or [llms.txt](/docs/agents/llms-txt) | | Run commands directly, such as in a script or CI | [`datumctl`](/docs/datumctl/overview) | | Diagnose an open-ended problem, such as why a workload isn't available | [Patch](/docs/datumctl/ai/assistant) | These surfaces layer together. A typical agent session loads one or two relevant Skills, calls Datum MCP or `datumctl` to act, and fetches pages through `llms.txt` when it needs deeper context. For an open-ended diagnostic question, or a risky change across several resources, it can hand the task to Patch, which uses the same diagnostic tools with Datum's own guidance. To get started, see [Skills](/docs/agents/skills/index). # Application Load Balancer skill Source: https://datum.net/docs/agents/skills/alb Teach agents to create and manage Datum Application Load Balancers with datumctl — hostnames, routes and origins, traffic protection, and the policies the CLI does not cover. The Application Load Balancer skill teaches agents to work with an [Application Load Balancer](/docs/alb/overview) as a product rather than as the objects it is stored in: create one, get the hostname to point DNS at, attach custom hostnames, add routes and the origins behind them, and configure traffic protection and authentication — all through `datumctl alb`. It also covers the policies the CLI does not wrap, for when you need WAF rule exclusions, rate limiting, circuit breaking or OIDC: `TrafficProtectionPolicy`, `SecurityPolicy` and `BackendTrafficPolicy`, attached via `targetRefs`. The skill ships in the `datum-networking` Claude Code plugin, alongside procedures for diagnosing a load balancer that isn't serving and for creating networks, using the same diagnosis Datum's own assistant runs. ## Install ```bash Claude Code theme={null} /plugin marketplace add datum-cloud/claude-code-plugins /plugin install datum-networking@datum-claude-code-plugins ``` Installing it also installs the `datum-cloud` base plugin, which runs [Datum MCP](/docs/agents/datum-mcp). For other agents, copy the [skill folder](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin/skills/alb) into your agent's skills directory. > [!NOTE] > > Once installed, the skill activates automatically when you ask an agent to work with Application Load Balancer on Datum. > [!WARNING] > > The skill teaches the `datumctl alb` commands, which come from a plugin that is not installed by default. Install it before an agent follows this skill, or every command it runs will fail: > > ```bash theme={null} > datumctl plugin install alb > datumctl alb version > ``` ## Capabilities * Create a load balancer and get the generated hostname to point DNS at * Attach custom hostnames and follow their verification, DNS and certificate progress * Add routes and the origins behind them, by URL or by network service * Split a route across several origins by weight, drain an origin, and set the load balancing algorithm and passive health checks * Configure traffic protection (WAF), request headers and basic authentication * Read access logs for one load balancer * Attach the advanced policies the CLI does not cover: WAF rule exclusions, rate limiting, circuit breaking, OIDC * Preview every change before applying it It also teaches the three ways this API's status misleads a naive reader — conditions that report on several hostnames without naming one, a condition that is true when something is wrong, and two reasons set true to mean "does not apply" — so an agent does not report a problem that is not there, or miss one that is. It does not cover metrics. There is no command for request rates, latency percentiles or error rates over time, so the skill sends people to the Metrics tab for that load balancer in the cloud portal instead of guessing from access logs. ## How it relates to other Datum surfaces * **Product docs:** [Application Load Balancer](/docs/alb/overview) — the human-facing reference for what an ALB is made of * **Datum MCP:** [Datum MCP](/docs/agents/datum-mcp) — for agents that need to read and write Datum resources during a session * **Datum's own assistant** — a separate surface with read-only Application Load Balancer tools of its own, entitled through the platform catalog. It answers questions about a load balancer in the portal without anything being installed; this skill is for driving `datumctl` yourself * **`datumctl`:** [datumctl overview](/docs/datumctl/overview) — the `datumctl alb` commands the skill teaches agents to use, plus `apply`, `diff` and `delete` for the advanced policies ## View the full skill The canonical SKILL.md lives in the `datum-networking` plugin in the [datum-cloud/network-services-operator](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin) repo: > [card] [On GitHub](https://github.com/datum-cloud/network-services-operator/blob/main/claude-plugin/skills/alb/SKILL.md) > > Browse the skill source, including frontmatter and full instructions. > [card] [Raw SKILL.md](https://raw.githubusercontent.com/datum-cloud/network-services-operator/main/claude-plugin/skills/alb/SKILL.md) > > Direct URL for agents that fetch skills by URL. # Client Traffic skill Source: https://datum.net/docs/agents/skills/client-traffic Not available yet. Teach agents to configure TLS termination, HTTP/3, connection limits, and client IP detection on Datum Cloud gateways. > [!WARNING] > > **Not available yet.** This skill configures `ClientTrafficPolicy`, which you can't create in Datum projects yet. Until you can, don't use it to change TLS, HTTP version, timeout, or client IP settings. The Client Traffic skill helps agents manage how Datum Cloud edge gateways accept and handle incoming client connections. It covers [Application Load Balancer](/docs/alb/overview) gateway-level settings — TLS termination, mutual TLS (mTLS), HTTP/2 and HTTP/3 (QUIC), connection timeouts and limits, and real client IP detection — all configured by attaching a `ClientTrafficPolicy` to a Gateway listener. The skill ships in the `datum-networking` Claude Code plugin, alongside procedures for diagnosing a load balancer that isn't serving and for creating networks, using the same diagnosis Datum's own assistant runs. ## Install ```bash Claude Code theme={null} /plugin marketplace add datum-cloud/claude-code-plugins /plugin install datum-networking@datum-claude-code-plugins ``` Installing it also installs the `datum-cloud` base plugin, which runs [Datum MCP](/docs/agents/datum-mcp). For other agents, copy the [skill folder](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin/skills/client-traffic) into your agent's skills directory. > [!NOTE] > > Once installed, the skill activates automatically when you ask an agent to configure client traffic settings on Datum. No further configuration needed. ## Capabilities * Configure TLS termination: minimum/maximum TLS version, cipher suites, ALPN protocols * Require and validate client certificates (mutual TLS / mTLS) * Enable HTTP/2 and HTTP/3 (QUIC) on gateway listeners * Set client-facing connection timeouts and idle timeouts * Enforce maximum concurrent connections and per-connection request limits * Detect real client IP from X-Forwarded-For headers or proxy protocol * Scope policies to a specific listener via `sectionName` * List, inspect, and update policies across a project * Preview changes before applying with `datumctl diff` * Delete policies safely * Check permissions before acting ## How it relates to other Datum surfaces * **Product docs:** [Application Load Balancer](/docs/alb/overview) — the human-facing reference for gateways and edge policies * **Datum MCP:** [Datum MCP](/docs/agents/datum-mcp) — for agents that need live read/write access to gateway resources during a session * **`datumctl`:** [datumctl overview](/docs/datumctl/overview) — `get`, `describe`, `apply`, `diff`, `delete` commands the skill teaches agents to use ## View the full skill The canonical SKILL.md lives in the `datum-networking` plugin in the [datum-cloud/network-services-operator](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin) repo: > [card] [On GitHub](https://github.com/datum-cloud/network-services-operator/blob/main/claude-plugin/skills/client-traffic/SKILL.md) > > Browse the skill source, including frontmatter and full instructions. > [card] [Raw SKILL.md](https://raw.githubusercontent.com/datum-cloud/network-services-operator/main/claude-plugin/skills/client-traffic/SKILL.md) > > Direct URL for agents that fetch skills by URL. # DNS skill Source: https://datum.net/docs/agents/skills/dns Teach agents to create and troubleshoot DNS zones and records in Datum Cloud, and to move a domain in from another provider. The DNS skills help agents work with [Authoritative DNS](/docs/domain-dns/dns) zones and records in Datum Cloud. They add records, find out why a domain isn't resolving, and move a domain in from another DNS provider. The troubleshooting steps are the same ones [Patch](/docs/datumctl/ai/assistant), Datum's own assistant, follows. Agents read live zone and record status through [Datum MCP](/docs/agents/datum-mcp). The skills ship in the `datum-dns` Claude Code plugin. Installing it also installs the `datum-cloud` plugin, which runs Datum MCP. Install the `datum-mcp` binary first; see [Install](/docs/agents/datum-mcp#install). ## Install ```bash Claude Code theme={null} /plugin marketplace add datum-cloud/claude-code-plugins /plugin install datum-dns@datum-claude-code-plugins ``` For other agents, copy the [skill folders](https://github.com/datum-cloud/dns-operator/tree/main/claude-plugin/skills) into your agent's skills directory, and connect [Datum MCP](/docs/agents/datum-mcp) so the agent can read zone and record status. > [!NOTE] > > Once installed, the skills activate automatically when you ask an agent to work with DNS on Datum. No further configuration needed. ## Capabilities * Find out why a domain isn't resolving, checking domain verification, the zone, registrar delegation and then individual records, in that order * Name exactly which nameservers are missing at the registrar * Explain why a record isn't live, or reports `NotOwner` or `Conflict`, and what has to change * Explain why a record managed by a Gateway, a connector or Datum itself won't stay edited, and where the real change belongs * Add a record, checking first that no other record set already holds the name, and write it only after you approve it * Move a domain in from another provider with `/datum-dns:move-zone`. It checks [DNSSEC](/docs/domain-dns/dns#dnssec) before the nameserver cutover, previews the import, and asks before every write * Run the [`datumctl dns`](/docs/domain-dns/operations) commands to create, list, change, export and delete zones and records ## How it relates to other Datum surfaces * **Product docs:** [Authoritative DNS](/docs/domain-dns/dns) — the human-facing reference for public DNS zones and records * **Datum MCP:** [Datum MCP](/docs/agents/datum-mcp) — the live DNS tools the skills use to read zone and record status * **`datumctl`:** the [`datumctl dns` plugin](/docs/domain-dns/operations) — the commands the skills run for changes you make from a terminal ## View the full skill The skills live in the `datum-dns` plugin in the [datum-cloud/dns-operator](https://github.com/datum-cloud/dns-operator/tree/main/claude-plugin) repo: > [card] [On GitHub](https://github.com/datum-cloud/dns-operator/tree/main/claude-plugin) > > Browse the plugin, its skills, and the zone move command. > [card] [Troubleshooting procedures](https://github.com/datum-cloud/dns-operator/tree/main/docs/agent/skills) > > The step-by-step procedures the skills load through Datum MCP. # Domains skill Source: https://datum.net/docs/agents/skills/domains Teach agents to attach and verify domains in Datum Cloud projects using datumctl. The Domains skill helps agents manage [domain](/docs/domain-dns/domains) resources in Datum Cloud — attaching domains to projects, retrieving verification tokens (DNS record or HTTP token), and applying changes idempotently. It teaches agents the correct workflow: apply the manifest, then check `status.verification` for the challenge before confirming ownership. The skill ships in the `datum-networking` Claude Code plugin, alongside procedures for diagnosing a load balancer that isn't serving and for creating networks, using the same diagnosis Datum's own assistant runs. ## Install ```bash Claude Code theme={null} /plugin marketplace add datum-cloud/claude-code-plugins /plugin install datum-networking@datum-claude-code-plugins ``` Installing it also installs the `datum-cloud` base plugin, which runs [Datum MCP](/docs/agents/datum-mcp). For other agents, copy the [skill folder](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin/skills/domains) into your agent's skills directory. > [!NOTE] > > Once installed, the skill activates automatically when you ask an agent to work with Domains on Datum. No further configuration needed. ## Capabilities * List and describe domains in a project * Attach domains to services via YAML manifests * Apply changes idempotently * Preview changes before applying with `datumctl diff` * Delete domains * Retrieve domain verification tokens (`status.verification.dnsRecord` or `status.verification.httpToken`) * Check permissions before acting ## How it relates to other Datum surfaces * **Product docs:** [Domains](/docs/domain-dns/domains) — the human-facing reference for domain management * **Datum MCP:** [Datum MCP](/docs/agents/datum-mcp) — for agents that need live read/write access to domain resources during a session * **`datumctl`:** [datumctl overview](/docs/datumctl/overview) — `get`, `describe`, `apply`, `diff`, `delete` commands the skill teaches agents to use ## View the full skill The canonical SKILL.md lives in the `datum-networking` plugin in the [datum-cloud/network-services-operator](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin) repo: > [card] [On GitHub](https://github.com/datum-cloud/network-services-operator/blob/main/claude-plugin/skills/domains/SKILL.md) > > Browse the skill source, including frontmatter and full instructions. > [card] [Raw SKILL.md](https://raw.githubusercontent.com/datum-cloud/network-services-operator/main/claude-plugin/skills/domains/SKILL.md) > > Direct URL for agents that fetch skills by URL. # HTTPRoute skill Source: https://datum.net/docs/agents/skills/httproute Teach agents to configure HTTP routing rules — path matching, traffic splitting, redirects, rewrites, and header manipulation on Datum Cloud. The HTTPRoute skill helps agents configure HTTP routing rules in Datum Cloud. It covers [Application Load Balancer](/docs/alb/overview) route definitions — matching requests by path, headers, method, and query parameters, then forwarding, splitting, redirecting, rewriting, or mirroring traffic using `HTTPRoute` resources attached to a Gateway. The skill ships in the `datum-networking` Claude Code plugin, alongside procedures for diagnosing a load balancer that isn't serving and for creating networks, using the same diagnosis Datum's own assistant runs. ## Install ```bash Claude Code theme={null} /plugin marketplace add datum-cloud/claude-code-plugins /plugin install datum-networking@datum-claude-code-plugins ``` Installing it also installs the `datum-cloud` base plugin, which runs [Datum MCP](/docs/agents/datum-mcp). For other agents, copy the [skill folder](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin/skills/httproute) into your agent's skills directory. > [!NOTE] > > Once installed, the skill activates automatically when you ask an agent to configure HTTP routing on Datum. No further configuration needed. ## Capabilities * Route traffic by path (exact, prefix, regex), HTTP method, headers, and query parameters * Split traffic across multiple backends by weight (canary, A/B, blue-green deployments) * Redirect HTTP to HTTPS and issue permanent or temporary redirects * Rewrite URL paths and hostnames before forwarding to backends * Add, set, or remove request and response headers per route rule * Mirror (shadow) a percentage of traffic to a secondary backend without affecting responses * Configure per-route request and backend timeouts * List, inspect, and update routes across a project * Preview changes before applying with `datumctl diff` * Delete routes safely * Check permissions before acting ## How it relates to other Datum surfaces * **Product docs:** [Application Load Balancer](/docs/alb/overview) — the human-facing reference for gateways and routing * **Datum MCP:** [Datum MCP](/docs/agents/datum-mcp) — for agents that need live read/write access to HTTPRoute resources during a session * **`datumctl`:** [datumctl overview](/docs/datumctl/overview) — `get`, `describe`, `apply`, `diff`, `delete` commands the skill teaches agents to use ## View the full skill The canonical SKILL.md lives in the `datum-networking` plugin in the [datum-cloud/network-services-operator](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin) repo: > [card] [On GitHub](https://github.com/datum-cloud/network-services-operator/blob/main/claude-plugin/skills/httproute/SKILL.md) > > Browse the skill source, including frontmatter and full instructions. > [card] [Raw SKILL.md](https://raw.githubusercontent.com/datum-cloud/network-services-operator/main/claude-plugin/skills/httproute/SKILL.md) > > Direct URL for agents that fetch skills by URL. # Overview Source: https://datum.net/docs/agents/skills/index Self-contained instructions that teach AI agents how to perform Datum Cloud tasks. Skills are folders containing a `SKILL.md` file with metadata and step-by-step instructions. Agents discover and load them on demand, with no scaffolding or config files. Datum's official skills cover the most common operations across the [Application Load Balancer](/docs/alb/overview), [gVPC](/docs/galactic-vpc/overview) networks, [authoritative DNS](/docs/domain-dns/dns), [domains](/docs/domain-dns/domains), and [metrics export](/docs/operations/metrics-export), with more on the way. These skills work with any agent that supports Skills, including Claude, Codex, Cursor, Pi, OpenCode, and Mistral Vibe. ## Install Datum's skills live at **[github.com/datum-cloud/skills](https://github.com/datum-cloud/skills)**. Install for your agent of choice: ```bash Claude Code theme={null} claude plugin marketplace add datum-cloud/claude-code-plugins && for p in datum-cloud datum-networking datum-dns datum-compute milo-observe; do claude plugin install "$p@datum-claude-code-plugins" || break; done ``` ```bash npx skills theme={null} npx skills add https://github.com/datum-cloud/skills ``` ```bash Cursor theme={null} # Settings → Rules → Add Rule → Remote Rule (GitHub) # Repository: datum-cloud/skills ``` In Claude Code, the `datum-cloud` plugin also starts [Datum MCP](/docs/agents/datum-mcp) and the [Datum Docs MCP](/docs/agents/datum-mcp#datum-docs-mcp) server for you, and adds skills to sign in and pick a project, set up a [service account](/docs/projects/service-accounts) for CI, and hand a question to [Patch](/docs/datumctl/ai/assistant). Install the `datum-mcp` binary first; see [Install](/docs/agents/datum-mcp#install). The command also installs the plugins for each Datum product, and is safe to run again. To pick plugins yourself, browse the marketplace with `/plugin` inside Claude Code. ## Available skills | Skill | Covers | Resources | | - | - | - | | [Application Load Balancer](/docs/agents/skills/alb) | Load balancers, hostnames, routes and origins, WAF, auth. Ships in the `datum-networking` Claude Code plugin | HTTPProxy, Domain, TrafficProtectionPolicy, SecurityPolicy, BackendTrafficPolicy | | [Client Traffic](/docs/agents/skills/client-traffic) (not available yet) | TLS termination, HTTP/3, connection limits. Ships in the `datum-networking` Claude Code plugin | ClientTrafficPolicy, Gateway | | [DNS](/docs/agents/skills/dns) | Zones and records, troubleshooting, and moving a domain in. Ships in the `datum-dns` Claude Code plugin | DNSZone, DNSRecordSet, DNSZoneClass | | [Domains](/docs/agents/skills/domains) | Domain attachment and verification. Ships in the `datum-networking` Claude Code plugin | Domain | | [HTTPRoute](/docs/agents/skills/httproute) | Path routing, traffic splitting, redirects, rewrites. Ships in the `datum-networking` Claude Code plugin | HTTPRoute, Gateway | | [Metrics Export](/docs/agents/skills/metrics-export) | Prometheus remote write pipelines. Ships in the `milo-observe` Claude Code plugin | ExportPolicy | | [Networks](/docs/agents/skills/networks) | gVPC networks, network services, and network interfaces. Ships in the `datum-networking` Claude Code plugin | Network, NetworkService, NetworkInterface | ## How skills work alongside Datum MCP and datumctl Skills tell the agent *how* to think about a Datum resource — the canonical patterns, the right flags, the order of operations. [Datum MCP](/docs/agents/datum-mcp) and `datumctl` are *how* the agent acts. A skill might instruct "always preview HTTPProxy changes with `--dry-run` before applying" — the agent then uses `datumctl` to actually run the command. For open-ended diagnostic questions, an agent can also hand off to [Patch](/docs/datumctl/ai/assistant), Datum's own assistant. See [Agents Overview](/docs/agents/overview) for the full picture. ## Contributing Found something missing or wrong? Skills are open-source (Apache-2.0) and accept community contributions. See the [contributing guide](https://github.com/datum-cloud/skills/blob/main/CONTRIBUTING.md) or open an issue at [github.com/datum-cloud/skills/issues](https://github.com/datum-cloud/skills/issues). # Metrics Export skill Source: https://datum.net/docs/agents/skills/metrics-export Teach agents to configure Prometheus remote write pipelines in Datum Cloud using ExportPolicy resources. The Metrics Export skill helps agents configure [metrics export](/docs/operations/metrics-export) pipelines in Datum Cloud. It defines named sources, each with a [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/) selector, and sinks (Prometheus remote write endpoints) that ship project metrics to external observability platforms such as [Grafana Cloud](https://grafana.com/products/cloud/). It teaches agents the manifest structure the API accepts, how to store credentials in a Secret, and how to check each sink's health. The skill ships in the `milo-observe` Claude Code plugin, alongside skills for querying project logs and reading the status metrics Datum publishes for your resources. ## Install ```bash Claude Code theme={null} /plugin marketplace add datum-cloud/claude-code-plugins /plugin install milo-observe@datum-claude-code-plugins ``` For other agents, copy the [skill folder](https://github.com/milo-os/telemetry/tree/main/claude-plugin/skills/export-metrics) into your agent's skills directory. > [!NOTE] > > Once installed, the skill activates automatically when you ask an agent to work with Metrics Export on Datum. No further configuration needed. ## Capabilities * List and describe export policies in a project * Create metrics export pipelines from YAML manifests * Select exported metrics with MetricsQL selector expressions, or export everything * Configure Prometheus remote write endpoints with credentials in a `kubernetes.io/basic-auth` Secret * Tune batching and retry behavior per sink * Apply changes idempotently * Preview changes before applying with `datumctl diff` and `--dry-run=server` * Check the policy's `Ready` condition and each sink's `Accepted` condition * Delete export policies safely ## How it relates to other Datum surfaces * **Product docs:** [Metrics Export](/docs/operations/metrics-export) — the human-facing reference for the export pipeline and ExportPolicy resource * **Datum MCP:** [Datum MCP](/docs/agents/datum-mcp) — for agents that need live read/write access to export policies during a session * **`datumctl`:** [datumctl overview](/docs/datumctl/overview) — `get`, `describe`, `apply`, `diff`, `delete` commands the skill teaches agents to use ## View the full skill The canonical SKILL.md lives in the `milo-observe` plugin in the [milo-os/telemetry](https://github.com/milo-os/telemetry/tree/main/claude-plugin) repo: > [card] [On GitHub](https://github.com/milo-os/telemetry/blob/main/claude-plugin/skills/export-metrics/SKILL.md) > > Browse the skill source, including frontmatter and full instructions. > [card] [Raw SKILL.md](https://raw.githubusercontent.com/milo-os/telemetry/main/claude-plugin/skills/export-metrics/SKILL.md) > > Direct URL for agents that fetch skills by URL. # Networks skill Source: https://datum.net/docs/agents/skills/networks Teach agents to create and manage gVPC networks and network services with datumctl, and to read why one is not working. The Networks skill teaches agents to work with [gVPC](/docs/galactic-vpc/overview) networks and [network services](/docs/galactic-vpc/networks/network-services) through `datumctl network`: create a network, attach workloads to it, give a workload a stable name and named ports for a load balancer, release addresses kept after an Instance is gone, and delete a network without surprising anyone. The skill ships in the `datum-networking` Claude Code plugin, alongside procedures for diagnosing a load balancer that isn't serving and for creating networks, using the same diagnosis Datum's own assistant runs. ## Install ```bash Claude Code theme={null} /plugin marketplace add datum-cloud/claude-code-plugins /plugin install datum-networking@datum-claude-code-plugins ``` Installing it also installs the `datum-cloud` base plugin, which runs [Datum MCP](/docs/agents/datum-mcp). For other agents, copy the [skill folder](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin/skills/networks) into your agent's skills directory. > [!NOTE] > > Once installed, the skill activates automatically when you ask an agent to work with networks on Datum. > [!WARNING] > > The skill teaches the `datumctl network` commands, which come from a plugin that is not installed by default. Install it before an agent follows this skill, or every command it runs will fail: > > ```bash theme={null} > datumctl plugin install network > datumctl network version > ``` ## Capabilities * Create, list, describe, change and delete networks * Create network services for a workload and point a load balancer at them * Change what a network service selects, or its ports * List Instances' network interfaces and release the ones kept after their Instance went away * Read why a network or a network service is not working, and who has to act * Preview every change before applying it It also teaches the readings that trip up a naive agent: a new network with no locations is normal, a service with no ready Instances is a workload problem rather than a network one, and networks never carry IPv4 — including for a workload that asks for a public IPv4 address. ## How it relates to other Datum surfaces * **Product docs:** [gVPC](/docs/galactic-vpc/overview) and [gVPC operations](/docs/galactic-vpc/operations) — the human-facing reference * **Datum MCP:** [Datum MCP](/docs/agents/datum-mcp) — its `network` and `vpc` toolsets give an agent live access to networks during a session * **[Patch](/docs/operations/datum-assistant)**, Datum's own assistant — a separate surface with read-only network tools of its own, entitled through the platform catalog. It answers questions about a network directly; this skill is for driving `datumctl` yourself * **[Application Load Balancer skill](/docs/agents/skills/alb)** — for putting a network service on the internet ## View the full skill The canonical SKILL.md lives in the `datum-networking` plugin in the [datum-cloud/network-services-operator](https://github.com/datum-cloud/network-services-operator/tree/main/claude-plugin) repo: > [card] [On GitHub](https://github.com/datum-cloud/network-services-operator/blob/main/claude-plugin/skills/networks/SKILL.md) > > Browse the skill source, including frontmatter and full instructions. > [card] [Raw SKILL.md](https://raw.githubusercontent.com/datum-cloud/network-services-operator/main/claude-plugin/skills/networks/SKILL.md) > > Direct URL for agents that fetch skills by URL. # Basic authentication Source: https://datum.net/docs/alb/authentication/basic-auth Require a username and password before requests reach your origin, with datumctl alb auth or an Envoy Gateway SecurityPolicy. HTTP basic authentication protects a load balancer with a username and password. [Envoy](/docs/platform/envoy) checks the credentials on every request, at the edge, before the request reaches your origin. Requests without valid credentials get `401 Unauthorized`. There are two ways to set it up: * **`datumctl alb auth`** is the quickest. One command sets the users, creates the Secret, and attaches the policy. * **A `SecurityPolicy` manifest** gives you full control, for example to keep the configuration in Git or to protect a single route. ## Set up basic auth with datumctl alb To protect a load balancer, pass one or more users, and give their password on standard input: > [tab] macOS / Linux > > ```bash theme={null} > read -s -p "Password: " password && echo > printf '%s' "$password" | datumctl alb auth set ALB_NAME --user admin --password-stdin > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $password = Read-Host "Password" -MaskInput > $password | datumctl alb auth set ALB_NAME --user admin --password-stdin > ``` To admit several users with the same password, repeat `--user`, as in `--user admin --user editor`. To check a change without applying it, add `--dry-run`. To see which usernames are allowed, run the following command: ```bash theme={null} datumctl alb auth list ALB_NAME ``` `datumctl alb describe ALB_NAME` also shows whether basic auth is on. To turn basic auth off, run the following command: ```bash theme={null} datumctl alb auth unset ALB_NAME ``` To confirm that it works, request the load balancer with and without credentials: ```bash theme={null} curl -s -o /dev/null -w '%{http_code}\n' https://HOSTNAME/ # 401 curl -s -o /dev/null -w '%{http_code}\n' -u admin https://HOSTNAME/ # prompts for the password, then 200 ``` ## Set up basic auth with a SecurityPolicy The rest of this page shows how to configure the same thing with [Envoy Gateway](https://gateway.envoyproxy.io/) resources, using `datumctl apply`. *** ### How it works At a high level, this setup: 1. Generates credentials in **htpasswd (SHA)** format 2. Stores credentials in a **Kubernetes Secret** 3. Labels the Secret so it is synced to edge clusters 4. Attaches a **SecurityPolicy** to an `HTTPRoute` 5. Verifies authentication behavior *** ## Prerequisites * `datumctl` installed and authenticated * A valid **Project** * Existing: * `Gateway` * `HTTPRoute` * Permission to create: * `Secret` * `SecurityPolicy` Verify access: ```bash theme={null} datumctl get gateway datumctl get httproute ``` *** ## Critical requirement: secret syncing Secrets and ConfigMaps referenced by gateway configurations are **not automatically synced** to edge clusters. To make a Secret available to edge gateways, it **must** include the following label: ```yaml theme={null} networking.datumapis.com/gateway-sync: "true" ``` ### Why this matters * Without this label, the gateway receives the policy but **not the Secret** * Envoy fails to load the Basic Auth credentials * All requests return **HTTP 500** instead of a login prompt This requirement exists to prevent accidental replication of unrelated secrets. *** ## Configuration steps ### Step 1: Set variables > [tab] macOS / Linux > > ```bash theme={null} > project="your-project-id" > namespace="default" > route="your-route-name" > username="demo" > password="ChangeMe123!" > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $project = "your-project-id" > $namespace = "default" > $route = "your-route-name" > $username = "demo" > $password = "ChangeMe123!" > ``` *** ### Step 2: Generate htpasswd entry (SHA) Envoy Gateway currently supports **SHA-based htpasswd entries only**. Other formats (bcrypt, APR1) are **not supported** and will cause failures. > [tab] macOS / Linux > > ```bash theme={null} > hash=$(printf "%s" "$password" | shasum | awk '{print $1}' | xxd -r -p | base64) > htpasswd=$(printf "%s:{SHA}%s\n" "$username" "$hash") > b64=$(printf "%s" "$htpasswd" | base64) > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $bytes = [Text.Encoding]::UTF8.GetBytes($password) > $sha1 = [System.Security.Cryptography.SHA1]::Create().ComputeHash($bytes) > $hash = [Convert]::ToBase64String($sha1) > > $htpasswd = "${username}:{SHA}${hash}`n" > $b64 = [Convert]::ToBase64String( > [Text.Encoding]::UTF8.GetBytes($htpasswd) > ) > ``` *** ### Step 3: Create the secret The Secret must: * Use the key `.htpasswd` * Exist in the same namespace as the `SecurityPolicy` * Include the **gateway-sync** label > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: v1 > kind: Secret > metadata: > name: ${route}-basic-auth > labels: > networking.datumapis.com/gateway-sync: "true" > type: Opaque > data: > .htpasswd: $b64 > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: v1 > kind: Secret > metadata: > name: ${route}-basic-auth > labels: > networking.datumapis.com/gateway-sync: "true" > type: Opaque > data: > .htpasswd: $b64 > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ### Step 4: Attach Basic Auth using a SecurityPolicy Attach the policy to the **HTTPRoute**. This is the most reliable attachment point. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: SecurityPolicy > metadata: > name: ${route}-basic-auth > spec: > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $route > basicAuth: > users: > kind: Secret > name: ${route}-basic-auth > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: SecurityPolicy > metadata: > name: ${route}-basic-auth > spec: > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $route > basicAuth: > users: > kind: Secret > name: ${route}-basic-auth > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ## Verification ### Unauthenticated request ```bash theme={null} curl -I https://your-app.example.com ``` Expected response: ``` HTTP/1.1 401 Unauthorized WWW-Authenticate: Basic ``` Browsers will display a login prompt. *** ### Authenticated request ```bash theme={null} curl -I -u demo:ChangeMe123! https://your-app.example.com ``` Expected response: ``` HTTP/1.1 200 OK ``` *** ## Cleanup / disable Basic Auth > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete securitypolicy ${route}-basic-auth \ > --project $project --namespace $namespace --ignore-not-found > > datumctl delete secret ${route}-basic-auth \ > --project $project --namespace $namespace --ignore-not-found > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete securitypolicy ${route}-basic-auth ` > --project $project --namespace $namespace --ignore-not-found > > datumctl delete secret ${route}-basic-auth ` > --project $project --namespace $namespace --ignore-not-found > ``` *** ## Troubleshooting ### Common failure modes | Symptom | Root cause | Resolution | | - | - | - | | HTTP 500 on all requests | Secret not synced to edge | Add `gateway-sync` label | | 500 even with credentials | Secret missing at edge | Re-apply Secret with label | | No login prompt | Policy not attached | Verify `targetRefs` | | Works after deleting policy | Invalid Secret reference | Fix Secret and label | ### Useful debug commands ```bash theme={null} datumctl get secret --project $project --namespace $namespace datumctl get securitypolicy --project $project --namespace $namespace datumctl get httproute --project $project --namespace $namespace ``` *** ## Best practices * Use Basic Auth for development, demos, and staging environments * Rotate credentials regularly * Store credentials securely using CI secrets or a secret manager * Use JWT, OIDC, or SSO for production authentication *** ## Summary * HTTP Basic Auth is configured using **Envoy Gateway `SecurityPolicy`** * Secrets **must be explicitly synced** to edge clusters using the `gateway-sync` label * `{SHA}` htpasswd format is required — bcrypt and APR1 are not supported * Attaching policies to `HTTPRoute` is the most reliable approach # Per-user access with Auth0 Source: https://datum.net/docs/alb/authentication/oidc-auth0 Configure per-user authentication and email-based authorization using Auth0 and Google social login for applications behind a Datum gateway. This page shows how to let only the people on your allow-list reach your app. Auth0 handles Google sign-in and checks each person's email against the list before it issues a token, so people who aren't allowed never reach Envoy. The configuration uses an [Envoy Gateway](https://gateway.envoyproxy.io/) `SecurityPolicy` and `datumctl`. *** ## How it works ```mermaid theme={null} sequenceDiagram autonumber participant B as Browser participant E as Envoy (your ALB) participant A as Auth0 participant G as Google participant App as Your app B->>E: GET /app E-->>B: No session: redirect to Auth0 B->>A: Sign in A->>G: Google sign-in G-->>B: Google sign-in page B->>A: Callback Note over A: Check the allow-list.
Only allowed users get a token. A-->>E: ID token Note over E: Validate the token:
signature, issuer, audience, expiry E-->>B: Session cookie B->>E: GET /app E->>App: Forward the request App-->>B: 200 OK ``` **Authentication** (who are you): Auth0 handles Google social login.\ **Authorization** (are you allowed): Auth0 checks the user against your allow-list before issuing a token. Unauthorized users never get a token — [Envoy](/docs/platform/envoy) never sees them. *** ## Prerequisites * `datumctl` installed and authenticated * A valid Datum **Project** * An existing **HTTPProxy** in the Datum UI (this provides the Gateway and HTTPRoute automatically) * An **Auth0 account** (free tier is sufficient) * A **Google Cloud** project for the social connection > **Note:** The HTTPProxy name is the name of the auto-generated HTTPRoute that the SecurityPolicy will target. Find it with: > > ```bash theme={null} > datumctl get httproute --project your-project-id --namespace default > ``` *** ## Part 1: Auth0 setup ### Step 1: Create an Auth0 tenant 1. Sign up at [auth0.com](https://auth0.com) and create a tenant 2. Note your **tenant domain** — it looks like `dev-xxxxxxxx.us.auth0.com` *** ### Step 2: Create an application 1. In the Auth0 dashboard go to **Applications** → **Applications** → **Create Application** 2. Name it (e.g. `datum-gateway-app`) 3. Select **Regular Web Applications** 4. Select **Create** 5. On the **Settings** tab, note your **Client ID** and **Client Secret** 6. Under **Allowed Callback URLs** add: ``` https:///oauth2/callback ``` 7. Under **Allowed Logout URLs** add: ``` https:// ``` 8. Select **Save Changes** *** ### Step 3: Enable Google social login 1. Go to **Authentication** → **Social** → **Create Connection** 2. Select **Google / Gmail** 3. Enter your **Google OAuth Client ID** and **Client Secret** (from [Google Cloud Console](https://console.cloud.google.com) → APIs & Services → Credentials) 4. Enable the connection for your application under the **Applications** tab of the connection 5. In **Google Cloud Console**, add Auth0's callback URL to your OAuth client's **Authorized redirect URIs**: ``` https:///login/callback ``` This is required so Google can redirect back to Auth0 after login. Without it, Google returns `Error 400: redirect_uri_mismatch`. > **Note:** If you do not have Google OAuth credentials yet, Auth0 provides a built-in dev key for testing. Go to the Google connection settings and leave Client ID/Secret blank — Auth0 will use its own dev credentials. Create dedicated Google OAuth credentials before moving to production. *** ### Step 4: Create an access control action This is where individual user access is enforced. Auth0 runs this code during login — users not in the list are denied before a token is ever issued. 1. Go to **Actions** → **Library** → **Build Custom** 2. Name it `enforce-email-allowlist` 3. Select **Login / Post Login** trigger 4. Replace the default code with: ```javascript theme={null} exports.onExecutePostLogin = async (event, api) => { const allowedEmails = [ "alice@example.com", "bob@datum.net", ]; if (!allowedEmails.includes(event.user.email)) { api.access.deny("Access denied: you are not authorized to use this application."); } }; ``` 5. Select **Deploy** 6. Go to **Actions** → **Triggers** → **post-login** 7. Drag your `enforce-email-allowlist` action from the right sidebar into the flow between **Start** and **Complete** 8. Select **Apply** To add or remove a user: edit the `allowedEmails` array and select **Deploy**. No gateway config changes required. *** ## Part 2: Datum gateway configuration The Datum HTTPProxy already manages the Gateway and HTTPRoute. You only need to create a **Secret** (to store the Auth0 client secret) and a **SecurityPolicy** (to attach OIDC to the route). ### Step 1: Set variables > [tab] macOS / Linux > > ```bash theme={null} > project="your-project-id" > namespace="default" > route="your-httpproxy-name" > hostname="your-app.example.com" > auth0Domain="dev-xxxxxxxx.us.auth0.com" > clientID="your-auth0-client-id" > clientSecret="your-auth0-client-secret" > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $project = "your-project-id" > $namespace = "default" > $route = "your-httpproxy-name" > $hostname = "your-app.example.com" > $auth0Domain = "dev-xxxxxxxx.us.auth0.com" > $clientID = "your-auth0-client-id" > $clientSecret = "your-auth0-client-secret" > ``` *** ### Step 2: Create the Auth0 client secret The Secret must include the `gateway-sync` label or the edge proxy will not receive it and all requests will return HTTP 500. > [tab] macOS / Linux > > ```bash theme={null} > b64Secret=$(printf "%s" "$clientSecret" | base64) > > cat < apiVersion: v1 > kind: Secret > metadata: > name: ${route}-oidc-secret > labels: > networking.datumapis.com/gateway-sync: "true" > type: Opaque > data: > client-secret: $b64Secret > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $b64Secret = [Convert]::ToBase64String( > [Text.Encoding]::UTF8.GetBytes($clientSecret) > ) > > @" > apiVersion: v1 > kind: Secret > metadata: > name: ${route}-oidc-secret > labels: > networking.datumapis.com/gateway-sync: "true" > type: Opaque > data: > client-secret: $b64Secret > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ### Step 3: Apply the SecurityPolicy This attaches OIDC authentication to the existing HTTPProxy route. > **Warning:** Use only the `oidc` block — do not add a `jwt` block, as it conflicts with the OIDC filter and causes HTTP 500. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: SecurityPolicy > metadata: > name: ${route}-oidc > spec: > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $route > oidc: > provider: > issuer: "https://${auth0Domain}" > clientID: "$clientID" > clientSecret: > name: ${route}-oidc-secret > redirectURL: "https://${hostname}/oauth2/callback" > scopes: > - openid > - email > - profile > EOF > ``` > > Allow 60 seconds for the policy to propagate to the edge before testing. > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: SecurityPolicy > metadata: > name: ${route}-oidc > spec: > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $route > oidc: > provider: > issuer: "https://${auth0Domain}" > clientID: "$clientID" > clientSecret: > name: ${route}-oidc-secret > redirectURL: "https://${hostname}/oauth2/callback" > scopes: > - openid > - email > - profile > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ## Verification ### Unauthenticated request — redirects to Auth0 ```bash theme={null} curl -I https://your-app.example.com ``` Expected: ``` HTTP/1.1 302 Found Location: https://dev-xxxxxxxx.us.auth0.com/authorize?... ``` ### Browser flow — allowed user 1. Open `https://your-app.example.com` in a browser 2. You are redirected to Auth0 → Google login 3. Sign in with an email in the allow-list 4. You are redirected back and the app loads — `HTTP 200` ### Browser flow — denied user 1. Sign in with an email **not** in the allow-list 2. Auth0 displays: `Access denied: you are not authorized to use this application.` 3. No token is issued — Envoy is never reached *** ## Managing access All access control is managed in the Auth0 dashboard — no gateway config changes needed. **Add a user:** 1. Go to **Actions** → **Library** → `enforce-email-allowlist` 2. Add the email to `allowedEmails` 3. Select **Deploy** **Remove a user:** 1. Remove the email from `allowedEmails` 2. Select **Deploy** 3. Optionally revoke any active sessions: **User Management** → find user → **Invalidate Sessions** *** ## Cleanup > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete securitypolicy ${route}-oidc \ > --project $project --namespace $namespace --ignore-not-found > > datumctl delete secret ${route}-oidc-secret \ > --project $project --namespace $namespace --ignore-not-found > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete securitypolicy ${route}-oidc ` > --project $project --namespace $namespace --ignore-not-found > > datumctl delete secret ${route}-oidc-secret ` > --project $project --namespace $namespace --ignore-not-found > ``` *** ## Troubleshooting | Symptom | Root cause | Resolution | | - | - | - | | HTTP 200 instead of redirect | SecurityPolicy not yet propagated | Wait 60 seconds; verify with `datumctl get securitypolicy` | | HTTP 500 on all requests (immediate) | `jwt` block present in SecurityPolicy | Remove `jwt` block — use `oidc` block only | | HTTP 500 on all requests (immediate) | WAF (`TrafficProtectionPolicy`) blocking OIDC | Remove the `TrafficProtectionPolicy` targeting the Gateway | | HTTP 500 on all requests | Secret missing `gateway-sync` label | Verify label is present; re-apply Secret if needed | | Auth0 error: `callback URL mismatch` | App hostname not in Auth0 Allowed Callback URLs | Add `https:///oauth2/callback` in Auth0 Application Settings | | Google error 400: `redirect_uri_mismatch` | Auth0 callback not in Google OAuth client | Add `https:///login/callback` to Google Cloud Console Authorized redirect URIs | | Redirect loop | `redirectURL` mismatch | Ensure it matches **Allowed Callback URLs** in Auth0 exactly | | User sees "Access denied" at Auth0 | Not in allow-list | Add email to the Action and redeploy | | User bypasses allow-list | Action not in Login flow | Verify action is added under **Actions** → **Triggers** → **post-login** | ### Useful debug commands ```bash theme={null} datumctl get securitypolicy --project $project --namespace $namespace datumctl get secret --project $project --namespace $namespace datumctl get httproute --project $project --namespace $namespace datumctl get trafficprotectionpolicy --project $project --namespace $namespace ``` *** ## Summary | Concern | Where it lives | | - | - | | Google social login | Auth0 social connection | | Who is allowed | Auth0 Action (`enforce-email-allowlist`) | | Token issuance | Auth0 — denied users never get a token | | Token validation | Envoy OIDC block (handled automatically) | | Session management | Envoy OIDC block (cookie) | | App changes required | None | # Sign in with Google (OIDC) Source: https://datum.net/docs/alb/authentication/oidc-google Configure OIDC authentication using Google OAuth 2.0 for applications behind a Datum gateway using datumctl and Envoy Gateway SecurityPolicy. This page shows how to require people to sign in with their Google account before requests reach your app, using an [Envoy Gateway](https://gateway.envoyproxy.io/) `SecurityPolicy` and `datumctl`. *** ## Overview OIDC authentication protects an application by requiring users to sign in with their Google account before requests are forwarded to the origin. Envoy Gateway handles the OAuth 2.0 flow transparently — users are redirected to Google, authenticate, and are redirected back without any changes to your application. At a high level, this setup: 1. Creates a **Google OAuth 2.0 Client** in Google Cloud Console 2. Stores the client secret in a **Kubernetes Secret** 3. Labels the Secret so it is synced to edge clusters 4. Attaches a **SecurityPolicy** with OIDC config to an `HTTPRoute` 5. Verifies authentication behavior *** ## Prerequisites * `datumctl` installed and authenticated * A valid **Project** * A **Google Cloud** account with a project * Existing: * `Gateway` * `HTTPRoute` * Permission to create: * `Secret` * `SecurityPolicy` Verify access: ```bash theme={null} datumctl get gateway datumctl get httproute ``` *** ## Critical requirement: secret syncing Secrets and ConfigMaps referenced by gateway configurations are **not automatically synced** to edge clusters. To make a Secret available to edge gateways, it **must** include the following label: ```yaml theme={null} networking.datumapis.com/gateway-sync: "true" ``` ### Why this matters * Without this label, the gateway receives the policy but **not the Secret** * [Envoy](/docs/platform/envoy) fails to load the OAuth client credentials * All requests return **HTTP 500** instead of redirecting to Google This requirement exists to prevent accidental replication of unrelated secrets. *** ## Configuration steps ### Step 1: Create Google OAuth 2.0 credentials 1. Go to [Google Cloud Console](https://console.cloud.google.com) → **APIs & Services** → **Credentials** 2. Select **Create Credentials** → **OAuth 2.0 Client ID** 3. Set **Application type** to **Web application** 4. Under **Authorized redirect URIs**, add: ``` https:///oauth2/callback ``` Replace `` with the public hostname of your `HTTPRoute`. 5. Select **Create** and copy the **Client ID** and **Client Secret** > **Note:** The redirect URI must exactly match the value you will configure in the `SecurityPolicy`. Any mismatch causes Google to reject the OAuth flow. *** ### Step 2: Set variables > [tab] macOS / Linux > > ```bash theme={null} > project="your-project-id" > namespace="default" > route="your-route-name" > hostname="your-app.example.com" > clientID="your-client-id.apps.googleusercontent.com" > clientSecret="your-client-secret" > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $project = "your-project-id" > $namespace = "default" > $route = "your-route-name" > $hostname = "your-app.example.com" > $clientID = "your-client-id.apps.googleusercontent.com" > $clientSecret = "your-client-secret" > ``` *** ### Step 3: Create the client secret The Secret must: * Use the key `client-secret` * Exist in the same namespace as the `SecurityPolicy` * Include the **gateway-sync** label > [tab] macOS / Linux > > ```bash theme={null} > b64Secret=$(printf "%s" "$clientSecret" | base64) > > cat < apiVersion: v1 > kind: Secret > metadata: > name: ${route}-oidc-secret > labels: > networking.datumapis.com/gateway-sync: "true" > type: Opaque > data: > client-secret: $b64Secret > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $b64Secret = [Convert]::ToBase64String( > [Text.Encoding]::UTF8.GetBytes($clientSecret) > ) > > @" > apiVersion: v1 > kind: Secret > metadata: > name: ${route}-oidc-secret > labels: > networking.datumapis.com/gateway-sync: "true" > type: Opaque > data: > client-secret: $b64Secret > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ### Step 4: Attach OIDC authentication using a SecurityPolicy Attach the policy to the **HTTPRoute**. This is the most reliable attachment point. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: SecurityPolicy > metadata: > name: ${route}-oidc > spec: > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $route > oidc: > provider: > issuer: "https://accounts.google.com" > clientID: "$clientID" > clientSecret: > name: ${route}-oidc-secret > redirectURL: "https://${hostname}/oauth2/callback" > scopes: > - openid > - email > - profile > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: SecurityPolicy > metadata: > name: ${route}-oidc > spec: > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $route > oidc: > provider: > issuer: "https://accounts.google.com" > clientID: "$clientID" > clientSecret: > name: ${route}-oidc-secret > redirectURL: "https://${hostname}/oauth2/callback" > scopes: > - openid > - email > - profile > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ## Verification ### Unauthenticated request ```bash theme={null} curl -I https://your-app.example.com ``` Expected response: ``` HTTP/1.1 302 Found Location: https://accounts.google.com/o/oauth2/v2/auth?... ``` Browsers will redirect automatically to the Google sign-in page. *** ### Authenticated session After completing the Google sign-in flow, subsequent requests will be forwarded to your origin with a session cookie managed by Envoy. ```bash theme={null} curl -I https://your-app.example.com ``` Expected response after authentication: ``` HTTP/1.1 200 OK ``` *** ## Cleanup / disable OIDC authentication > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete securitypolicy ${route}-oidc \ > --project $project --namespace $namespace --ignore-not-found > > datumctl delete secret ${route}-oidc-secret \ > --project $project --namespace $namespace --ignore-not-found > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete securitypolicy ${route}-oidc ` > --project $project --namespace $namespace --ignore-not-found > > datumctl delete secret ${route}-oidc-secret ` > --project $project --namespace $namespace --ignore-not-found > ``` *** ## Troubleshooting ### Common failure modes | Symptom | Root cause | Resolution | | - | - | - | | HTTP 500 on all requests | Secret not synced to edge | Add `gateway-sync` label | | Redirect loop | Redirect URI mismatch | Ensure `redirectURL` matches Google Console exactly | | `redirect_uri_mismatch` from Google | URI not registered | Add URI to Google Cloud Credentials | | HTTP 401 after login | Scopes insufficient | Verify `openid` scope is included | | Works after deleting policy | Invalid Secret reference | Fix Secret name and `gateway-sync` label | *** ### Useful debug commands ```bash theme={null} datumctl get secret --project $project --namespace $namespace datumctl get securitypolicy --project $project --namespace $namespace datumctl get httproute --project $project --namespace $namespace ``` *** ## Best practices * Register **separate OAuth clients** for each environment (dev, staging, prod) * Restrict **authorized redirect URIs** to known hostnames only * Limit **OAuth scopes** to the minimum required (`openid`, `email`) * Rotate the **client secret** periodically and re-apply the Secret * Use OIDC for production — Basic Auth is suited for development and demos *** ## Summary * Google OAuth is configured using **Envoy Gateway `SecurityPolicy`** with an `oidc` block * The client secret is stored as a Kubernetes Secret with key **`client-secret`** * Secrets **must be explicitly synced** to edge clusters using the `networking.datumapis.com/gateway-sync: "true"` label * The **`redirectURL`** in the policy must exactly match the URI registered in Google Cloud Console * Attaching policies to `HTTPRoute` is the most reliable approach # Authentication Source: https://datum.net/docs/alb/authentication/overview Require people to sign in before requests reach your origin, with basic authentication, Google sign-in, or per-user access through Auth0. A load balancer can require people to prove who they are before a request ever reaches your origin. [Envoy](/docs/platform/envoy) checks every request at the edge, in every Datum location, so your app doesn't need to change and unauthenticated traffic never reaches it. ## Choose a method | Method | Who can get in | Best for | Guide | | - | - | - | - | | Basic authentication | Anyone with a username and password you set | Staging sites, internal tools, and quick protection for anything | [Basic authentication](/docs/alb/authentication/basic-auth) | | Sign in with Google (OIDC) | Anyone who signs in with a Google account | Requiring a real identity, such as for a team that uses Google Workspace | [Sign in with Google](/docs/alb/authentication/oidc-google) | | Per-user access with Auth0 | Only the people on your allow-list | Controlling exactly which people get in, by email address | [Per-user access with Auth0](/docs/alb/authentication/oidc-auth0) | The quickest way to start is basic authentication with `datumctl alb`: ```bash theme={null} printf '%s' "$password" | datumctl alb auth set ALB_NAME --user admin --password-stdin ``` ## How it works Authentication is configured with an [Envoy Gateway](https://gateway.envoyproxy.io/) `SecurityPolicy`. The policy names what it protects, such as a load balancer's route, and how people sign in. Datum sends the policy to Envoy in every location, and Envoy enforces it on every request: ```mermaid theme={null} flowchart LR B["Browser or client"] E["Envoy · your ALB
checks the SecurityPolicy"] I["Identity provider
Google or Auth0"] O["Your origin"] B -->|"request"| E E -.->|"no session: sign in"| I I -.->|"token"| E E -->|"signed in: forward"| O classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 class B,E,I,O box linkStyle 0,3 stroke:#5A5F6B,stroke-width:1.4px linkStyle 1,2 stroke:#B5605E,stroke-width:1.4px,stroke-dasharray:5 4 ``` * **Basic authentication** checks a username and password against an `htpasswd` list stored in a Secret. A request without valid credentials gets `401 Unauthorized`. * **OIDC** redirects people without a session to an identity provider, such as Google or Auth0. Envoy validates the token the provider returns, then sets a session cookie so later requests go straight through. Authentication runs alongside the [web application firewall](/docs/alb/configuration/waf). A request has to pass both before it reaches your origin. ## Secrets must be labeled to reach the edge Each method stores a credential in a Secret: an `htpasswd` list for basic authentication, or a client secret for OIDC. Datum doesn't send Secrets to the edge unless you ask it to, so that unrelated Secrets in your project are never copied. To make a Secret available to your load balancer, give it this label: ```yaml theme={null} metadata: labels: networking.datumapis.com/gateway-sync: "true" ``` > [!WARNING] > > Without this label, Envoy receives the policy but not the Secret, and every request to the load balancer returns `500` instead of asking people to sign in. `datumctl alb auth set` adds the label for you. ## Check authentication To see whether a load balancer has basic authentication, run `datumctl alb describe ALB_NAME`. To see every `SecurityPolicy` in your project, run `datumctl get securitypolicies`. To confirm that a protected load balancer rejects anonymous requests, request it without credentials: ```bash theme={null} curl -s -o /dev/null -w '%{http_code}\n' https://HOSTNAME/ ``` Basic authentication returns `401`. OIDC returns `302`, a redirect to the sign-in page. | Symptom | Likely cause | Fix | | - | - | - | | Every request returns `500` | The Secret isn't labeled to reach the edge, or its name is wrong. | Add the `gateway-sync` label, and check the Secret name in the policy. | | Requests reach the origin without signing in | The policy targets a different route or load balancer. | Check `targetRefs` in the policy. | | A redirect loop after signing in | The redirect URL doesn't match the one registered with the provider. | Make the two match exactly. | For more, see the troubleshooting section of each guide. # Failover between origins Source: https://datum.net/docs/alb/configuration/failover Configure automatic failover between a primary and fallback backend for applications behind a Datum gateway using datumctl and Envoy Gateway Backend, BackendTLSPolicy, and BackendTrafficPolicy resources. *** ## Overview Failover routes traffic to a fallback origin only after the primary origin starts failing, and returns traffic to the primary once it recovers. At a high level, this setup: 1. Defines two **`Backend`** resources — a primary and a fallback origin 2. Pairs each `Backend` with a **`BackendTLSPolicy`** so [Envoy](/docs/platform/envoy) presents the correct SNI to the origin 3. References both backends from an **`HTTPRoute`**, with the fallback `Backend` marked `fallback: true` 4. Applies a **`BackendTrafficPolicy`** that configures passive (outlier-detection) health checking 5. Verifies failover by forcing the primary to fail, then verifies automatic recovery *** ## Prerequisites * `datumctl` installed and authenticated * A valid **Project** * Existing: * `Gateway` * `HTTPRoute` * Two backend origins reachable over TLS on port 443 — a primary and a fallback — each with its own publicly resolvable hostname * Permission to create: * `Backend` * `BackendTLSPolicy` * `BackendTrafficPolicy` Verify access: ```bash theme={null} datumctl get gateway datumctl get httproute ``` *** ## Critical requirement: BackendTLSPolicy for FQDN backends A `Backend` that targets an external FQDN **must** be paired with a `BackendTLSPolicy` that sets `spec.validation.hostname`. ### Why this matters * An FQDN `Backend`'s inline `spec.tls` block alone does **not** set outbound SNI * Origins that route by SNI (most shared-IP hosting platforms) reject connections with no or incorrect SNI * This fails as `upstream_reset_before_response_started{remote_connection_failure}` — it is **not** a certificate-trust error, and setting `insecureSkipVerify: true` will not fix it Origins that do not inspect SNI (single-tenant IPs, some test endpoints) will work without this, which can hide the problem until you move to a shared-IP host. *** ## Configuration steps ### Step 1: Set variables > [tab] macOS / Linux > > ```bash theme={null} > project="your-project-id" > namespace="default" > route="your-route-name" > gateway="your-gateway-name" > primaryHostname="primary.your-origin.example.com" > fallbackHostname="fallback.your-origin.example.com" > primaryBackend="primary-backend" > fallbackBackend="fallback-backend" > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $project = "your-project-id" > $namespace = "default" > $route = "your-route-name" > $gateway = "your-gateway-name" > $primaryHostname = "primary.your-origin.example.com" > $fallbackHostname = "fallback.your-origin.example.com" > $primaryBackend = "primary-backend" > $fallbackBackend = "fallback-backend" > ``` *** ### Step 2: Create the primary and fallback backends Set `fallback: true` on the fallback `Backend` only. Envoy sends traffic to a `fallback: true` backend only once every non-fallback backend referenced by the same route is ejected. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: Backend > metadata: > name: $primaryBackend > spec: > endpoints: > - fqdn: > hostname: $primaryHostname > port: 443 > fallback: false > tls: > wellKnownCACertificates: System > --- > apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: Backend > metadata: > name: $fallbackBackend > spec: > endpoints: > - fqdn: > hostname: $fallbackHostname > port: 443 > fallback: true > tls: > wellKnownCACertificates: System > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: Backend > metadata: > name: $primaryBackend > spec: > endpoints: > - fqdn: > hostname: $primaryHostname > port: 443 > fallback: false > tls: > wellKnownCACertificates: System > --- > apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: Backend > metadata: > name: $fallbackBackend > spec: > endpoints: > - fqdn: > hostname: $fallbackHostname > port: 443 > fallback: true > tls: > wellKnownCACertificates: System > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ### Step 3: Attach a BackendTLSPolicy to each backend > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: gateway.networking.k8s.io/v1 > kind: BackendTLSPolicy > metadata: > name: ${primaryBackend}-tls > spec: > targetRefs: > - group: gateway.envoyproxy.io > kind: Backend > name: $primaryBackend > validation: > hostname: $primaryHostname > wellKnownCACertificates: System > --- > apiVersion: gateway.networking.k8s.io/v1 > kind: BackendTLSPolicy > metadata: > name: ${fallbackBackend}-tls > spec: > targetRefs: > - group: gateway.envoyproxy.io > kind: Backend > name: $fallbackBackend > validation: > hostname: $fallbackHostname > wellKnownCACertificates: System > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: gateway.networking.k8s.io/v1 > kind: BackendTLSPolicy > metadata: > name: ${primaryBackend}-tls > spec: > targetRefs: > - group: gateway.envoyproxy.io > kind: Backend > name: $primaryBackend > validation: > hostname: $primaryHostname > wellKnownCACertificates: System > --- > apiVersion: gateway.networking.k8s.io/v1 > kind: BackendTLSPolicy > metadata: > name: ${fallbackBackend}-tls > spec: > targetRefs: > - group: gateway.envoyproxy.io > kind: Backend > name: $fallbackBackend > validation: > hostname: $fallbackHostname > wellKnownCACertificates: System > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ### Step 4: Update the HTTPRoute to reference both backends Add both backends to the route's `backendRefs`. Give them equal `weight` — the fallback's `fallback: true` flag, not the weight, is what keeps it out of rotation while the primary is healthy. > **Warning:** This manifest replaces the `HTTPRoute`'s entire `spec`. If your existing `HTTPRoute` has additional rules, matches, or filters, merge them into the manifest below instead of applying this as-is. Adjust `sectionName` to match the listener your route currently attaches to. > **Warning:** Do not add a `hostnames` field to the `HTTPRoute`. Hostname routing is controlled solely by the parent `Gateway`'s listener, and `spec.hostnames` on the route will be rejected by admission. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: gateway.networking.k8s.io/v1 > kind: HTTPRoute > metadata: > name: $route > spec: > parentRefs: > - name: $gateway > sectionName: default-https > rules: > - matches: > - path: > type: PathPrefix > value: / > backendRefs: > - group: gateway.envoyproxy.io > kind: Backend > name: $primaryBackend > port: 443 > weight: 1 > - group: gateway.envoyproxy.io > kind: Backend > name: $fallbackBackend > port: 443 > weight: 1 > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: gateway.networking.k8s.io/v1 > kind: HTTPRoute > metadata: > name: $route > spec: > parentRefs: > - name: $gateway > sectionName: default-https > rules: > - matches: > - path: > type: PathPrefix > value: / > backendRefs: > - group: gateway.envoyproxy.io > kind: Backend > name: $primaryBackend > port: 443 > weight: 1 > - group: gateway.envoyproxy.io > kind: Backend > name: $fallbackBackend > port: 443 > weight: 1 > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ### Step 5: Apply passive health checking with a BackendTrafficPolicy > **Note:** Only passive (outlier-detection) health checking is supported. A `healthCheck.active` block is rejected by admission — active `/healthz` polling is not available on this platform. The thresholds below eject a backend after a single error and hold it out of rotation for 30 seconds. Tune `consecutive5XxErrors`, `consecutiveGatewayErrors`, and `interval` for production traffic (see [Best practices](#best-practices)), but leave `maxEjectionPercent` at `100`. With a single primary backend, a lower value can prevent ejection from happening at all. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: BackendTrafficPolicy > metadata: > name: ${route}-failover-policy > spec: > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $route > healthCheck: > passive: > baseEjectionTime: 30s > consecutive5XxErrors: 1 > consecutiveGatewayErrors: 1 > consecutiveLocalOriginFailures: 1 > interval: 30s > maxEjectionPercent: 100 > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: gateway.envoyproxy.io/v1alpha1 > kind: BackendTrafficPolicy > metadata: > name: ${route}-failover-policy > spec: > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $route > healthCheck: > passive: > baseEjectionTime: 30s > consecutive5XxErrors: 1 > consecutiveGatewayErrors: 1 > consecutiveLocalOriginFailures: 1 > interval: 30s > maxEjectionPercent: 100 > "@ | datumctl apply --project $project --namespace $namespace -f - > ``` *** ## Verification ### Primary healthy ```bash theme={null} curl -I https://your-app.example.com ``` Expected response: ``` HTTP/1.1 200 OK ``` Traffic is served by the primary backend. *** ### Primary failing > **Warning:** This test interrupts live traffic to the primary. Run it against a non-production hostname or during a maintenance window. Make the primary origin return a 5xx status (or take it offline). Passive health checking is re-evaluated on each `interval` (30 seconds in this example), not per request, so poll for at least that long: ```bash theme={null} for i in $(seq 1 10); do curl -s -o /dev/null -w "%{http_code}\n" https://your-app.example.com sleep 5 done ``` Expected behavior: * Requests may still reach the failing primary for up to one `interval` after it starts failing * Once `consecutive5XxErrors`/`consecutiveGatewayErrors` is reached at the next interval, the primary is ejected for `baseEjectionTime` * Remaining requests return `200 OK`, served by the fallback backend *** ### Recovery Restore the primary origin to a healthy state and wait out `baseEjectionTime`: ```bash theme={null} sleep 30 curl -I https://your-app.example.com ``` Expected response: ``` HTTP/1.1 200 OK ``` Traffic returns to the primary automatically — no manual re-enable step is required. *** ## Cleanup / disable failover > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete backendtrafficpolicy ${route}-failover-policy \ > --project $project --namespace $namespace --ignore-not-found > > datumctl delete backendtlspolicy ${primaryBackend}-tls ${fallbackBackend}-tls \ > --project $project --namespace $namespace --ignore-not-found > > datumctl delete backend $primaryBackend $fallbackBackend \ > --project $project --namespace $namespace --ignore-not-found > ``` > > > **Note:** Remove the fallback `backendRef` from the `HTTPRoute` as well if you no longer want it defined, or the route will be left pointing at a deleted `Backend`. > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete backendtrafficpolicy ${route}-failover-policy ` > --project $project --namespace $namespace --ignore-not-found > > datumctl delete backendtlspolicy ${primaryBackend}-tls ${fallbackBackend}-tls ` > --project $project --namespace $namespace --ignore-not-found > > datumctl delete backend $primaryBackend $fallbackBackend ` > --project $project --namespace $namespace --ignore-not-found > ``` *** ## Troubleshooting | Symptom | Root Cause | Resolution | | - | - | - | | `upstream_reset_before_response_started{remote_connection_failure}` even though the origin is healthy | Missing `BackendTLSPolicy` — Envoy sent no or incorrect SNI | Add a `BackendTLSPolicy` targeting the `Backend` with `validation.hostname` set to the backend's FQDN | | `HTTPRoute` rejected by admission | `spec.hostnames` set on the `HTTPRoute` | Remove `hostnames` — hostname routing belongs to the `Gateway` listener | | `BackendTrafficPolicy` rejected by admission | `healthCheck.active` block included | Remove `active` — only `passive` (outlier detection) health checking is supported | | Traffic never shifts to the fallback backend | Fallback `Backend` missing `fallback: true`, or `BackendTrafficPolicy.spec.targetRefs` doesn't match the `HTTPRoute` | Set `fallback: true` on the fallback `Backend`; confirm the `targetRefs` name matches | | Fallback keeps serving traffic after the primary recovers | Still inside the `baseEjectionTime` window | Wait out the ejection window — Envoy re-admits the primary automatically once it passes health checks again | ### Useful debug commands ```bash theme={null} datumctl get backend --project $project --namespace $namespace datumctl get backendtlspolicy --project $project --namespace $namespace datumctl get backendtrafficpolicy --project $project --namespace $namespace datumctl get httproute $route --project $project --namespace $namespace -o yaml ``` *** ## Best practices * Set `consecutive5XxErrors` / `consecutiveGatewayErrors` above `1` in production to avoid ejecting on a single transient error * Keep `maxEjectionPercent` at `100` when there is only one primary backend — a lower value can block ejection entirely since there is only one host to eject from * Keep the fallback origin provisioned at production-equivalent capacity — it may receive 100% of traffic during an outage * Test failover on a recurring schedule, not only at initial setup * Monitor ejection and recovery events directly rather than relying on `Gateway` status conditions as the sole health signal *** ## Summary * Failover is expressed with two `Backend` resources — the fallback one set to `fallback: true` * FQDN backends require a paired `BackendTLSPolicy` setting `validation.hostname`, or SNI-sensitive origins will reset the connection * Only passive (outlier-detection) health checking is supported — `active` health checks are rejected by admission * The `HTTPRoute` cannot set `spec.hostnames` — hostname routing belongs to the `Gateway` listener * Recovery to the primary is automatic once `baseEjectionTime` elapses and it passes health checks again # Load balancing and health checks Source: https://datum.net/docs/alb/configuration/load-balancing Split traffic across several origins on a Datum Application Load Balancer with weights, choose the load balancing algorithm, and turn on passive health checks. An Application Load Balancer can send one route's traffic to several origins. Each origin has a weight that sets its share of requests. Two settings apply across every route: the algorithm that picks an endpoint for each request, and passive health checks. ## Several origins on one route A route takes up to 16 origins. They can be any mix of public URLs and Datum compute workloads. Datum sends each URL origin its own hostname as the Host header, so origins on different hostnames can share a route. Two things to watch: * **A connector origin must be the only origin on its route.** * **A Host override on the route sends the same Host to every origin.** In a pool spanning several hostnames, any origin that serves by hostname (Vercel, Netlify, Fly.io, Cloudflare Pages) then answers the wrong site or a 404. See [Host header override](/docs/alb/httpproxy/host-header-override). ## Weights The load balancer splits requests on a route across its origins in proportion to their weights. Weights are relative: origins at 1 and 3 get 25% and 75% of requests. An origin with no weight counts as 1, so a pool with no weights set splits evenly. **Weight 0 drains an origin.** It stays in the pool and gets no requests, which lets you take it out of service without deleting it. At least one origin on a route has to keep a weight above 0, or the route serves nothing. ## The load balancing algorithm The algorithm decides which endpoint gets each request. It applies to every route on the load balancer. | Algorithm | Behavior | | - | - | | **Least request** | Sends each request to the endpoint with the fewest requests in flight. The default when nothing is set. | | **Round robin** | Cycles through endpoints in order. | | **Random** | Picks an endpoint at random. | | **Consistent hash** | Sends requests that hash the same way to the same endpoint while the pool is stable. Hash on the client IP, or on a request header such as `x-user-id`. | > [!NOTE] > > With consistent hash on the client IP, a load test run from one machine sends every request to the same origin. That is the algorithm working. Test from several addresses, or hash on a header your test varies. ## Passive health checks Passive health checks watch real responses. When an endpoint returns a run of 5xx responses, the load balancer ejects it. The endpoint gets no requests for a while, then the load balancer lets it back in. If it is still failing, the load balancer ejects it again for longer each time. Health checks are off until you turn them on. When on, the defaults are: | Setting | Default | Meaning | | - | - | - | | Consecutive 5xx | 5 | 5xx responses in a row that eject an endpoint | | Base ejection | 30s | How long the first ejection lasts. Each repeat lasts longer. | | Max ejected % | 50 | The most of one origin's endpoints that can be ejected at once | There are no active probes. Datum never requests a health path, so an endpoint that gets no requests is never judged. ### What health checks do and do not do **They act inside one origin.** They move requests off a failing endpoint onto the same origin's other endpoints. A URL origin whose hostname resolves to several addresses has several endpoints. So does a compute workload with several instances. **They never move an origin's share to another origin.** Weights decide that split, whatever an origin's health. A broken origin keeps failing its share of requests until you drain or remove it. To send traffic to a fallback origin automatically when the primary fails, see [Failover](/docs/alb/configuration/failover). **An origin with one endpoint gains little.** Once fewer than half of an origin's endpoints are left, what happens depends on the route: * If every origin on the route is a URL, the load balancer spreads requests over all of that origin's endpoints again, failing ones included. * If any origin on the route is a compute workload, nothing is sent to the failing endpoints, and those requests fail straight away. ## In the cloud portal Open the load balancer and go to the **Backends** tab. * **Add an origin.** Select **Add backend**, pick an **Origin type** (**Public internet** or **Datum compute**), and set its **Weight**. The dialog shows the share the new origin will get. A new backend starts at the average weight of the pool, so it joins with an even share. * **Change a weight.** Use **Edit backend** on the row. **Drain backend** sets the weight to 0. * **Change the algorithm.** Use the **Algorithm** menu next to **Add backend**. Choosing **Consistent hash** asks what to hash on: **Client IP** or **Request header**. * **Turn on health checks.** Select **Edit** on the **Health checks** card, switch on **Outlier detection**, and adjust **Consecutive 5xx**, **Base ejection** and **Max ejected %** if you need to. The portal manages one route: the pool of origins on `/`. If a load balancer has more routes, a path match or a per-origin filter, the Backends tab is read-only, including the algorithm and health checks. Manage that load balancer with `datumctl alb` instead. ## With datumctl Weights are set per origin: ```bash theme={null} datumctl alb route backend add my-app --path / --endpoint https://api-2.example.com --weight 3 datumctl alb route backend update my-app --path / --endpoint https://api.example.com --weight 0 datumctl alb route backend list my-app ``` `route backend add` without `--weight` gives the new origin the average weight of the pool, like the portal. `route backend list` shows each origin's `WEIGHT` and `SHARE`. A change that would leave every origin on a route at weight 0 is refused. See [Send paths to different origins](/docs/alb/operations/routes) for routes and pools. The algorithm and health checks are set on the load balancer: ```bash theme={null} datumctl alb update my-app --algorithm round-robin datumctl alb update my-app --hash-header x-user-id datumctl alb update my-app --algorithm default datumctl alb update my-app --health-checks datumctl alb update my-app --consecutive-5xx 3 --base-ejection-time 1m --max-ejection-percent 50 datumctl alb update my-app --no-health-checks ``` `--algorithm` takes `round-robin`, `random`, `least-request` or `consistent-hash`. Plain `consistent-hash` hashes the client IP. `--hash-header` hashes a header and implies `consistent-hash`. `default` removes the setting, which leaves least request in place. Any tuning flag turns health checks on and keeps the values already set. `datumctl alb describe my-app` shows each origin's weight and share, the algorithm and the health check settings. > [!NOTE] > > These flags need a recent version of the `alb` plugin. Run `datumctl plugin upgrade alb` if `datumctl alb update --help` does not list `--algorithm`. ## In a manifest ```yaml theme={null} apiVersion: networking.datumapis.com/v1alpha kind: HTTPProxy metadata: name: my-app spec: rules: - backends: - endpoint: https://api.example.com weight: 3 - endpoint: https://api-2.example.com weight: 1 loadBalancer: type: ConsistentHash consistentHash: type: Header header: x-user-id healthCheck: passive: consecutive5xxErrors: 5 baseEjectionTime: 30s maxEjectionPercent: 50 ``` * `spec.rules[].backends[].weight` is 0 to 1,000,000 and defaults to 1. * `spec.loadBalancer.type` is `RoundRobin`, `Random`, `LeastRequest` or `ConsistentHash`. `consistentHash` is required for `ConsistentHash`, with `type: SourceIP` or `type: Header` and a `header`. Leave `loadBalancer` out for the default. * `spec.healthCheck.passive: {}` turns health checks on with the defaults. Leave `healthCheck` out to turn them off. # ALB configuration Source: https://datum.net/docs/alb/configuration/overview The settings that make up a Datum Application Load Balancer, and where to configure each one. An Application Load Balancer is configured through a few platform resources: an `HTTPProxy` for hostnames, routes, and origins, a `TrafficProtectionPolicy` for the web application firewall, and an optional `SecurityPolicy` for authentication. For how they fit together, see [What an ALB is made of](/docs/alb/overview#what-an-alb-is-made-of). In the [Cloud Portal](https://cloud.datum.net), a load balancer's **Configuration** tab gathers its settings in one place: general details, custom hostnames, the backend pool, TLS and certificates, security and the web application firewall, and access control. ![The Configuration tab of a load balancer with sections for General, Custom Hostnames, Backend pool, TLS and Certificates, Security and WAF in Enforce mode, and Access Control](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/alb-configuration.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=51afa854ba4a4b471485b3681552059e) A load balancer's settings in the Cloud Portal | To configure | See | | - | - | | The web application firewall | [Web application firewall](/docs/alb/configuration/waf) | | A wildcard hostname, such as `*.example.com` | [Wildcard hostnames](/docs/alb/configuration/wildcard-hostnames) | | Routes, redirects, rewrites, and headers on the proxy | [HTTPProxy](/docs/alb/httpproxy/overview) | | A custom domain with `datumctl` | [Serve traffic on your own domain](/docs/alb/operations/custom-domains) | | Several origins and path routes with `datumctl` | [Send paths to different origins](/docs/alb/operations/routes) | | Origin weights, the load balancing algorithm, and passive health checks | [Load balancing and health checks](/docs/alb/configuration/load-balancing) | | Basic or OIDC authentication | [Authentication](/docs/alb/authentication/overview) | | Failover between a primary and a fallback origin | [Failover](/docs/alb/configuration/failover) | # WAF Configuration with TrafficProtectionPolicy Source: https://datum.net/docs/alb/configuration/waf Enable and configure the Coraza OWASP WAF for a Datum Application Load Balancer using TrafficProtectionPolicy. When NSO reconciles an `HTTPProxy`, it creates a `gateway.networking.k8s.io/v1 HTTPRoute` with the same name and namespace. The `TrafficProtectionPolicy` targets this `HTTPRoute` — targeting the `HTTPProxy` directly is not supported. The WAF is powered by [Coraza](https://www.coraza.io/) and the [OWASP Core Rule Set](https://coreruleset.org/) (CRS). It inspects HTTP requests and responses for common attack patterns including SQL injection, XSS, and other OWASP Top 10 threats. *** ## Overview At a high level, this setup: 1. Creates a `TrafficProtectionPolicy` in `Observe` mode to baseline traffic without blocking 2. Reviews detected violations 3. Transitions to `Enforce` mode once the ruleset is tuned > [!NOTE] > > WAF policies default to `Observe` mode when `mode` is not specified. Always start in `Observe` mode before enforcing — this prevents unexpected blocking of legitimate traffic. *** ## Prerequisites * `datumctl` installed and authenticated * A valid **Project** * An existing `HTTPProxy` to attach the policy to Verify your proxy exists: ```bash theme={null} datumctl get httpproxy --project $PROJECT --namespace $NAMESPACE ``` *** ## Configuration steps ### Step 1: Set variables > [tab] macOS / Linux > > ```bash theme={null} > PROJECT="your-project-id" > NAMESPACE="default" > PROXY_NAME="your-proxy-name" > WAF_NAME="${PROXY_NAME}-waf" > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $PROJECT = "your-project-id" > $NAMESPACE = "default" > $PROXY_NAME = "your-proxy-name" > $WAF_NAME = "${PROXY_NAME}-waf" > ``` *** ### Step 2: Apply WAF in observe mode Start by attaching the WAF in `Observe` mode. Violations are logged but traffic is not blocked. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: networking.datumapis.com/v1alpha > kind: TrafficProtectionPolicy > metadata: > name: $WAF_NAME > namespace: $NAMESPACE > spec: > mode: Observe > ruleSets: > - type: OWASPCoreRuleSet > owaspCoreRuleSet: > paranoiaLevels: > detection: 2 > blocking: 1 > scoreThresholds: > inbound: 5 > outbound: 4 > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $PROXY_NAME > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: networking.datumapis.com/v1alpha > kind: TrafficProtectionPolicy > metadata: > name: $WAF_NAME > namespace: $NAMESPACE > spec: > mode: Observe > ruleSets: > - type: OWASPCoreRuleSet > owaspCoreRuleSet: > paranoiaLevels: > detection: 2 > blocking: 1 > scoreThresholds: > inbound: 5 > outbound: 4 > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $PROXY_NAME > "@ | datumctl apply --project $PROJECT --namespace $NAMESPACE -f - > ``` *** ### Step 3: Verify policy attachment ```bash theme={null} datumctl get trafficprotectionpolicy $WAF_NAME \ --project $PROJECT --namespace $NAMESPACE -o yaml ``` Confirm the policy shows `Accepted=True` and is attached to the correct proxy. *** ### Step 4: Switch to enforce mode After observing traffic and confirming no false positives, switch the mode to `Enforce`. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: networking.datumapis.com/v1alpha > kind: TrafficProtectionPolicy > metadata: > name: $WAF_NAME > namespace: $NAMESPACE > spec: > mode: Enforce > ruleSets: > - type: OWASPCoreRuleSet > owaspCoreRuleSet: > paranoiaLevels: > detection: 2 > blocking: 1 > scoreThresholds: > inbound: 5 > outbound: 4 > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $PROXY_NAME > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: networking.datumapis.com/v1alpha > kind: TrafficProtectionPolicy > metadata: > name: $WAF_NAME > namespace: $NAMESPACE > spec: > mode: Enforce > ruleSets: > - type: OWASPCoreRuleSet > owaspCoreRuleSet: > paranoiaLevels: > detection: 2 > blocking: 1 > scoreThresholds: > inbound: 5 > outbound: 4 > targetRefs: > - group: gateway.networking.k8s.io > kind: HTTPRoute > name: $PROXY_NAME > "@ | datumctl apply --project $PROJECT --namespace $NAMESPACE -f - > ``` *** ## Configuration reference ### Paranoia levels Paranoia level controls how aggressively the CRS applies rules. Higher levels catch more threats but increase false-positive risk. | Level | Behavior | | - | - | | 1 | Core rules only. Lowest false-positive risk. Recommended starting point. | | 2 | Adds more rules. Suitable for most production applications. | | 3 | Stricter rules. May require tuning for complex applications. | | 4 | Maximum coverage. High false-positive risk without tuning. | `detection` and `blocking` can be set independently. A common pattern is to detect at a higher level than you block, allowing you to see what stricter rules would catch before enforcing them. *** ### Score thresholds The OWASP CRS uses anomaly scoring — each rule match adds to a running score. The request is blocked when the score exceeds the threshold. | Field | Default | Description | | - | - | - | | `inbound` | 5 | Score threshold to block an inbound request | | `outbound` | 4 | Score threshold to block an outbound response | Lower thresholds block more aggressively. The defaults (5 inbound, 4 outbound) are the OWASP CRS recommended starting values. *** ### Rule exclusions Use rule exclusions to suppress specific rules causing false positives. Exclusions apply globally to the policy. ```yaml theme={null} owaspCoreRuleSet: ruleExclusions: ids: - 920350 # Disable a specific rule by ID idRanges: - "941100-941200" # Disable a range of rules tags: - "attack-sqli" # Disable all rules with a tag ``` Use the minimum exclusion scope needed — prefer specific `ids` over broad `tags`. *** ### Targeting a specific rule (sectionName) By default, a `TrafficProtectionPolicy` applies to the entire `HTTPProxy`. To scope it to a single named rule, use `sectionName`: ```yaml theme={null} targetRefs: - group: gateway.networking.k8s.io kind: HTTPRoute name: your-proxy-name sectionName: your-rule-name ``` `sectionName` must match the `name` field of a rule in the `HTTPProxy`. NSO creates the `HTTPRoute` with matching rule names. *** ## Verification ### Test with a known attack pattern Send a request containing a SQL injection pattern to confirm the WAF detects it: ```bash theme={null} curl -sS -o /dev/null -w "%{http_code}\n" -G \ --data-urlencode "id=1' OR '1'='1" \ https://your-app.example.com/ ``` In `Observe` mode: request passes through, violation is logged. In `Enforce` mode: expected response is `403 Forbidden`. *** ## Cleanup > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete trafficprotectionpolicy $WAF_NAME \ > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete trafficprotectionpolicy $WAF_NAME ` > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` *** ## Troubleshooting | Symptom | Root Cause | Resolution | | - | - | - | | Policy not attaching | `targetRefs.group` incorrect | Use `gateway.networking.k8s.io` | | Policy not attaching | `targetRefs.kind` incorrect | Use `HTTPRoute` (not `HTTPProxy`) | | Legitimate traffic blocked | Paranoia level too high | Lower `blocking` level or add rule exclusions | | Known attack not blocked | Mode is `Observe` | Switch `mode` to `Enforce` | | Known attack not blocked | Score threshold too high | Lower `inbound` threshold | | `sectionName` not found | Rule name mismatch | Verify rule `name` in the `HTTPProxy` spec | *** ## Best practices * Always start in `Observe` mode and run for at least a few days of representative traffic before enforcing * Set `detection` one level higher than `blocking` to preview the impact of stricter rules before applying them * Use specific rule `ids` for exclusions rather than broad `tags` — this minimizes the attack surface opened by exclusions * Keep `scoreThresholds.inbound` at 5 or higher initially; tuning downward after false positives are addressed * Apply one policy per proxy, targeting with `sectionName` when you need different rules for different routes *** ## Summary * WAF policies use `kind: TrafficProtectionPolicy` at `networking.datumapis.com/v1alpha` * `mode` defaults to `Observe` — always confirm before assuming enforcement is active * `ruleSets[].type` must be `OWASPCoreRuleSet` — it is currently the only supported ruleset type * `targetRefs` must use `group: gateway.networking.k8s.io` and `kind: HTTPRoute` — NSO creates an `HTTPRoute` with the same name as the `HTTPProxy`; use `sectionName` to scope to a single route rule * Start with paranoia level 1 blocking, level 2 detection, and OWASP default score thresholds (5/4) # Serve a wildcard hostname Source: https://datum.net/docs/alb/configuration/wildcard-hostnames Attach a wildcard hostname such as *.example.com to a Datum Application Load Balancer and get a certificate through a DNS record. A wildcard hostname lets one load balancer serve every name under a domain you own, such as `app1.example.com` and `app2.example.com`, with one certificate. Use it when you serve many subdomains and do not want to attach each one. > [!NOTE] > > Datum enables wildcard hostnames per project. Contact Datum to turn them on for yours. ## How it differs from an exact hostname | | Exact hostname | Wildcard hostname | | - | - | - | | Example | `app.example.com` | `*.example.com` | | Certificate | Issued once the name points at Datum | Issued after you add one `_acme-challenge` CNAME | | Domain ownership | [As in the custom domain guide](/docs/alb/operations/custom-domains) | Proven by a DNS TXT record | | Enabled by | Nothing extra | Datum, per project | A wildcard covers names one label down. `*.example.com` serves `app.example.com` but not `a.b.example.com`. It also reserves every name beneath it for your project, so another project cannot attach one of those names. ## Attach it Quote the wildcard so your shell leaves the `*` alone. ```bash theme={null} datumctl alb hostname add my-app '*.example.com' datumctl alb describe my-app ``` `hostname add` does not wait. If the wildcard is refused, `describe` shows the reason under the hostname. ## Publish the DNS records `describe` ends with the records you still need to create at your DNS provider. ```text theme={null} DNS records to publish: NAME TYPE CONTENT PURPOSE *.example.com CNAME my-app-hrkgk.datumproxy.net Routing _acme-challenge.example.com CNAME k3f9q2x7.acme-dns.example.net Certificate ``` | Purpose | What it does | | - | - | | Ownership | A TXT record proving you own the domain. A wildcard needs the domain verified by DNS; a domain verified another way does not count. | | Certificate | Delegates the certificate challenge to Datum, so the certificate is issued before traffic moves. | | Routing | Sends traffic for the wildcard to the load balancer. | Create each record exactly as shown. The values above are examples; use the ones `describe` prints for your load balancer. ## See the status The same information is in the `HTTPProxy` status. This example shows a wildcard whose certificate is waiting on the project being enabled. ```yaml theme={null} apiVersion: networking.datumapis.com/v1alpha kind: HTTPProxy metadata: name: my-app spec: hostnames: - "*.example.com" rules: - backends: - endpoint: https://origin.example.com status: hostnameStatuses: - hostname: "*.example.com" conditions: - type: Available status: "True" reason: Claimed - type: Verified status: "True" reason: Verified - type: CertificateReady status: "False" reason: WildcardNotEntitled message: >- Wildcard hostnames are not enabled for this project, so no certificate is issued. Contact Datum to enable them, or use an exact hostname. dnsRecords: - name: "*.example.com" type: CNAME content: my-app-hrkgk.datumproxy.net purpose: Routing managedBy: User state: Missing - name: _acme-challenge.example.com type: CNAME content: k3f9q2x7.acme-dns.example.net purpose: Certificate managedBy: User state: Missing ``` Each record's `state` is `Present` once it takes effect on the public internet, and `Missing` while it is absent or wrong. `managedBy` is `Platform` for records Datum publishes in a Datum DNS zone, and `User` for records you publish. Once everything is in place, `CertificateReady` becomes `True` with reason `CertificateIssued`. ## When the certificate is not issued | `describe` shows | Cause | Fix | | - | - | - | | Wildcard hostnames are not enabled for this project | The project is not enabled | Contact Datum, or use an exact hostname | | Verified is `False` | The domain is not yet proven by its DNS TXT record | Publish the Ownership record | | `_acme-challenge` record is `Missing` | The Certificate CNAME is absent or wrong | Publish it exactly as shown, without the zone name appended twice | Exact hostnames on the same load balancer are unaffected and keep their own certificates. ## If wildcards are later disabled A certificate that has already been issued keeps serving. Disabling wildcards for a project stops new wildcard certificates from being issued, not existing ones. # Host Header Override with HTTPProxy Source: https://datum.net/docs/alb/httpproxy/host-header-override Rewrite the Host header on upstream requests using an HTTPProxy RequestHeaderModifier filter. This is useful when: * The upstream expects a specific `Host` header * You are proxying to a shared or virtual-hosted service * The backend's DNS name does not match the public hostname *** ## Overview At a high level, this setup: 1. Accepts public traffic at a configured hostname 2. Forwards all requests to an upstream backend 3. Rewrites the `Host` header **before** the request reaches the upstream *** ## Prerequisites * `datumctl` installed and authenticated * A valid **Project** *** ## Configuration steps ### Step 1: Set variables > [tab] macOS / Linux > > ```bash theme={null} > PROJECT="your-project-id" > NAMESPACE="default" > PROXY_NAME="host-header-demo" > HOSTNAME="your-app.example.com" > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $PROJECT = "your-project-id" > $NAMESPACE = "default" > $PROXY_NAME = "host-header-demo" > $HOSTNAME = "your-app.example.com" > ``` *** ### Step 2: Apply host header configuration The `RequestHeaderModifier` filter overwrites the `Host` header before the request is forwarded upstream. The `type` field is required. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: networking.datumapis.com/v1alpha > kind: HTTPProxy > metadata: > name: $PROXY_NAME > spec: > hostnames: > - $HOSTNAME > rules: > - name: host-header-override > matches: > - path: > type: PathPrefix > value: / > filters: > - type: RequestHeaderModifier > requestHeaderModifier: > set: > - name: Host > value: example.internal > backends: > - endpoint: https://httpbin.org > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: networking.datumapis.com/v1alpha > kind: HTTPProxy > metadata: > name: $PROXY_NAME > spec: > hostnames: > - $HOSTNAME > rules: > - name: host-header-override > matches: > - path: > type: PathPrefix > value: / > filters: > - type: RequestHeaderModifier > requestHeaderModifier: > set: > - name: Host > value: example.internal > backends: > - endpoint: https://httpbin.org > "@ | datumctl apply --project $PROJECT --namespace $NAMESPACE -f - > ``` *** ## Verification `httpbin.org/headers` echoes the request headers received by the upstream, making it easy to confirm the rewrite. ```bash theme={null} curl https://$HOSTNAME/headers ``` Look for `Host` in the response: ```json theme={null} { "headers": { "Host": "example.internal", ... } } ``` If `Host` shows `example.internal`, the rewrite is working correctly. *** ## Cleanup > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete httpproxy $PROXY_NAME \ > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete httpproxy $PROXY_NAME ` > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` *** ## Troubleshooting | Symptom | Root Cause | Resolution | | - | - | - | | API rejects the resource | `type` field missing from filter | Add `type: RequestHeaderModifier` to the filter | | `Host` header not rewritten | Filter applied at wrong level | Confirm `filters` is under `rules[].filters`, not `backends[].filters` | | Upstream returns 404 or 421 | Upstream rejects the rewritten `Host` value | Verify the upstream accepts the hostname you configured | | `Host` shows original value | `set` vs `add` confusion | Use `set` to overwrite; `add` appends to an existing value | *** ## Best practices * Use `set` to overwrite the `Host` header — `add` appends to any existing value instead of replacing it * Keep the rewritten hostname consistent with what the upstream backend expects in its virtual host configuration * Test with `httpbin.org/headers` before pointing at a real upstream — it removes backend variables from the equation *** ## Summary * Host header rewriting uses a `RequestHeaderModifier` filter on the rule's `filters` list * The `type: RequestHeaderModifier` field is required — omitting it will cause the API to reject the resource * `set` overwrites the header; `add` appends * The rewrite happens at the proxy — the client sees no change to the URL or response # HTTP to HTTPS Redirect with HTTPProxy Source: https://datum.net/docs/alb/httpproxy/https-redirect Force HTTPS by redirecting all plain HTTP requests using a RequestRedirect filter on a Datum HTTPProxy. > [!NOTE] > > A load balancer created with `datumctl alb create` forces HTTPS by default, because the command adds a redirect rule like the one on this page. An `HTTPProxy` that you write yourself doesn't redirect unless you add the rule. This guide applies when you manage the `HTTPProxy` directly, or when you need a custom redirect — for example, from a specific path or with a different status code. > > The rule matches only requests whose `x-forwarded-proto` header is `http`. The load balancer listens on both HTTP and HTTPS, so a rule that matched every request would also redirect HTTPS requests to themselves. *** ## Overview A `RequestRedirect` filter responds to the client with a redirect response directly — no backend is reached. The [Application Load Balancer](/docs/alb/overview) handles the redirect at the proxy layer. At a high level, this setup: 1. Matches all incoming requests on a hostname 2. Returns a `301 Moved Permanently` redirect to the same URL using `https` *** ## Prerequisites * `datumctl` installed and authenticated * A valid **Project** * An existing `HTTPProxy` hostname, or create a new one with this guide *** ## Configuration steps ### Step 1: Set variables > [tab] macOS / Linux > > ```bash theme={null} > PROJECT="your-project-id" > NAMESPACE="default" > PROXY_NAME="https-redirect-demo" > HOSTNAME="your-app.example.com" > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $PROJECT = "your-project-id" > $NAMESPACE = "default" > $PROXY_NAME = "https-redirect-demo" > $HOSTNAME = "your-app.example.com" > ``` *** ### Step 2: Apply the redirect configuration The `RequestRedirect` filter requires `type: RequestRedirect`. No backend is needed — the redirect is returned immediately by the proxy. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: networking.datumapis.com/v1alpha > kind: HTTPProxy > metadata: > name: $PROXY_NAME > spec: > hostnames: > - $HOSTNAME > rules: > - name: https-redirect > matches: > - path: > type: PathPrefix > value: / > headers: > - name: x-forwarded-proto > type: Exact > value: http > filters: > - type: RequestRedirect > requestRedirect: > scheme: https > statusCode: 301 > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: networking.datumapis.com/v1alpha > kind: HTTPProxy > metadata: > name: $PROXY_NAME > spec: > hostnames: > - $HOSTNAME > rules: > - name: https-redirect > matches: > - path: > type: PathPrefix > value: / > headers: > - name: x-forwarded-proto > type: Exact > value: http > filters: > - type: RequestRedirect > requestRedirect: > scheme: https > statusCode: 301 > "@ | datumctl apply --project $PROJECT --namespace $NAMESPACE -f - > ``` *** ## Verification Test that a plain HTTP request receives a redirect: ```bash theme={null} curl -I http://$HOSTNAME/ ``` Expected response: ``` HTTP/1.1 301 Moved Permanently Location: https://your-app.example.com/ ``` Confirm that the `Location` header uses `https` and preserves the path. *** ## Cleanup > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete httpproxy $PROXY_NAME \ > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete httpproxy $PROXY_NAME ` > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` *** ## Troubleshooting | Symptom | Root Cause | Resolution | | - | - | - | | API rejects the resource | `type` field missing from filter | Add `type: RequestRedirect` to the filter | | Redirect loop | Both HTTP and HTTPS rules redirect | Ensure the redirect rule only targets HTTP traffic | | `Location` header has wrong hostname | Hostname not set | Add `hostname` to `requestRedirect` to override | | Client receives `302` instead of `301` | `statusCode` is unset (the default is `302`) or set to `302` | Set `statusCode: 301` for a permanent redirect | *** ## Best practices * Use `statusCode: 301` for permanent redirects — browsers and crawlers cache these * Use `statusCode: 302` only when the redirect is temporary (e.g., maintenance) * If redirecting to a different hostname, set `requestRedirect.hostname` explicitly rather than relying on the `Host` header *** ## Summary * HTTP to HTTPS redirects use `type: RequestRedirect` with `scheme: https` * The `type` field is required on all filters * No backend is needed — `RequestRedirect` is a terminal filter that responds to the client directly * `statusCode` accepts `301`, `302`, `303`, `307`, or `308`, and defaults to `302` # HTTPProxy and WAF Capabilities Source: https://datum.net/docs/alb/httpproxy/overview Reference overview of the HTTPProxy and TrafficProtectionPolicy resources that make up Datum's Application Load Balancer. > [!NOTE] > > **Working with this resource via an AI agent?** Datum publishes an [HTTPRoute skill](/docs/agents/skills/httproute) for path routing, traffic splitting, redirects, and rewrites, and an [Application Load Balancer skill](/docs/agents/skills/alb) for the load balancer as a whole. Datum's Application Load Balancer consists of two primary resources that work together (plus operator-provisioned Gateway, DNS, and TLS): * **`HTTPProxy`** — Layer 7 routing and traffic control * **`TrafficProtectionPolicy`** — [OWASP Core Rule Set](https://coreruleset.org/) WAF protection Together with the Gateway the network-services operator creates from the HTTPProxy (same name), these are what make up an ALB today. Optional basic auth adds a `SecurityPolicy` and Secret. See the [overview](/docs/alb/overview#what-an-alb-is-made-of) for the full composition diagram. The proxy determines where traffic goes and how requests are shaped before reaching your origin. The WAF determines whether traffic is observed or blocked based on security rules. The WAF attaches to a proxy using `targetRefs`. *** ## HTTPProxy The `HTTPProxy` resource handles Layer 7 routing and request processing. It supports hostname routing, path and header matching, redirects, rewrites, CORS, header manipulation, weighted traffic splitting, and TLS configuration to origin. ### Inspect the schema ```bash theme={null} datumctl explain httpproxy --recursive ``` ### List and inspect proxies ```bash theme={null} # List all proxies datumctl get httpproxies --namespace default # View a specific proxy datumctl get httpproxy --namespace default -o yaml ``` ### Feature reference ```text theme={null} HTTPProxy ├── metadata │ ├── name │ ├── namespace │ └── annotations │ └── spec ├── hostnames[] │ └── rules[] ├── name │ ├── matches[] │ ├── path │ │ ├── type (Exact | PathPrefix | RegularExpression) │ │ └── value │ ├── headers[] │ │ ├── name │ │ ├── type (Exact | RegularExpression) │ │ └── value │ ├── queryParams[] │ └── method (GET | POST | PUT | ...) │ ├── filters[] (rule-level) │ ├── RequestRedirect │ ├── RequestHeaderModifier │ ├── ResponseHeaderModifier │ ├── URLRewrite │ ├── CORS │ └── ExtensionRef │ └── backends[] ├── endpoint ├── connector ├── tls └── filters[] (backend-level) ``` > [!NOTE] > > A rule can list up to 16 backends, and each backend's `weight` (default `1`) sets its share of traffic. A `connector` backend must be the only backend in its rule. *** ## TrafficProtectionPolicy (WAF) The `TrafficProtectionPolicy` resource provides application-layer security using the OWASP Core Rule Set. It attaches to the `gateway.networking.k8s.io/v1 HTTPRoute` that NSO creates from an `HTTPProxy` (using the same name), scoping the WAF to that proxy's traffic. Use `sectionName` to target a specific named rule. ### Inspect the schema ```bash theme={null} datumctl explain trafficprotectionpolicy --recursive ``` ### List and inspect WAF policies ```bash theme={null} # List all WAF policies datumctl get trafficprotectionpolicies --namespace default # View a specific policy datumctl get trafficprotectionpolicy --namespace default -o yaml ``` ### Feature reference ```text theme={null} TrafficProtectionPolicy ├── metadata │ └── spec ├── mode (Observe | Enforce | Disabled) ├── samplingPercentage ├── ruleSets[] │ └── OWASPCoreRuleSet │ ├── paranoiaLevels │ │ ├── detection │ │ └── blocking │ ├── scoreThresholds │ │ ├── inbound │ │ └── outbound │ └── ruleExclusions │ ├── ids │ ├── idRanges │ └── tags │ └── targetRefs[] ├── group ├── kind ├── name └── sectionName (optional — target a specific rule) ``` **Mode values:** * `Observe` *(default)* — Logs rule matches without blocking traffic. Use this to evaluate impact before enforcing. * `Enforce` — Blocks requests that exceed the score threshold. * `Disabled` — WAF is inactive. **Paranoia levels** control how aggressively rules are applied. Higher levels catch more threats but increase false-positive risk. Separate levels can be set for detection (logging) and blocking. **`samplingPercentage`** controls what fraction of traffic is evaluated by the WAF. Useful for gradual rollout or high-throughput environments. *** ## Next steps * [Path-Based Routing](/docs/alb/httpproxy/path-routing) — Route requests to different backends by URL path * [HTTP Basic Authentication](/docs/alb/authentication/basic-auth) — Add username/password protection to a route * [OIDC with Google](/docs/alb/authentication/oidc-google) — Protect a route with Google sign-in * [OIDC with Auth0](/docs/alb/authentication/oidc-auth0) — OIDC with email-based allow-lists via Auth0 # Path-Based Routing with HTTPProxy Source: https://datum.net/docs/alb/httpproxy/path-routing Route requests to different backends based on URL paths using a Datum HTTPProxy. The example routes one exact path to a dedicated backend and sends all remaining traffic to a default backend. *** ## Overview Path-based routing lets you split traffic across backends based on the URL path, without changing your application or DNS. At a high level, this setup: 1. Defines an **exact** path match for a specific route (e.g., `/login`) 2. Adds a **catch-all** prefix rule for all other traffic 3. Applies both rules in a single `HTTPProxy` resource *** ## Prerequisites * `datumctl` installed and authenticated * A valid **Project** Verify access: ```bash theme={null} datumctl get httpproxy ``` *** ## Configuration steps ### Step 1: Set variables > [tab] macOS / Linux > > ```bash theme={null} > PROJECT="your-project-id" > NAMESPACE="default" > PROXY_NAME="my-proxy" > HOSTNAME="app.example.com" > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $PROJECT = "your-project-id" > $NAMESPACE = "default" > $PROXY_NAME = "my-proxy" > $HOSTNAME = "app.example.com" > ``` *** ### Step 2: Apply path routing configuration The `/login` path is matched exactly. All other requests fall through to the catch-all `/` rule. Place the exact match rule **before** the catch-all. Rules are evaluated in order, and a prefix of `/` matches everything. > [!NOTE] > > A rule can list up to 16 backends with a `weight` on each to split traffic. This guide uses one backend per rule. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: networking.datumapis.com/v1alpha > kind: HTTPProxy > metadata: > name: $PROXY_NAME > spec: > hostnames: > - $HOSTNAME > rules: > - name: login-exact > matches: > - path: > type: Exact > value: /login > backends: > - endpoint: https://auth-origin.example > > - name: catch-all > matches: > - path: > type: PathPrefix > value: / > backends: > - endpoint: https://web-origin.example > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: networking.datumapis.com/v1alpha > kind: HTTPProxy > metadata: > name: $PROXY_NAME > spec: > hostnames: > - $HOSTNAME > rules: > - name: login-exact > matches: > - path: > type: Exact > value: /login > backends: > - endpoint: https://auth-origin.example > > - name: catch-all > matches: > - path: > type: PathPrefix > value: / > backends: > - endpoint: https://web-origin.example > "@ | datumctl apply --project $PROJECT --namespace $NAMESPACE -f - > ``` *** ## Verification ### Exact path match ```bash theme={null} curl -v https://app.example.com/login ``` Expected: request reaches `auth-origin.example`. *** ### Negative test: exact is truly exact ```bash theme={null} curl -v https://app.example.com/login/reset ``` Expected: `/login/reset` does **not** match the `Exact /login` rule and falls through to the catch-all backend. Verify this to confirm the exact match boundary is working correctly. *** ### Catch-all ```bash theme={null} curl -v https://app.example.com/ ``` Expected: request reaches `web-origin.example`. *** ## Cleanup > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete httpproxy $PROXY_NAME \ > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete httpproxy $PROXY_NAME ` > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` *** ## Best practices * Always include a `PathPrefix /` catch-all so no requests are dropped unexpectedly * Place more-specific rules (e.g., `Exact`, longer prefixes) before less-specific ones * Use distinct, descriptive rule `name` values — they appear in logs and aid debugging * Test both the matching path and a near-miss (e.g., `/login/reset`) to confirm exact boundaries *** ## Summary * Path-based routing is configured using a **`HTTPProxy`** resource at `networking.datumapis.com/v1alpha` * Use `type: Exact` for single endpoints like `/login` * Use `type: PathPrefix` with value `/` as a catch-all for all remaining traffic * Rule order matters — place specific matches before the catch-all # URL Rewrite with HTTPProxy Source: https://datum.net/docs/alb/httpproxy/url-rewrite Rewrite the path or hostname on upstream requests using a URLRewrite filter on a Datum HTTPProxy. URL rewriting is useful when: * Your public API path structure differs from your backend's internal paths * You want to strip a path prefix before forwarding (e.g., `/api/v1/users` → `/users`) * You need to replace the full path for a specific route * You need to rewrite the hostname the backend sees (separate from `Host` header modification) *** ## Overview The `URLRewrite` filter modifies the path and/or hostname of the forwarded request. Unlike `RequestHeaderModifier`, which sets arbitrary headers, `URLRewrite` is purpose-built for path and authority rewriting and is the recommended approach for path manipulation. At a high level, this setup: 1. Matches requests on a path prefix 2. Rewrites the path before forwarding to the backend 3. The client sees no change to the URL *** ## Prerequisites * `datumctl` installed and authenticated * A valid **Project** *** ## Configuration steps ### Step 1: Set variables > [tab] macOS / Linux > > ```bash theme={null} > PROJECT="your-project-id" > NAMESPACE="default" > PROXY_NAME="url-rewrite-demo" > HOSTNAME="your-app.example.com" > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $PROJECT = "your-project-id" > $NAMESPACE = "default" > $PROXY_NAME = "url-rewrite-demo" > $HOSTNAME = "your-app.example.com" > ``` *** ### Step 2: Apply URL rewrite configuration This example strips the `/api` prefix. Requests to `/api/users` are forwarded to the backend as `/users`. `ReplacePrefixMatch` must be paired with a `PathPrefix` match on the same prefix — mixing match types will cause the route to be rejected. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: networking.datumapis.com/v1alpha > kind: HTTPProxy > metadata: > name: $PROXY_NAME > spec: > hostnames: > - $HOSTNAME > rules: > - name: strip-api-prefix > matches: > - path: > type: PathPrefix > value: /api > filters: > - type: URLRewrite > urlRewrite: > path: > type: ReplacePrefixMatch > replacePrefixMatch: / > backends: > - endpoint: https://api-backend.example.com > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: networking.datumapis.com/v1alpha > kind: HTTPProxy > metadata: > name: $PROXY_NAME > spec: > hostnames: > - $HOSTNAME > rules: > - name: strip-api-prefix > matches: > - path: > type: PathPrefix > value: /api > filters: > - type: URLRewrite > urlRewrite: > path: > type: ReplacePrefixMatch > replacePrefixMatch: / > backends: > - endpoint: https://api-backend.example.com > "@ | datumctl apply --project $PROJECT --namespace $NAMESPACE -f - > ``` *** ## Path rewrite options ### Replace prefix Replaces the matched prefix and preserves the rest of the path. ```yaml theme={null} filters: - type: URLRewrite urlRewrite: path: type: ReplacePrefixMatch replacePrefixMatch: /new-prefix ``` | Incoming Path | Match Prefix | Replace With | Forwarded Path | | - | - | - | - | | `/api/users` | `/api` | `/` | `/users` | | `/api/orders/123` | `/api` | `/` | `/orders/123` | | `/api/v2/items` | `/api/v2` | `/v1` | `/v1/items` | The match rule's `path.value` and `replacePrefixMatch` must use the same prefix. Using `ReplacePrefixMatch` with an `Exact` or `RegularExpression` match will cause the route to be rejected. *** ### Replace full path Replaces the entire path, ignoring what was matched. ```yaml theme={null} filters: - type: URLRewrite urlRewrite: path: type: ReplaceFullPath replaceFullPath: /health ``` | Incoming Path | Forwarded Path | | - | - | | `/status` | `/health` | | `/ping` | `/health` | | `/anything` | `/health` | Useful for mapping multiple public paths to a single backend endpoint. *** ### Hostname rewrite Rewrites the authority (hostname) the backend sees without changing request headers separately. ```yaml theme={null} filters: - type: URLRewrite urlRewrite: hostname: internal-api.example.com ``` `hostname` and `path` can be combined in a single `URLRewrite` filter. *** ## Verification `httpbin.org/anything` reflects the path received by the upstream. Use it to confirm the rewrite before pointing at a real backend. Update your rule's `backends[].endpoint` to `https://httpbin.org` temporarily, then: ```bash theme={null} curl https://$HOSTNAME/api/users ``` Look for the `url` field in the response — it should show `/users`, not `/api/users`. *** ## Cleanup > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete httpproxy $PROXY_NAME \ > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete httpproxy $PROXY_NAME ` > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` *** ## Troubleshooting | Symptom | Root Cause | Resolution | | - | - | - | | Route rejected (`Accepted=False`) | `ReplacePrefixMatch` used with non-prefix match | Change match `type` to `PathPrefix` | | Path not rewritten | `type` field missing from `urlRewrite.path` | Add `type: ReplacePrefixMatch` or `type: ReplaceFullPath` | | API rejects the resource | `type` field missing from filter | Add `type: URLRewrite` to the filter | | Prefix partially matched | Prefix match is not path-element-aware | `/api` matches `/api/`, `/api/foo` but not `/apiv2` — this is correct behavior | *** ## Best practices * Use `ReplacePrefixMatch` for stripping versioned path prefixes (e.g., `/v1`, `/api`) * Use `ReplaceFullPath` when mapping multiple routes to a single backend endpoint * Test rewrites with `httpbin.org/anything` before switching to a production backend — it reflects the exact path the upstream receives * Keep match prefix and `replacePrefixMatch` prefix consistent to avoid rejected routes *** ## Summary * URL path rewriting uses `type: URLRewrite` with a `path` block * `path.type` is required: `ReplacePrefixMatch` or `ReplaceFullPath` * `ReplacePrefixMatch` must be paired with a `PathPrefix` match on the same prefix * `hostname` can be set alongside `path` in the same filter * The rewrite is transparent to the client — only the upstream sees the modified path # ALB limitations and roadmap Source: https://datum.net/docs/alb/limitations-and-roadmap Review what the Datum Application Load Balancer doesn't support or document yet, and where to follow what comes next. The Application Load Balancer is still being built. This page lists known limitations and where to follow what comes next. ## Limitations * There is no single composite ALB API yet. An ALB is an `HTTPProxy`, a `TrafficProtectionPolicy`, and optional authentication resources. See [What an ALB is made of](/docs/alb/overview#what-an-alb-is-made-of). * Public traffic is HTTP and HTTPS only. Raw TCP and UDP exposure isn't available. * Rule exclusions, score thresholds, rate limiting, and circuit breaking aren't available through `datumctl alb`. Use manifests. See [Web application firewall](/docs/alb/configuration/waf). * A `connector` backend must be the only backend in its rule. * You can't choose which Datum locations serve a load balancer or set its capacity. [Envoy](/docs/platform/envoy) in every location serves every load balancer, and Datum scales the fleet. * Envoy connects to Compute Instances over plain HTTP. TLS ends at the load balancer. * Status shows when a change is published, not when every location serves it, and there is no per-location status. See [When a load balancer is not serving](/docs/alb/troubleshooting). * Request timeouts aren't documented yet, including how long a streaming response can stay open: Server-Sent Events (SSE), WebSockets, or a long MCP response. `HTTPProxy` has no timeout setting, so test long-lived responses before you rely on them. * The headers that Datum adds to requests it forwards to your origin, such as the client's IP address (`X-Forwarded-For`) and the original protocol (`X-Forwarded-Proto`), aren't documented yet. * `ClientTrafficPolicy` isn't available in projects, so client-side TLS versions, HTTP versions, and connection limits can't be changed. * For limits on load balancers attached to Compute workloads, see [Compute and ALB](/docs/compute/compute-and-alb#limitations). ## Roadmap For what Datum is building next across the platform, see the [Platform roadmap](/docs/platform/roadmap). > [!NOTE] > > Plans can change. This list isn't a commitment to deliver any feature or to meet any timeline. # ALB limits and quotas Source: https://datum.net/docs/alb/limits-and-quotas The fixed limits for an Application Load Balancer's HTTPProxy: hostnames, rules, matches, backends, and weights. This page lists the fixed limits of the `HTTPProxy` resource behind every [Application Load Balancer](/docs/alb/overview). Quotas for load balancers are not documented yet. ## Limits | Item | Limit | | - | - | | Hostnames for each `HTTPProxy` | 16 | | Rules for each `HTTPProxy` | 16 | | Matches for each rule | 64 | | Matches across all rules | 128 | | Backends for each rule | 16 | | Backend `weight` | 0 to 1,000,000, default `1` | | ALBs attached to each workload with `datumctl compute deploy` | 1, on one port | For features that load balancers don't support yet, see [Limitations and roadmap](/docs/alb/limitations-and-roadmap). # ALB observability Source: https://datum.net/docs/alb/observability Status, metrics, and access logs for Datum Application Load Balancers. This page covers how to see what an [Application Load Balancer](/docs/alb/overview) is doing. [Envoy](/docs/platform/envoy) in every Datum location serves your load balancer, and the views on this page bring what each location sees together in one place. ## Status and metrics The load balancer's overview in the [Cloud Portal](https://cloud.datum.net) shows whether it is serving, the live request rate, error rate, latency, and requests blocked by the firewall. Its **Metrics** tab breaks these down by location, response code, and firewall rule. With `datumctl`, run `datumctl alb describe ALB_NAME` for status, hostnames, and certificates: ```text theme={null} $ datumctl alb describe api Name: api Status: Active Status detail: The HTTPProxy has been programmed Default hostname: bright-oak-4f7k2.datumproxy.net Force HTTPS: yes Routes: / api:http (network-service) Traffic protection: Enforce (paranoia 1) Certificates: True (AllCertificatesReady) ``` Datum also publishes the measurements behind these charts to your project's metrics store. To query them in your own tools, [export them to Grafana Cloud](/docs/operations/metrics-export) or any Prometheus remote-write endpoint. ## Metrics ### Traffic and latency Envoy, the proxy that serves your load balancer, records every request it sends to your origin. | Metric | Type | What it measures | | - | - | - | | `envoy_vhost_vcluster_upstream_rq` | Counter | Requests sent to your origin, with the response code. Use `rate()` for requests per second. | | `envoy_vhost_vcluster_upstream_rq_time_bucket` | Histogram | Time for your origin to respond, in milliseconds. Use `histogram_quantile()` for percentiles. | | Label | Meaning | | - | - | | `gateway_name` | The load balancer's name, as shown by `datumctl alb list`. | | `envoy_response_code` | The HTTP response code, such as `200` or `503`. | | `label_topology_kubernetes_io_region` | The Datum location that served the request, such as `us-west-1`. | | `resourcemanager_datumapis_com_project_name` | Your project ID. | ### Firewall The [web application firewall](/docs/alb/configuration/waf) records one event for each request that matches a rule. | Metric | Type | What it measures | | - | - | - | | `coraza_envoy_filter_request_events_total` | Counter | Firewall events, with the outcome and the rule that matched. | | Label | Meaning | | - | - | | `coraza_outcome` | `allowed`, `blocked`, or `dropped`. | | `coraza_rule_id`, `coraza_rule_file` | The [OWASP Core Rule Set](https://coreruleset.org/) rule that matched, and the file it comes from. | | `coraza_rule_severity` | The rule's severity, such as `critical`. | | `http_method` | The request's HTTP method. | | `label_topology_kubernetes_io_region` | The Datum location that served the request. | ### Status | Metric | What it reports | | - | - | | `datum_cloud_networking_http_proxy_status_condition` | `1` when a condition is true. The `condition` label is `Accepted` or `Programmed`, and `reason` explains it. `resource_name` is the load balancer's name. | | `datum_cloud_networking_http_proxy_custom_hostname` | One series for each custom hostname, in the `hostname` label. | ### Example queries Replace `PROJECT_ID` and `ALB_NAME`. These use [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/), which also accepts standard PromQL. Requests per second, by location: ```promql theme={null} sum by (label_topology_kubernetes_io_region) ( rate(envoy_vhost_vcluster_upstream_rq{resourcemanager_datumapis_com_project_name="PROJECT_ID", gateway_name="ALB_NAME"}[5m]) ) ``` Share of responses that are errors (`4xx` and `5xx`): ```promql theme={null} sum(rate(envoy_vhost_vcluster_upstream_rq{resourcemanager_datumapis_com_project_name="PROJECT_ID", gateway_name="ALB_NAME", envoy_response_code=~"[45].."}[5m])) / sum(rate(envoy_vhost_vcluster_upstream_rq{resourcemanager_datumapis_com_project_name="PROJECT_ID", gateway_name="ALB_NAME"}[5m])) ``` 95th percentile origin response time, in milliseconds: ```promql theme={null} histogram_quantile(0.95, sum by (le) (rate(envoy_vhost_vcluster_upstream_rq_time_bucket{resourcemanager_datumapis_com_project_name="PROJECT_ID", gateway_name="ALB_NAME"}[5m])) ) ``` Requests the firewall blocked in the last hour, by rule: ```promql theme={null} sum by (coraza_rule_id, coraza_rule_severity) ( increase(coraza_envoy_filter_request_events_total{resourcemanager_datumapis_com_project_name="PROJECT_ID", gateway_name="ALB_NAME", coraza_outcome="blocked"}[1h]) ) ``` Datum also publishes prebuilt Grafana dashboards for these metrics. See [Datum dashboards in Grafana Cloud](/docs/operations/metrics-export#datum-dashboards-in-grafana-cloud). ## Access logs To see the requests that reached a load balancer, including the ones the firewall blocked, run the following command: ```text theme={null} $ datumctl alb logs api --since 30m --limit 5 TIME METHOD STATUS PATH HOST DURATION 01:29:34 GET 404 /does-not-exist-116 bright-oak-4f7k2.datumproxy.net 3ms 01:29:34 GET 302 / bright-oak-4f7k2.datumproxy.net 4ms 01:29:34 GET 403 /?id=1'%20OR%20'1'='1 bright-oak-4f7k2.datumproxy.net 2ms 01:29:34 GET 404 /does-not-exist-104 bright-oak-4f7k2.datumproxy.net 3ms 01:29:34 GET 302 / bright-oak-4f7k2.datumproxy.net 7ms ``` In the Cloud Portal, a load balancer's **Logs** tab shows the same requests, with filters for time range, method, status code, and host. ![The Logs tab of a load balancer, with filters on the left and a table of requests showing time, status, host, and path, including 404 responses and a 403 for a SQL injection attempt](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/alb-logs-portal.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=9aada428550817b66861fb39f830505e) Access logs in the Cloud Portal, including a request the firewall blocked with 403 Logs can take a few minutes to appear. Useful flags: * `--code 403` narrows the logs to one response code, such as the requests the firewall blocked. Repeat it for several codes. * `--follow` keeps querying for new requests as they arrive. * `--method` and `--host` narrow the logs to one HTTP method or one hostname. * `-o wide` adds the request ID and the upstream that served the request. An upstream of `-` means Envoy answered without reaching your origin, as it does when the firewall blocks a request. See [An origin that cannot be reached](/docs/alb/troubleshooting#an-origin-that-cannot-be-reached). * `-o json` returns the logs as JSON for scripts. ```text theme={null} $ datumctl alb logs api --since 30m --code 403 --limit 2 -o wide TIME METHOD STATUS PATH HOST DURATION REQUEST ID UPSTREAM 01:29:34 GET 403 /?id=1'%20OR%20'1'='1 bright-oak-4f7k2.datumproxy.net 2ms eae36a5d-01ca-49a0-b2e7-2d6235de5e77 - 01:29:33 GET 403 /?id=1'%20OR%20'1'='1 bright-oak-4f7k2.datumproxy.net 2ms 1d3064a3-5698-4c13-abc6-aa1b43fba4fc - ``` ### Query access logs across load balancers The `telemetry` plugin queries every log line in your project, including load balancer access logs, with a [LogQL](https://grafana.com/docs/loki/latest/query/) stream selector. To install it, run `datumctl plugin install telemetry`. For example, to see every request that any load balancer in the project answered with `403`: ```bash theme={null} datumctl telemetry logs '{response_code="403"}' --since 1h ``` Each access log line carries fields such as `authority` (the hostname), `method`, `path`, `response_code`, `response_flags`, `duration`, `upstream_host`, `user_agent`, `request_id`, and `zone_name` (the Datum location that served the request). To list every label you can match on, run `datumctl telemetry labels`. For a load balancer attached to a Compute workload, `datumctl compute logs WORKLOAD_NAME --alb` shows the same access logs. See [Compute observability](/docs/compute/observability). ## Activity To see who changed a load balancer and when, see [Activity logs](/docs/operations/activity-logs). # Serve traffic on your own domain Source: https://datum.net/docs/alb/operations/custom-domains Attach a custom hostname to a Datum Application Load Balancer with datumctl, prove you own the domain, and get a certificate. A new load balancer serves on a generated hostname like `a1b2c3d4.datumproxy.net`. This guide attaches a hostname of your own to it and gets HTTPS working on that name. ## Two ways in, and they fail differently **Keep your own DNS.** Create a CNAME from your hostname to the generated one. Datum writes nothing, and there is nothing to wait for. It needs none of the steps below. **Attach a custom hostname.** Datum serves the name directly, which is what the rest of this guide does. It takes four steps, and each one gates the next. To serve every name under a domain, see [Serve a wildcard hostname](/docs/alb/configuration/wildcard-hostnames). ## Attach it ```bash theme={null} datumctl alb hostname add my-app app.example.com datumctl alb hostname list my-app ``` Custom hostnames must be unique across Datum. If the name is already claimed, `describe` reports it. Who holds it is not visible to you, since it may be in another customer's project. If you believe it should be yours, raise it with Datum rather than going looking. ## Watch the four steps ```bash theme={null} datumctl alb describe my-app ``` Each custom hostname passes through these, in order: | Step | What it means | If it is stuck | | - | - | - | | Claimed | This load balancer holds the name | Something else on Datum has it | | Ownership verified | You proved you own the domain | Below | | DNS record | A record points the name here | Below | | Certificate | HTTPS works on it | Almost always DNS, not the certificate | ## Prove you own the domain Datum gives you a record to create. Look it up on the domain: ```bash theme={null} datumctl get domains --project your-project-id datumctl describe domain example-com --project your-project-id ``` Create that record exactly as given, at your DNS provider. Then leave it — Datum picks it up on its own once it is publicly visible. There is no command to force a re-check, and no deadline, because this step waits on you rather than on Datum. Three things look like a missing record and are not: * **The record is only visible internally.** Datum queries it the way the public internet does, so a record in a split-horizon or private zone does not count. * **It has not propagated yet.** A record created a minute ago may not be visible from where Datum looks. * **The name got the zone appended twice.** Many DNS interfaces add the zone automatically, turning `_datum-challenge.example.com` into `_datum-challenge.example.com.example.com`. Check the record's full name as your provider displays it. This is the most common cause of a verification that will not complete on a record you are certain you created. ## Let Datum answer DNS, or keep your own Once ownership is proven, whether Datum can write the DNS record depends on who answers for the domain. If Datum runs the domain's DNS, the record appears on its own. To point the domain's apex and other subdomains at the load balancer too, see [Point a domain at a load balancer](/docs/domain-dns/point-to-alb). If it does not, `describe` says so — and that is a normal arrangement, not a fault. Create a CNAME to the generated hostname at your own provider and you are done. > [!WARNING] > > Moving a domain's nameservers to Datum moves **all** of its DNS, not just this hostname. Anything else answered by your current provider needs to exist in the new zone first, or it stops resolving. This is the step that causes outages. ## The certificate Once the name resolves publicly to the load balancer, the certificate is issued automatically. Until then it cannot be — so a certificate that is not issued is usually a symptom, and the DNS step above is the cause. If DNS is genuinely in place and the certificate still fails, check for a CAA record on the domain that does not permit Datum's certificate authority. Customers often do not know one is there. ## Confirm it ```bash theme={null} curl -sSI https://app.example.com ``` Status alone is not enough here — public DNS caches mean the configuration can be right and the name still not resolve for a while. ## Removing a hostname ```bash theme={null} datumctl alb hostname remove my-app app.example.com ``` This does not ask for confirmation. If Datum manages DNS for the domain, the records it created go with it. # ALB operations Source: https://datum.net/docs/alb/operations/overview Create, inspect, update, and delete Application Load Balancers with the datumctl alb plugin. To install the `datumctl alb` plugin, run `datumctl plugin install alb`. ## Create a load balancer To create a load balancer in front of an origin, run the following command: ```bash theme={null} datumctl alb create ALB_NAME --endpoint ORIGIN_URL ``` Replace the following: * `ALB_NAME`: a name for the load balancer, such as `my-app`. * `ORIGIN_URL`: the `http` or `https` URL of your origin, such as `https://origin.example.com`. The command waits up to 2 minutes for Datum to assign a generated hostname, then prints it: ```text theme={null} Application load balancer "my-app" created. Hostname: piece-build-byrkx.datumproxy.net ``` Point your custom domains at this hostname with a `CNAME` record. Pass `--no-wait` to return as soon as the load balancer exists. A load balancer created with `datumctl alb create` starts with the following defaults: * **Force HTTPS:** on. HTTP requests are redirected to HTTPS. To turn it off, pass `--no-force-https`. * **Traffic protection:** `Enforce` at sensitivity 1. To change it, pass `--waf-mode Observe` or `--paranoia LEVEL`, or pass `--no-waf` to create the load balancer without a policy. Before you put an existing site behind it, see [Roll out traffic protection safely](/docs/alb/operations/traffic-protection). * **Load balancing:** least request, with health checks off. See [Load balancing and health checks](/docs/alb/configuration/load-balancing). A load balancer that `datumctl compute deploy --http-port` attaches to a workload doesn't use these defaults. To check what a load balancer has, run `datumctl alb describe ALB_NAME`. The following flags change what you create: | Flag | What it does | | - | - | | `--endpoint URL` | Adds a URL origin. Repeat it to split requests evenly across several origins. | | `--network-service NAME --port PORT_NAME` | Sends traffic to an existing [network service](/docs/galactic-vpc/networks/network-services) on its named port, such as `http`. Repeat the pair for several services. | | `--hostname HOSTNAME` | Attaches a custom hostname at create time. See [Serve traffic on your own domain](/docs/alb/operations/custom-domains). | | `--host-header HOST` | Sets the Host header that Datum sends to the origin. | | `--tls-hostname HOSTNAME` | Sets the hostname that Datum uses to verify TLS when an `--endpoint` origin is an IP address. | | `--display-name NAME` | Sets the name shown in the Cloud Portal. If you omit `ALB_NAME`, Datum derives one from the display name. | | `--dry-run` | Validates the load balancer with the API server without creating it. | For example, to put a load balancer in front of a workload's network service and attach a custom hostname, run the following command: ```bash theme={null} datumctl alb create my-app \ --network-service storefront --port http \ --hostname app.example.com ``` ## Inspect a load balancer To see a load balancer's status, hostnames, routes, traffic protection, and certificates, run the following command: ```bash theme={null} datumctl alb describe ALB_NAME ``` `Status: Active` means Datum has published the configuration. It doesn't mean a request has succeeded, so confirm with a request such as `curl -I https://HOSTNAME/`. ## How changes roll out You change a load balancer in one place, your project, and Datum rolls the change out to Envoy in every Datum location. There's nothing to deploy per location and no capacity to change: Datum runs and scales the proxies. For how this works, see [Envoy](/docs/platform/envoy). A change that Datum accepts is published within seconds, but each location applies it on its own schedule. For a short time, some locations can serve the old configuration and others the new one. A change doesn't drop connections that are already open. To confirm a change, make a request rather than reading status. If a change seems to have no effect, see [A change that was accepted and never took effect](/docs/alb/troubleshooting#a-change-that-was-accepted-and-never-took-effect). ## Update a load balancer The `datumctl alb` plugin changes one part of a load balancer at a time: * **Hostnames:** see [Serve traffic on your own domain](/docs/alb/operations/custom-domains). * **Routes and origins:** see [Send paths to different origins](/docs/alb/operations/routes). * **Origin weights, load balancing, and health checks:** see [Load balancing and health checks](/docs/alb/configuration/load-balancing). * **Traffic protection:** see [Roll out traffic protection safely](/docs/alb/operations/traffic-protection). * **Basic authentication:** run `datumctl alb auth set`. See [Authentication](/docs/alb/authentication/overview). To change the underlying resources directly, edit the `HTTPProxy` and `TrafficProtectionPolicy` and apply them with `datumctl apply`. See [HTTPProxy](/docs/alb/httpproxy/overview). ## Delete a load balancer To delete a load balancer, run the following command: ```bash theme={null} datumctl alb delete ALB_NAME --yes ``` This also removes the traffic protection policy and any basic authentication attached to it. If you create a load balancer again with the same name, it gets a new generated hostname, so update any `CNAME` records that pointed at the old one. Without `--yes`, the command asks you to type the name to confirm. A load balancer that `datumctl compute deploy` attached to a workload is deleted with the workload by `datumctl compute destroy`. See [Compute operations](/docs/compute/operations#delete-a-workload). # Send paths to different origins Source: https://datum.net/docs/alb/operations/routes Add routes to a Datum Application Load Balancer with datumctl, split traffic across several weighted origins, and understand what the cloud portal does with the result. A load balancer starts with one route: everything on `/` goes to the origin you gave it. This guide adds more. ## Add a route ```bash theme={null} datumctl alb route list my-app datumctl alb route add my-app --path /api --endpoint https://api.example.com ``` Paths are prefix matches. `/api` catches `/api`, `/api/v1` and everything below it. The most specific match wins, so `/` keeps serving everything else. To change where an existing route points: ```bash theme={null} datumctl alb route update my-app --path /api --endpoint https://api-new.example.com ``` `route update` replaces every origin on that path and leaves the other routes alone. Preview anything you are unsure about first: ```bash theme={null} datumctl alb route add my-app --path /api --endpoint https://api.example.com --dry-run ``` ## Remove a route ```bash theme={null} datumctl alb route remove my-app --path /api ``` The `/` route cannot be removed while other routes exist unless you pass `--force`, and the last remaining route cannot be removed at all. > [!WARNING] > > `route remove` asks for confirmation when you run it in a terminal and **assumes yes when you do not** — in a pipeline or CI it proceeds. Use `--dry-run` first in automation. ## Several origins on one route A route takes up to 16 origins, and requests are split across them by weight: ```bash theme={null} datumctl alb route backend list my-app datumctl alb route backend add my-app --path /api --endpoint https://api-2.example.com datumctl alb route backend remove my-app --path /api --endpoint https://api.example.com ``` Removing the last origin on a route is refused. Remove the route instead. Each URL origin is sent its own hostname as the Host header, so origins on different hostnames can share a route. A Host override on the route is the trap: it sends the same Host to every origin, so any origin that serves by hostname (Vercel, Netlify, Fly.io, Cloudflare Pages) answers the wrong site or a 404. It looks like it worked. A connector origin has to be the only origin in its route. ### Weights Weights are relative: origins at 1 and 3 get 25% and 75% of requests. An origin with no weight counts as 1. `route backend list` shows each origin's `WEIGHT` and `SHARE`. ```bash theme={null} datumctl alb route backend add my-app --path /api --endpoint https://api-3.example.com --weight 2 datumctl alb route backend update my-app --path /api --endpoint https://api.example.com --weight 0 ``` * Without `--weight`, `route backend add` gives the new origin the average weight of the pool, so adding one beside two origins at 50 gives it 50 rather than 1. A pool where no origin has a weight stays that way. * `route backend update --weight 0` drains an origin: it stays in the pool and gets no requests. * A change that would leave every origin on a route at weight 0 is refused, because the route would serve nothing. * `route update` keeps the weight, TLS hostname and filters of any origin that stays in the new list. The load balancing algorithm and passive health checks apply to every route. See [Load balancing and health checks](/docs/alb/configuration/load-balancing). ## Routes this plugin will not edit `route list` marks some routes `advanced`. Those use path matches, header matches or filters beyond what the plugin represents — it leaves them alone rather than risk rewriting them. Edit those with `datumctl apply -f`. Force HTTPS also appears in `route list`, marked `system`. It is a redirect rule with no origins behind it, and `route remove` will not delete it — use `datumctl alb update my-app --no-force-https`. ## What the cloud portal does with this The portal manages one route: the pool of origins on `/`, with their weights, the load balancing algorithm and health checks. Those are the same fields `datumctl alb` writes, so you can change them from either. The portal has no way to show a second route, a path match or a per-origin filter. When a load balancer has any of them, the portal makes its Backends tab and origin settings read-only, so it never drops what it cannot show. Custom hostnames, traffic protection and basic authentication stay editable there. Manage a load balancer with more than one route with `datumctl alb`. # Roll out traffic protection safely Source: https://datum.net/docs/alb/operations/traffic-protection Turn on the Datum web application firewall with datumctl without blocking real users, using Observe mode and access logs to tune it first. Every load balancer created with `datumctl alb create` gets traffic protection in **Enforce** mode at sensitivity 1. That is a sensible default for something new. It is not a sensible thing to switch on in front of a live site without looking first. This guide covers doing it in the order that does not block your own users. ## Where it stands ```bash theme={null} datumctl alb waf describe my-app ``` Three states: | Mode | What happens | | - | - | | `Observe` | Requests are inspected and logged. Nothing is blocked. | | `Enforce` | Matching requests are blocked. | | `Disabled` | The policy exists but does nothing. | Sensitivity — the paranoia level — runs 1 to 4. Higher catches more and produces more false positives. > [!NOTE] > > The cloud portal offers sensitivity 1 and 2 only. Setting 3 or 4 here works, but the portal will not be able to change it afterwards. ## Putting an existing site behind it Do not go straight to `Enforce` on a site that already has users. **1. Start in Observe** ```bash theme={null} datumctl alb waf set my-app --mode Observe --paranoia 1 ``` Requests are inspected and logged, and nothing is blocked. **2. Let it see real traffic** Long enough to include whatever runs weekly — reports, exports, batch jobs, a partner's nightly integration. A weekday is rarely enough. **3. Switch to Enforce** ```bash theme={null} datumctl alb waf set my-app --mode Enforce --paranoia 1 ``` **4. Check what is being blocked** ```bash theme={null} datumctl alb logs my-app --since 1h --code 403 ``` Look for legitimate traffic in there: file uploads, rich text or Markdown in request bodies, SQL-like strings typed into a search box, long base64 values, API clients sending unusual headers. All of these trip rules at higher sensitivities. If you find any, see [When real traffic is being blocked](#when-real-traffic-is-being-blocked). **5. Raise sensitivity one level at a time** Repeat the same check at each level. Do not jump from 1 to 4. ## When real traffic is being blocked The signature is requests reaching the load balancer and coming back `403` without ever reaching your origin. Your origin's own logs show nothing at all, which is what makes it confusing — from the origin's side the traffic simply vanished. ```bash theme={null} datumctl alb logs my-app --since 1h --code 403 -o wide ``` The fix is almost never to turn protection off. In order of preference: drop to `Observe` while you investigate, lower the sensitivity, or narrow what runs. ```bash theme={null} datumctl alb waf set my-app --mode Observe ``` Turning it off entirely is a decision to make knowingly: ```bash theme={null} datumctl alb waf disable my-app ``` > [!WARNING] > > `waf disable` deletes the policy without asking for confirmation. ## A load balancer with no protection at all This is worth checking for, because nothing reports it as a problem. Creating a load balancer in the cloud portal attaches protection on a best-effort basis. If that attach fails, the load balancer is created anyway, you are not told, and the security card simply reads as disabled. The same is true of `datumctl alb create`: if the load balancer is created and the policy then fails, the command exits non-zero having left an unprotected load balancer behind. So after any failed create: ```bash theme={null} datumctl alb waf describe my-app ``` If it reports off, attach one: ```bash theme={null} datumctl alb waf set my-app --mode Enforce --paranoia 1 ``` ## Rule exclusions and rate limiting `datumctl alb waf` covers mode and sensitivity. Excluding specific rules, setting score thresholds, rate limiting and circuit breaking are manifests — see [WAF configuration](/docs/alb/configuration/waf) and [HTTPProxy and WAF capabilities](/docs/alb/httpproxy/overview). Keep the detection level at or above the blocking level so you can preview stricter rules in the logs before enforcing them. # ALB Source: https://datum.net/docs/alb/overview Datum's Application Load Balancer (ALB) is built on Envoy and provides Layer 7 routing with a Coraza-based Web Application Firewall (WAF) to protect and route internet traffic to your backend services. > [!NOTE] > > **Working with this resource via an AI agent?** Datum publishes three skills covering Application Load Balancer: [Application Load Balancer](/docs/agents/skills/alb) (creating and managing one, plus WAF and auth), [Client Traffic](/docs/agents/skills/client-traffic) (TLS termination, HTTP/3, connection limits; not available yet), and [HTTPRoute](/docs/agents/skills/httproute) (path routing, traffic splitting, redirects). Datum's Application Load Balancer (ALB) is an edge proxy built on [Envoy](/docs/platform/envoy), with a [Coraza](https://www.coraza.io/)-based [web application firewall](/docs/alb/configuration/waf) (WAF). It started life as Datum's "AI Gateway", and it's built for AI and agent traffic as much as for websites and APIs. Application Load Balancers help you protect and route internet traffic to your backend services. We support HTTP(S) 1.1, HTTP/2, gRPC, and WebSockets. > [!TIP] > > Running NGINX, Apache, Caddy, or Traefik in front of your app? To replace that container with an ALB, see [Use ALB as your HTTP proxy](/docs/guides/alb-as-httpd). ![The Datum Cloud console showing an Application Load Balancer's overview, with live request rate, error rate, p95 latency, requests blocked by the firewall, and a panel for attaching custom hostnames](https://mintcdn.com/datum-4926dda5/REoQYaLkj7V4WzRK/images/alb-overview.png?fit=max\&auto=format\&n=REoQYaLkj7V4WzRK\&q=85\&s=8a7b903e39d9a02c32250e2d2e6fb727) The overview for a load balancer: whether it is serving, live request rate, error rate, latency and blocked requests, and the hostnames it answers on ## How it works An ALB isn't a server you size or place. You declare it once in your project, and Datum serves it from Envoy in every Datum location: ```mermaid theme={null} flowchart LR C["Clients
browsers · apps · AI agents"] subgraph loc["Nearest Datum location"] E["Envoy · your ALB
TLS · WAF · routes · auth"] end O["Public origin
an existing site or API"] I["Compute Instances
on your gVPC network"] T["Connector tunnel
a service on your machine"] C -->|"HTTPS · anycast"| E E -->|"HTTPS · internet"| O E -->|"HTTP · private network"| I E -->|"Iroh tunnel"| T classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 class C,E,O,I,T box style loc fill:#EDEDEC,stroke:#8A8E99,color:#B5605E linkStyle 0,1,2,3 stroke:#5A5F6B,stroke-width:1.4px ``` * **Clients reach the nearest location.** Your ALB's hostnames are anycast, so each request arrives at the nearest Datum location. Envoy there terminates TLS, runs the firewall, and applies your routes. * **Datum runs and scales the proxies.** Envoy runs as a shared fleet in every location, serving every customer's ALBs. Creating an ALB doesn't start new infrastructure, and you don't choose locations or capacity. * **Changes reach every location.** When you change an ALB, Datum publishes the new configuration to Envoy in every location. * **Your origin can be anywhere.** An origin can be a public site, reached over HTTPS on the internet; [Compute Instances](/docs/compute/compute-and-alb), reached over plain HTTP on your private [gVPC](/docs/galactic-vpc/overview) network; or a service on your own machine, reached through a [Connector tunnel](/docs/connectors/tunnels). For Compute origins, Envoy prefers an Instance in the same location, and sends the request to the nearest location with a healthy Instance when there isn't one. For how Datum runs Envoy, including how it reaches your private network without ever reaching another customer's, see [Envoy](/docs/platform/envoy). ## What an ALB is made of In Datum today, an Application Load Balancer is a product experience over several platform resources — not a single API kind. Creating an ALB in the [Cloud Portal](https://cloud.datum.net) stitches these together for you: ```text theme={null} Application Load Balancer (product) ├── HTTPProxy (user-created) — L7 routing, origin, Force HTTPS, Host header │ └── operator materializes Gateway (same name), canonical hostname, │ DNSRecordSet(s), and TLS certificates (ACME) ├── TrafficProtectionPolicy (created with the proxy, same name) — Coraza/OWASP WAF │ └── targetRefs → Gateway └── [optional] SecurityPolicy + Secret *-basic-auth — basic authentication ``` | Piece | Role | | - | - | | **`HTTPProxy`** | Primary user-facing resource for hostnames, origin backends, redirects, and request filters | | **`TrafficProtectionPolicy`** | WAF policy (Observe / Enforce / Disabled) attached 1:1 to the proxy's Gateway | | **Gateway** (operator) | Provisioned from the HTTPProxy; WAF and auth policies target it | | **DNS + TLS** (operator) | Canonical hostname, DNSRecordSets, and ACME certificates from proxy status | | **SecurityPolicy** (optional) | Basic auth via [Envoy Gateway](https://gateway.envoyproxy.io/), backed by a Secret | Portal create always POSTs an `HTTPProxy`, then attempts a matching `TrafficProtectionPolicy` (best-effort today if WAF attach fails). There's no composite ALB API yet. With `datumctl`, `datumctl alb create` creates the `HTTPProxy` and its `TrafficProtectionPolicy` for you (see [Create a load balancer](/docs/alb/operations/overview#create-a-load-balancer)), or you can apply the resources yourself. To learn how Datum runs Envoy, see [Envoy](/docs/platform/envoy). See [HTTPProxy](/docs/alb/httpproxy/overview) for schemas and `datumctl` examples. ## Features Each Application Load Balancer deploys with a default hostname and enforces HTTPS (you can disable this). * **Protection** - The [**OWASP Coraza Web Application Firewall (WAF)**](https://www.coraza.io/) provides built-in protection with the [OWASP Core Rule Set](https://coreruleset.org/), running inside Envoy before requests reach your origin. A load balancer created with `datumctl alb create` starts in `Enforce` mode at sensitivity 1. A `TrafficProtectionPolicy` that doesn't set a mode runs in `Observe` mode. See [Roll out traffic protection safely](/docs/alb/operations/traffic-protection). * **Custom Hostnames** – Verify domains (or use already verified domain resources) to use custom hostnames with an Application Load Balancer. * **Basic Authentication** - Add a username and password to restrict access to an Application Load Balancer. * **Load Balancing** - Split traffic across several origins by weight, choose the load balancing algorithm, and turn on passive health checks. See [Load balancing and health checks](/docs/alb/configuration/load-balancing). To monitor the health of your Application Load Balancers, you can use: * **Status** – View status “Accepted” and “Programmed”, Generated hostname(s), Custom hostname verification status, and TLS verification status. * **Metrics** - The Datum Cloud portal provides a variety of baseline metrics, filtered by region, including: global upstream latency percentile, regional upstream RPS, and regional upstream response. * **Activity Logs** - View and filter activity across a project, and filter by resource type (including Application Load Balancer) and action. # When a load balancer is not serving Source: https://datum.net/docs/alb/troubleshooting Work out why a Datum Application Load Balancer is not serving traffic, in the order that finds the cause fastest, using datumctl. This guide goes in order. Each step rules something out, so work down rather than jumping. ## Start with describe ```bash theme={null} datumctl alb describe my-app ``` That is the whole picture: status, the generated hostname, each custom hostname with its progress, routes and their origins, traffic protection, and basic auth. Two things to read carefully. **Status conditions report on several hostnames at once and name none of them.** A load balancer reporting a partial failure is telling you something is wrong with at least one hostname, not which. The per-hostname lines below are where the answer is. **"Published" is not "serving".** A load balancer reports its configuration as published (`Programmed=True`) within seconds of a change, while Datum's edge is still catching up, and nothing publishes when that finishes. There is no documented figure for how long, so do not wait on a number. Make a request instead. ## Is anything actually reaching it? ```bash theme={null} curl -sSI https:// datumctl alb logs my-app --since 1h ``` Test the **generated** hostname first. It bypasses every question about your own DNS, so it splits the problem cleanly: * Generated hostname works, custom one does not → a hostname problem, below. * Neither works → the load balancer or its origin, further below. > [!NOTE] > > No traffic in the logs is not a diagnosis. A load balancer nobody has called looks exactly like one that is broken. If nothing is arriving, send a request yourself and look again. ## A hostname that does not work `describe` shows four steps per custom hostname. Work on the first one that is not done and ignore the rest — each gates the next. | Step not done | Where to go | | - | - | | Claimed | The name is held elsewhere on Datum. Who holds it is not visible to you | | Ownership verified | [Serve traffic on your own domain](/docs/alb/operations/custom-domains) | | DNS record | Same guide — usually Datum does not answer for the domain, which is normal | | Certificate | Almost always DNS. Fix the step above and it resolves itself | Two states that look like failures and are not: * **Datum does not manage the domain's DNS.** Reported as such, set deliberately. It means "create a CNAME yourself", not "something went wrong". Nothing is pending. * **The generated hostname reports nothing about its DNS record.** No status is published for it at all, so its absence tells you nothing either way. ## An origin that cannot be reached If the generated hostname does not serve either, the problem is between the load balancer and your origin. ```bash theme={null} datumctl alb route list my-app datumctl alb logs my-app --since 1h -o wide ``` Read the response codes: | What you see | What it means | | - | - | | `200`, `404`, other application responses | Traffic is reaching your origin and it is answering. The problem is in your application | | `403` with nothing at your origin | Blocked before it got there — [traffic protection](/docs/alb/operations/traffic-protection) | | `502` / `503` | Datum could not get a usable answer. Origin down, refusing connections, or too slow | | `429` | Rate limited | The response flags in `-o wide` are the most useful field when nothing else explains it — they say whether the connection to your origin failed, timed out, or found nothing healthy. For a network service origin, check the service exists and declares the port name you used. `--port` takes a name like `http`, never a number, and Datum will not invent a service that does not exist. ## A change that was accepted and never took effect This one is easy to miss. If a load balancer reports as waiting to be published, read the message rather than waiting. A message saying it cannot be published means Datum has decided it cannot use these settings at all — the load balancer keeps serving whatever it published last, and waiting will never clear it. To read the message, run `datumctl alb describe my-app`. It names the setting that Datum can't use. Fix that setting, and Datum publishes the change. ## Nothing reports a fault and it still does not work In rough order of likelihood: 1. **Nothing points at it.** No CNAME, no custom hostname attached. Common, and invisible in status. 2. **It was created moments ago.** The edge is still converging. Retry the request rather than reading status again. 3. **The origin is still starting.** A new Compute Instance can report `Available` a minute or two before the app inside it listens. The load balancer returns `503` until it does. 4. **The origin is refusing connections.** The logs show it; status does not. 5. **Traffic protection is blocking real requests.** `Enforce` at a high sensitivity will do this. ## It works from one place and not another Each client reaches the Datum location nearest to it, and [Envoy](/docs/platform/envoy) in that location serves the request. If a load balancer behaves differently for users in different regions: 1. **Find which location you reach.** Visit [edge.datum.net](https://edge.datum.net/) from the affected network. It shows the Datum location your traffic arrives at. 2. **Check whether a change is still rolling out.** Each location applies a change on its own schedule. If you changed the load balancer moments ago, wait briefly and try again. 3. **For a Compute origin, check where your Instances run.** Envoy prefers an Instance in its own location. If that location has no healthy Instance, Envoy sends the request to the nearest location that has one, which adds latency for users near the empty location. Run `datumctl compute workloads describe WORKLOAD_NAME` to see the health of the Instances in each location. See [Placement and scaling](/docs/compute/placement-and-scaling). 4. **For a public origin, test it from that region.** Envoy connects to a public origin over the internet from the location that received the request. An origin that blocks or rate-limits some regions or IP ranges can fail from one location and work from another. ## Permission errors A command that fails with a permission error rather than a not-found means your account cannot act on that resource in this project. Ask whoever administers the project; there is nothing to change on the load balancer. # Changelog Source: https://datum.net/docs/changelog New features, improvements, and releases across Datum Cloud, newest first. Each entry links to the original announcement in our [changelog discussions](https://github.com/orgs/datum-cloud/discussions/categories/changelog), where you can ask questions and leave feedback. To see what is coming next, read the [roadmap](/docs/platform/roadmap). **October 29, 2026** datumctl ## Correct version reporting for Nix installs `datumctl` installed from the Nix flake now reports a valid version. * **Fixed:** `datumctl version` no longer crashes when `datumctl` is installed with `nix profile install` or run with `nix run`. * **Fixed:** plugin compatibility checks and update checks now read a real version in Nix builds instead of a placeholder. To pick up the fix, reinstall or upgrade `datumctl` from the flake. See [Install datumctl](/docs/datumctl/install). **October 21, 2026** Application Load Balancer ## Run gRPC APIs on Datum The Application Load Balancer now supports HTTP/2 from clients to origins, so gRPC APIs can run on Datum and be called at their ALB hostname. * **Compute:** add `--http2` at deploy time to prepare the workload's ALB for gRPC clients. The setting persists across later deploys. Requires the compute plugin v0.12.11 or later. * **Network services:** mark a service port as HTTP/2 so any ALB using that port can serve gRPC. Requires the network and alb plugins v0.32.26 or later. * **Hostnames:** generated and custom hostnames both support HTTP/2. HTTP/1.1 clients are unaffected. * **TLS:** the ALB terminates TLS, so your server speaks plain gRPC. Origins added by URL, including servers outside Datum, still use HTTP/1.1. If your app uses WebSockets, serve them on a separate port without HTTP/2. [Read the announcement](https://github.com/orgs/datum-cloud/discussions/310) **October 13, 2026** datumctl ## New datumctl plugins and assistant capabilities Install plugins from the official catalog with `datumctl plugin install `. * **compute:** deploy, scale, restart, and roll out workloads, stream `logs`, run `exec` inside an instance, publish images with `build`, and check remaining project `quota`. * **alb:** create a load balancer with an immediate hostname, add custom domains with verification tracking, route paths and split traffic by weight, and manage WAF, basic auth, headers, and request logs. * **network:** create private IPv6 networks, see where they run, give workloads stable service names and named ports, and manage instance interfaces. Deletions list affected resources and ask for confirmation. Patch, the Datum assistant, can now diagnose why a network or network service isn't working or receiving traffic, and can create networks and network services. It shows exactly what it will change and waits for your approval. Use it in the [cloud portal](https://cloud.datum.net) or with `datumctl plugin install assistant`. [Read the announcement](https://github.com/orgs/datum-cloud/discussions/306) **October 13, 2026** datumctl v0.22.0 ## Upgrade every plugin in one command * `datumctl plugin upgrade --all` upgrades every installed plugin that has a newer release and skips the rest. * Each plugin is fetched from the catalog or GitHub repository it was installed from, with the same checksum and compatibility checks as a fresh install. * If one plugin fails, the others still upgrade and the failures are listed at the end. * Fixed upgrading a third-party catalog plugin by name, which previously looked only in the official catalog. [Read the announcement](https://github.com/orgs/datum-cloud/discussions/304) **October 7, 2026** Application Load Balancer ## Wildcard hostnames An Application Load Balancer can now serve every subdomain of a domain you own. Add a hostname such as `*.example.com` and get one automatically issued and renewed certificate that covers all names one level below the domain. ```bash theme={null} datumctl alb hostname add my-app '*.example.com' datumctl alb describe my-app ``` Publish the `_acme-challenge` CNAME record that `describe` shows. The certificate is typically issued within a few minutes, and the record stays the same at renewal. This is an early-access capability that isn't turned on by default, so contact Datum to enable it for your project. It requires the alb plugin v0.32.3 or later (`datumctl plugin upgrade alb`). A wildcard matches one label level only: `*.example.com` covers `app.example.com` but not `a.b.example.com`. [Read the announcement](https://github.com/orgs/datum-cloud/discussions/303) **September 29, 2026** Patch ## Patch can work with your compute, DNS, and load balancing Patch can now query Compute, DNS, and Application Load Balancer data directly, so troubleshooting follows the actual cause instead of inferring it from generic resource reads. * **Compute:** follows top-level reasons such as `Available=False` with `QuotaNotGranted` down to the instance-level condition that holds the cause. * **DNS:** diagnoses zones, records, and delegation, including domain verification, conflicting records, and zone imports. * **Load balancing:** covers certificates, unreachable backends, edge propagation, and access logs. The service tools are read-only. Any change goes through a plan you review and approve. Which tools are available depends on what each project is entitled to. In the terminal, run `datumctl assistant` for a full-screen chat, `datumctl assistant chat "..."` for a one-shot answer in scripts, or `datumctl assistant resume` to search project conversations. Install it with `datumctl plugin install assistant`. [Read the announcement](https://github.com/orgs/datum-cloud/discussions/300) # Serve public traffic with ALB Source: https://datum.net/docs/compute/compute-and-alb Attach Datum's Application Load Balancer (ALB) to a workload to serve public HTTP and HTTPS traffic, and add your own domain. Instances have private IPv6 addresses only, so the internet can't reach them directly. `--http-port` is the quickest way to attach an ALB to a workload: it declares the port your container serves HTTP on and creates an `HTTPProxy` for the workload. Datum terminates TLS at the ALB, sends each request to the nearest location with a healthy Instance, and connects to your Instances over plain HTTP on your private network. > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Before you begin * Select a project, install the `compute` plugin, and get access to Compute. For more information, see [Compute commands](/docs/datumctl/compute/overview). * Make sure that you have permission to create networking resources in the project, in addition to compute resources. The **Network Admin** role includes the permissions that publishing needs. The **Gateway Admin** role alone isn't enough. * Make sure that your app serves plain HTTP, not HTTPS, on the port that you publish, and listens on IPv6. ## Attach an ALB with datumctl To attach an ALB to a workload, run `datumctl compute deploy` with `--http-port`: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME \ --image=IMAGE \ --location=LOCATION \ --http-port=PORT ``` Replace the following: * `WORKLOAD_NAME`: the name of your workload. * `IMAGE`: the full image reference, including the registry host. * `LOCATION`: one or more locations, separated by commas. * `PORT`: the port that your container serves HTTP on, such as `8080`. `--http-port` works when you create a workload and when you update one that you deployed with flags. > [!WARNING] > > A flag-based `datumctl compute deploy` replaces the workload's whole template and placements with what the flags describe. It keeps environment variables, ConfigMap and Secret imports, volumes, and labels. It removes registry credentials, custom commands, and extra placements that you set in a manifest, and it sets the Instance count to the `--min` value, which defaults to `1`. To publish a workload that you manage with a manifest, see [Publish a workload you manage with manifests](#publish-a-workload-you-manage-with-manifests). The command declares the port on the container, waits for the rollout, and then prints the publishing progress and the ALB's hostname as the last line of output: ```text theme={null} Publishing... Backends 2 healthy across us-central-1, us-east-1 Edge programmed Certificate issued https://stark-raven-b7k2m.datumproxy.net ``` Your hostname is different. A new hostname can take a few minutes to resolve. If a request fails with a name resolution error, wait and try again. If you press Control+C while the command publishes, publishing continues in the background. If your app speaks HTTP/2, such as a gRPC server, also add `--http2` to the command, and serve HTTP/2 without TLS (h2c) on `PORT`. Without the flag, Datum connects to your Instances over HTTP/1.1, which most apps expect. gRPC needs HTTP/2 on every connection, so with the flag, Datum connects to your Instances over HTTP/2 too. Clients connect to the ALB hostname on port 443, such as `grpcurl stark-raven-b7k2m.datumproxy.net:443 list`. Later deploys keep the setting, and `--http2=false` turns it off. ## Find your workload's ALB hostname To see the ALB hostname of every workload in your project, run the following command: ```bash theme={null} datumctl compute workloads ``` The hostname is in the `URL` column. To see the hostname and the health of the Instances behind it in each location, run the following command: ```bash theme={null} datumctl compute workloads describe WORKLOAD_NAME ``` In the [Cloud Portal](https://cloud.datum.net), a workload's **Overview** tab shows its hostname, and a topology of the ALB in front of the workload and its Instances in each location, with live traffic, error rate, and latency. ![A workload Overview tab showing Serving normally, and a topology diagram linking the load balancer, with traffic, errors, p99 latency, and hostname, to the workload, with 6 of 6 Instances ready across 3 locations](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/compute-workload-topology.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=ebaca5e2ce09e908280ba29698a8cb57) A published workload: its ALB, the workload, and its Instances in each location If you have `jq` installed, you can read the hostname in a script. Run the following command: ```bash theme={null} datumctl compute workloads -o json | jq -r '.[] | select(.name=="WORKLOAD_NAME") | .url' ``` Replace `WORKLOAD_NAME` with the name of your workload. ## Change or remove the ALB The ALB keeps its hostname when you redeploy the workload. If you run `datumctl compute deploy` without `--http-port`, the workload keeps its current port and the ALB keeps its hostname. To change or remove the ALB, do one of the following: * To serve on a different port, run `datumctl compute deploy` again with the new `--http-port` value. The hostname stays the same. * To stop serving the workload publicly, run `datumctl compute deploy` with `--no-http`. This removes the ALB, and the hostname stops responding. Datum generates the hostname when it creates the ALB. If you remove the ALB and publish again, the workload gets a new hostname. `datumctl compute destroy` removes the workload's ALB along with the workload and its Instances. A plain `datumctl delete workload` deletes the workload and its Instances, but leaves the ALB in place. For more information, see [Delete a workload](/docs/compute/operations#delete-a-workload). ## Use a custom domain You can serve a published workload on your own domain, such as `api.example.com`, in addition to its generated hostname. The ALB is an `HTTPProxy` resource named after the workload. For more on custom hostnames, TLS, and WAF protection, see [ALB](/docs/alb/overview). To add a custom domain, follow these steps: 1. Add and verify your domain in Datum. For more information, see [Domains](/docs/domain-dns/domains). 2. To open the workload's `HTTPProxy` for editing, run the following command: ```bash theme={null} datumctl edit httpproxy WORKLOAD_NAME ``` Replace `WORKLOAD_NAME` with the name of your workload. 3. Add your hostname to `spec.hostnames`, and then save and close the file: ```yaml theme={null} spec: hostnames: - api.example.com ``` 4. Point your hostname at the workload's generated hostname with a `CNAME` record. If Datum hosts your DNS, see [Point a domain at a load balancer](/docs/domain-dns/point-to-alb). `datumctl compute deploy` keeps custom hostnames when it updates a workload. `datumctl compute destroy` removes the ALB, including its custom hostnames, but keeps your domain resources. ## Publish a workload you manage with manifests A workload manifest can't declare an ALB, and `datumctl compute deploy -f` doesn't create or remove one. To attach an ALB to a manifest-managed workload, follow these steps: 1. Create the workload with `datumctl compute deploy` and `--http-port`, as described in [Attach an ALB with datumctl](#attach-an-alb-with-datumctl). The command saves the workload to `workload.yaml` in the current directory. 2. Rename `workload.yaml`, and make your changes in the renamed file. Keep the container port that you published. 3. To apply later changes, run `datumctl compute deploy -f FILE`, where `FILE` is the path to your manifest. The workload keeps its ALB. ## Limitations In addition to the [limitations that apply to all of Compute](/docs/compute/limitations-and-roadmap), the ALB has the following limitations: * Clients always connect over HTTPS. Datum connects to your Instances over plain HTTP, and TLS between Datum and your Instances isn't supported. * `datumctl compute deploy` attaches one ALB to each workload, on one port. * With `--http2`, WebSocket connections also reach your Instances over HTTP/2, and most servers accept WebSockets only over HTTP/1.1. To serve WebSockets alongside gRPC, serve them on a separate port through a [network service](/docs/galactic-vpc/networks/network-services). ## What's next * To run in more locations so that more users reach a nearby Instance, see [Placement and scaling](/docs/compute/placement-and-scaling). * To learn more about `HTTPProxy` routing and options, see [Application Load Balancer](/docs/alb/overview). * To learn how Instances connect privately, see [Traffic and connecting](/docs/galactic-vpc/networks/traffic). # Configure a workload Source: https://datum.net/docs/compute/configuration Pass environment variables, ConfigMaps, and Secrets to your Instances, and pull images from a private registry. > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Before you begin * Select a project, install the `compute` plugin, and get access to Compute. For more information, see [Compute commands](/docs/datumctl/compute/overview). * Make sure that you can read the ConfigMaps and Secrets that your workload references. When you apply a workload, Datum checks that you have `get` permission on each one, and rejects the request if you don't. You can set environment variables, and load or mount whole ConfigMaps and Secrets, with `datumctl compute deploy` flags. These flags require compute plugin v0.10.7 or later. For anything that the flags can't set, such as a variable from one key of a Secret or credentials for a private registry, write a workload manifest. Apply the manifest with `datumctl compute deploy -f FILE` or `datumctl apply -f FILE`, where `FILE` is the path to your manifest. You can't combine the configuration flags with `-f`. A flag-based deploy saves its result to `workload.yaml`, which you can edit as a starting point. Each flag-based deploy overwrites `workload.yaml` in the current directory, so keep edited manifests under another name. > [!WARNING] > > A flag-based `datumctl compute deploy` replaces the workload's whole template and placements with what the flags describe. It keeps environment variables, ConfigMap and Secret imports, volumes, and labels. It removes registry credentials, custom commands, and extra placements that you set in a manifest, and it sets the Instance count to the `--min` value, which defaults to `1`. After you add settings that the flags can't set, update the workload with `datumctl compute deploy -f FILE`. ## Set environment variables To set an environment variable to a fixed value, pass `--env KEY=VALUE` when you deploy. Repeat the flag for each variable. Values can contain commas. The following command sets two variables: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME \ --image=IMAGE \ --location=LOCATION \ --env=LOG_LEVEL=info \ --env=ALLOWED_ORIGINS=a.example,b.example ``` Replace the following: * `WORKLOAD_NAME`: the name of your workload. * `IMAGE`: the full image reference, including the registry host. * `LOCATION`: one or more locations, separated by commas. Variables carry forward. A later deploy that doesn't name a variable keeps its value. To remove a variable, add a trailing `-`, such as `--env=LOG_LEVEL-`. In a manifest, add the variable to the container's `env` list, as in the following example: ```yaml theme={null} containers: - name: api image: ghcr.io/example/api:1.4.2 env: - name: LOG_LEVEL value: info ``` ## Create a ConfigMap or Secret ConfigMaps hold non-sensitive settings, and Secrets hold sensitive values such as passwords and tokens. Both live in your project, and a workload can reference any ConfigMap or Secret in the same project. To create a ConfigMap and a Secret, follow these steps: 1. Save the following manifest as `api-config.yaml`: ```yaml theme={null} apiVersion: v1 kind: ConfigMap metadata: name: api-config data: config.yaml: | listen: "[::]:8080" cache_ttl: 60s REGION_LABEL: north-america --- apiVersion: v1 kind: Secret metadata: name: api-secrets type: Opaque stringData: db-password: PASSWORD ``` Replace `PASSWORD` with your database password. 2. To create the objects, run the following command: ```bash theme={null} datumctl apply -f api-config.yaml ``` For more information about Secrets in the portal, see [Secrets](/docs/operations/secrets). ## Set a variable from a ConfigMap or Secret key The `deploy` flags can't set one variable from one key, so use a manifest. To set one environment variable from one key, use `valueFrom` with `configMapKeyRef` or `secretKeyRef`. This method works with both [runtime classes](/docs/compute/runtimes/overview). The following example sets one variable from a Secret key and one from a ConfigMap key: ```yaml theme={null} env: - name: DATABASE_PASSWORD valueFrom: secretKeyRef: name: api-secrets key: db-password - name: REGION_LABEL valueFrom: configMapKeyRef: name: api-config key: REGION_LABEL ``` ## Set variables from a whole ConfigMap or Secret On `general-purpose` workloads, you can turn every key of a ConfigMap or Secret into an environment variable. To do so with flags, pass `--env-from-configmap` or `--env-from-secret` as `NAME` or `NAME:PREFIX_`. The following command loads every key from `api-config` with the prefix `APP_`, and every key from `api-secrets` without a prefix: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME \ --image=IMAGE \ --location=LOCATION \ --env-from-configmap=api-config:APP_ \ --env-from-secret=api-secrets ``` Replace the following: * `WORKLOAD_NAME`: the name of your workload. * `IMAGE`: the full image reference, including the registry host. * `LOCATION`: one or more locations, separated by commas. To remove an import, add a trailing `-` to the name, such as `--env-from-secret=api-secrets-`. In a manifest, use `envFrom`. Each entry names exactly one source. The optional `prefix` is added to each variable name, and must be a valid variable name itself. The following manifest example loads the same keys: ```yaml theme={null} envFrom: - configMapRef: name: api-config prefix: APP_ - secretRef: name: api-secrets ``` The `unikernel` runtime class doesn't support `envFrom`. On `unikernel` workloads, set each variable with `valueFrom` instead. ## Mount a ConfigMap or Secret as files To mount a ConfigMap or Secret as files with flags, pass `--configmap` or `--secret` as `NAME:/mount/path`. Each key becomes a file in the mount directory. The following command mounts `api-config` at `/etc/api` and `api-secrets` at `/etc/api-secrets`: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME \ --image=IMAGE \ --location=LOCATION \ --configmap=api-config:/etc/api \ --secret=api-secrets:/etc/api-secrets ``` Replace the following: * `WORKLOAD_NAME`: the name of your workload. * `IMAGE`: the full image reference, including the registry host. * `LOCATION`: one or more locations, separated by commas. To remove a mount, add a trailing `-` to the name, such as `--secret=api-secrets-`. In a manifest, declare a volume in `spec.template.spec.volumes`, and attach it to a container with `volumeAttachments`. The following manifest example mounts the same ConfigMap and Secret: ```yaml theme={null} spec: template: spec: runtime: sandbox: containers: - name: api image: ghcr.io/example/api:1.4.2 volumeAttachments: - name: config mountPath: /etc/api - name: secrets mountPath: /etc/api-secrets volumes: - name: config configMap: name: api-config - name: secrets secret: secretName: api-secrets ``` Keep the following rules in mind: * A ConfigMap volume uses `configMap.name`, but a Secret volume uses `secret.secretName`. * Attach every volume that you declare to at least one container, and give each attachment a `mountPath`. * To mount only some keys, or to choose file names, add `items` with a `key` and a relative `path` for each file. A `path` can't contain `..`. * To set file permissions, set `defaultMode`. ConfigMap and Secret volumes work with both runtime classes. ## Apply a configuration change Running Instances don't restart when a ConfigMap or Secret changes. To make your app pick up the new values, restart the workload: ```bash theme={null} datumctl compute restart WORKLOAD_NAME ``` Replace `WORKLOAD_NAME` with the name of your workload. The restart replaces Instances one at a time in each location. For more information, see [Placement and scaling](/docs/compute/placement-and-scaling#restart-instances). ## Use images from a private registry To pull an image from a private registry, store the registry credentials in a Secret and reference it in `imagePullSecrets`. This method works with both runtime classes. To use a private registry, follow these steps: 1. Save the following manifest as `registry-credentials.yaml`: ```yaml theme={null} apiVersion: v1 kind: Secret metadata: name: registry-credentials type: kubernetes.io/dockerconfigjson stringData: .dockerconfigjson: | {"auths":{"REGISTRY_HOST":{"username":"REGISTRY_USERNAME","password":"REGISTRY_TOKEN"}}} ``` Replace the following: * `REGISTRY_HOST`: the registry host name, such as `ghcr.io`. For Docker Hub, use `https://index.docker.io/v1/`. * `REGISTRY_USERNAME`: the user name for the registry. * `REGISTRY_TOKEN`: a password or access token with permission to pull the image. Use a read-only token. 2. To create the Secret, run the following command: ```bash theme={null} datumctl apply -f registry-credentials.yaml ``` 3. In your workload manifest, add the Secret to `spec.template.spec.runtime.sandbox.imagePullSecrets`: ```yaml theme={null} runtime: sandbox: imagePullSecrets: - name: registry-credentials containers: - name: api image: ghcr.io/example/api:1.4.2 ``` 4. To apply the manifest, run the following command: ```bash theme={null} datumctl compute deploy -f workload.yaml ``` Replace `workload.yaml` with your manifest file if it has a different name. A manifest deploy doesn't create or remove a public URL. To publish a workload that you manage with manifests, see [Publish a workload you manage with manifests](/docs/compute/compute-and-alb#publish-a-workload-you-manage-with-manifests). ## Complete example The following manifest combines the settings on this page for a `general-purpose` workload: ```yaml theme={null} apiVersion: compute.datumapis.com/v1alpha kind: Workload metadata: name: api spec: template: spec: runtime: class: general-purpose resources: instanceType: datumcloud/d1-standard-2 sandbox: imagePullSecrets: - name: registry-credentials containers: - name: api image: ghcr.io/example/api:1.4.2 args: ["--config", "/etc/api/config.yaml"] env: - name: LOG_LEVEL value: info - name: DATABASE_PASSWORD valueFrom: secretKeyRef: name: api-secrets key: db-password envFrom: - configMapRef: name: api-config prefix: APP_ ports: - name: http port: 8080 volumeAttachments: - name: config mountPath: /etc/api volumes: - name: config configMap: name: api-config networkInterfaces: - network: name: default placements: - name: us-east locations: - name: us-east-1 scaleSettings: minReplicas: 1 ``` To use this example on the `unikernel` class, change `runtime.class` to `unikernel` and remove the `envFrom` entry. ## Limits ConfigMaps and Secrets that a workload references have the following size limits: * 256 KiB for each ConfigMap or Secret. * 1 MiB in total for all the ConfigMaps and Secrets that a workload references in one location. If a workload exceeds a limit, Datum accepts the request, but the workload reports `ReferencedDataNotReady` and its Instances report `SourceTooLarge`. For other configuration errors, see [Compute troubleshooting](/docs/compute/troubleshooting). ## What's next * To serve the workload on a public URL, see [Publish a workload](/docs/compute/compute-and-alb). * To check status and fix problems, see [Compute operations](/docs/compute/operations) and [Troubleshooting](/docs/compute/troubleshooting). # Open a shell in an Instance Source: https://datum.net/docs/compute/instance-shell Run an interactive shell or a single command in a container of a running Instance, from the Cloud Portal or datumctl. A shell session connects you to a container of a running Instance, so you can run an interactive shell or a single command against it. Use it to inspect a process, read a file, or check a configuration without changing how the workload runs. > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. Shell sessions are available for `general-purpose` Instances only. A `general-purpose` Instance runs in its own virtual machine, so a shell session reaches a real container through that isolation boundary. `unikernel` Instances don't have a shell, so the **Shell** tab and `datumctl compute exec` aren't available for them. For more about the two classes, see [Choose a runtime class](/docs/compute/runtimes/overview). ## How a session works Opening a shell creates a session: a request to run one command, with or without a terminal, in one container of one Instance. The session has a time limit, and it ends when the command exits, when you close it, when it reaches its time limit, or when the platform can't keep it open. A session is single-use. Reconnecting after it ends, or running another command, starts a new session. Two ways to open a session: > [card] [Cloud Portal](/docs/compute/instance-shell#cloud-portal) > > Open an interactive shell from the Instance's **Shell** tab. > [card] [datumctl](/docs/compute/instance-shell#datumctl) > > Run `datumctl compute exec` for an interactive shell or a single command. ## Cloud Portal Open an interactive shell in a running `general-purpose` Instance from the [Datum Cloud portal](https://cloud.datum.net). ### Before you begin * Deploy a `general-purpose` workload with at least one `Available` Instance. For more information, see [Run a container image](/docs/compute/runtimes/general-compute). * Your account needs the **Compute Admin** role to open a shell. For more information, see [Permissions](#permissions). ### Open a shell 1. Open your project in the [Datum Cloud portal](https://cloud.datum.net). 2. Go to **Compute**, then **Workloads**. 3. Select the workload, then select the Instance you want a shell in. 4. Select the **Shell** tab. 5. Choose a **Container**. If the Instance runs only one container, it's already selected. 6. Enter a **Command**, or select one of the quick picks, `/bin/sh` or `/bin/bash`. The field defaults to `/bin/sh`. 7. Select **Connect**. The terminal connects and runs the command you chose. If the container has no shell, or doesn't have the command you asked for, the session ends immediately and the page explains why. For the full list of end reasons, see [Why a session ended](#why-a-session-ended). ### Open the shell in its own window Select **Open in new window** to continue the session in a console-only window, separate from the rest of the portal. The option is hidden on phones and other small screens, where the shell stays in the page instead. Opening a shell from a shared link only fills in the **Container** and **Command** fields. Nothing runs until you select **Connect** yourself. ### End a session Select **Close shell** to end the session and stop the command. Leaving the page or closing the tab also ends the session. ## datumctl `datumctl compute exec` runs a command, interactively or once, in a container of a running `general-purpose` Instance. ### Before you begin * Select a project, install the `compute` plugin, and get access to Compute. For more information, see [Compute commands](/docs/datumctl/compute/overview). * `exec` needs a version of the `compute` plugin that includes it. If `datumctl compute exec --help` doesn't show the command, upgrade the plugin: ```bash theme={null} datumctl plugin upgrade compute ``` For more on installing and upgrading plugins, see [Using plugins](/docs/datumctl/plugins/using-plugins). * Deploy a `general-purpose` workload with at least one `Available` Instance. For more information, see [Run a container image](/docs/compute/runtimes/general-compute). ### Open an interactive shell Run the following command, replacing `INSTANCE_NAME` with the name of the Instance: ```bash theme={null} datumctl compute exec INSTANCE_NAME -it -- sh ``` `-i` keeps standard input open, and `-t` allocates a terminal. Together, they give you an interactive shell. The following example opens one in an Instance named `api-dfw-0`: ```bash theme={null} datumctl compute exec api-dfw-0 -it -- sh ``` ### Run a single command Omit `-it` to run a command once and print its output, without an interactive shell: ```bash theme={null} datumctl compute exec api-dfw-0 -- cat /etc/os-release ``` ### Choose a container If the Instance runs more than one container, name the one to run the command in with `-c`: ```bash theme={null} datumctl compute exec api-dfw-0 -c worker -- ps aux ``` Without `-c`, the command runs in the Instance's only container. `general-purpose` Instances run exactly one container today, so you only need `-c` once an Instance runs more than one. ### Set the session time limit A command can run for up to 15 minutes by default. Use `--ttl` to choose a limit from 1 minute to 1 hour. The time starts when the command starts running: ```bash theme={null} datumctl compute exec api-dfw-0 --ttl 30m -it -- sh ``` The flag accepts durations such as `1m`, `30m`, and `1h`. If you omit it, the API's 15-minute default applies. A session still ends when its command exits or you close it, even if its time limit has not been reached. ### Exit codes `datumctl compute exec` passes through the result of the command or the session: | Exit code | Meaning | | - | - | | The command's own exit code | The command ran and exited on its own. | | `1` | The platform ended the session instead of the command exiting, for example because it expired or the Instance stopped running. `datumctl` prints why. | | `2` | A usage error, such as a missing instance name or command. | | `130` | You interrupted `datumctl` (Ctrl+C) before the command finished. | For the full list of reasons the platform can end a session, see [Why a session ended](#why-a-session-ended). For the full flag reference, run `datumctl compute exec --help`. ## Security * **End-to-end encryption.** The connection between your browser or `datumctl` and the Instance's container is encrypted all the way to the cell that runs the Instance. * **A one-time key that stays local.** Each session generates its own key pair. The private key never leaves your browser or your machine, and it isn't sent to Datum. * **Every session is recorded.** Opening, connecting, and ending a session each appear in the project's [Activity](/docs/operations/activity-logs), with the end reason and exit code. * **Containers in the same Instance share a trust domain.** Two shell sessions into the same Instance can see each other's processes and environment variables, because they run in the same container. Treat every shell into an Instance as having the same level of access as any other shell into it. * **Ending a session stops what it started, with one exception.** A process that detaches itself from the session, for example with `setsid`, can keep running after the session ends. A background job that was killed can also stay behind as a defunct process in a container whose main process doesn't reap its children. This is a known limitation. ## Permissions Creating a shell session requires the **Compute Admin** role, which includes `compute.datumapis.com/instanceconsolesessions.create`. The **Compute Viewer** role can list and inspect sessions, including who opened one and how it ended, but can't open or close one. For more on assigning roles, see [Assign roles](/docs/projects/service-accounts#assign-roles). ## Limits | Limit | Value | | - | - | | Open sessions per Instance | 3 | | Open sessions per project | 10 by default. Datum can grant a project more. Contact [support@datum.net](mailto:support@datum.net) to request an increase. | | Time to connect after creating a session | 60 seconds. A session nobody connects to by then ends with `NotConnected`. | | Session time limit | 15 minutes by default, up to 1 hour. | ## Why a session ended The **Shell** tab and `datumctl compute exec` both report why a session ended. The following table lists every reason: | Reason | Meaning | What to do | | - | - | - | | `Completed` | The command exited on its own. The exit code is included. | Nothing. This is a normal end. | | `ClosedByUser` | You closed the shell, or left or closed the page. | Nothing. Open a new session to continue. | | `Disconnected` | The connection broke before the command exited, so the platform stopped it. | Check your network connection and reconnect. | | `Expired` | The session reached its time limit. | Open a new session. With `datumctl`, use `--ttl` for a longer limit, up to 1 hour; see [Set the session time limit](#set-the-session-time-limit). | | `Revoked` | The session was closed from elsewhere, such as another client deleting it. | Open a new session if you still need one. | | `NotConnected` | Nothing connected within 60 seconds of creating the session. | Try again. If it keeps happening, check your network connection to Datum. | | `TooManySessions` | The Instance already has 3 open sessions. | Close another shell into the same Instance, then try again. | | `NoShell` | The container has no `sh`. Datum runs every command through a shell, so no command can run in a container without one. | Use an image that includes a shell, or run the command a different way, such as in the image's entrypoint. | | `CommandUnavailable` | The container doesn't have the command you asked for. | Check the command name and that it's installed in the image. | | `InstanceNotRunning` | The Instance or its container isn't running. | Wait for the Instance to become `Available`, then try again. See [Compute operations](/docs/compute/operations). | | `InstanceNotFound` | The Instance named in the session no longer exists, for example because it was replaced. | Open a new session against the current Instance. | | `AgentShutdown` | Datum closed the session for maintenance. | Connect again to continue. | | `AgentLost` | Datum lost contact with the session unexpectedly. | Connect again to continue. | | `Invalid` | The request couldn't be served as written. | Read the message that comes with this reason; it says why. | | `Unavailable` | No part of Datum took the session in time. | Try again shortly. | If your project's shell session limit is `0`, the **Shell** tab and `datumctl compute exec` both report that shell sessions aren't enabled for the project. Request an increase from [support@datum.net](mailto:support@datum.net). ## What's next * [Compute operations](/docs/compute/operations) * [Check Compute limits and quotas](/docs/compute/limits-and-quotas) # Compute limitations and roadmap Source: https://datum.net/docs/compute/limitations-and-roadmap Review what the Compute preview doesn't support yet and where to follow what Datum is building next. Compute is in preview and still being built. This page lists what it doesn't support yet. For quota and fixed limits, see [Limits and quotas](/docs/compute/limits-and-quotas). > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Limitations The Compute preview has the following limitations. For limitations of each [runtime class](/docs/compute/runtimes/overview), see [Run a container image](/docs/compute/runtimes/general-compute#limitations) and [Build and deploy a unikernel](/docs/compute/runtimes/unikernels#limitations). For limitations of the ALB, see [Publish a workload](/docs/compute/compute-and-alb#limitations). ### Runtime and sizing * One instance type is available, and you can't set CPU or memory for each container. * Persistent disks, raw devices, GPUs, and custom virtual machine images aren't available. Files written at runtime are lost when Datum replaces an Instance. * Instances with more than one container aren't supported. Run one container per workload. * Health probes, such as readiness and liveness checks, aren't available. An Instance is ready when its containers are running. * You can't suspend, resume, or snapshot Instances. You can open a shell or run a command in a running `general-purpose` Instance; see [Instance shell sessions](/docs/compute/instance-shell). * A workload's runtime class and network can't change after you create it. ### Placement and scaling * Compute runs in `us-central-1` (Dallas), `us-east-1` (Ashburn), and `us-west-1` (San Jose). * Workloads don't scale automatically, and they can't scale to zero. Don't set `scaleSettings.maxReplicas` or `scaleSettings.metrics`. * Rollouts replace Instances one at a time in each location, with no option to add extra Instances during the rollout. ### Networking * During the preview, Instances have private IPv6 addresses only. Private IPv4 addresses and public IP addresses are a preview limitation. * Private DNS names for Instances are on the [roadmap](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). * Firewall rules to filter traffic between Instances on a network are on the [roadmap](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). * Public access is HTTP and HTTPS only, served through Datum's [ALB](/docs/alb/overview). Raw TCP and UDP exposure is a preview limitation. For more information about networking, see [gVPC limitations and roadmap](/docs/galactic-vpc/limitations-and-roadmap). To learn which networking capabilities Datum is working on, see [Planned capabilities](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). ### Operations * Read Instance output and ALB access logs in the [Cloud Portal](https://cloud.datum.net) or with [`datumctl compute logs`](/docs/datumctl/compute/viewing-logs). * Compute publishes per-Instance CPU and memory measurements. See [Observability](/docs/compute/observability#metrics) for the metric names and queries. ## Roadmap Networking capabilities for Compute Instances, such as private DNS and firewall rules, are tracked in [gVPC planned capabilities](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). For the overall Datum roadmap, see [Platform roadmap](/docs/platform/roadmap). > [!NOTE] > > Plans can change. This list isn't a commitment to deliver any feature or to meet any timeline. ## What's next * To check quota and fixed limits, see [Limits and quotas](/docs/compute/limits-and-quotas). * To fix common problems, see [Troubleshooting](/docs/compute/troubleshooting). # Compute limits and quotas Source: https://datum.net/docs/compute/limits-and-quotas Request access to Compute, check your project's compute quota, and review the fixed limits of the Compute preview. For features that Compute doesn't support yet, see [Limitations and roadmap](/docs/compute/limitations-and-roadmap). > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Request access Compute is enabled for each project separately, and Datum approves each request manually. Until a project has access, every `datumctl compute` command except `access` and `help` stops and asks you to request access. This includes `build`. To request access for the active project, follow these steps: 1. To check the current state, run the following command: ```bash theme={null} datumctl compute access ``` The `Status` line shows a state such as `Not requested`, `Pending approval`, `Active`, `Denied`, or `Revoked`. 2. To submit a request and wait for a decision, run the following command: ```bash theme={null} datumctl compute access request --message="MESSAGE" --wait ``` Replace `MESSAGE` with a short description of what you plan to run. The message helps Datum review your request. 3. Wait until the `Status` line shows `Active`. You can then deploy workloads. If a request is denied or access is revoked, submit a new request with `datumctl compute access request --renew`. For all access options, see [Compute commands](/docs/datumctl/compute/overview#requesting-access). ## Quotas Your project's compute quota limits how many workloads and Instances you can run, how much vCPU and memory they use, and how many shell sessions can be open at once. Datum sets the quota. You can't change it yourself. Each Instance uses the following quota: * 1 Instance * 1 vCPU * 2 GiB (2,048 MiB) of memory Quota is shared across [runtime classes](/docs/compute/runtimes/overview). A `general-purpose` Instance and a `unikernel` Instance use the same amount. ### Check your quota To see your project's limits and usage, run the following command: ```bash theme={null} datumctl compute quota ``` The output is similar to the following: ```text theme={null} Quota for project your-project-id RESOURCE UNIT LIMIT USED AVAILABLE USAGE Workloads workloads 10 3 7 [######--------------] 30% Instances instances 50 12 38 [#####---------------] 24% vCPUs vCPUs 32 8 24 [#####---------------] 25% Memory MiB 65536 16384 49152 [#####---------------] 25% ``` To show only the resources that are at their limit, add `--constrained`. ### If your quota is used up If your project doesn't have enough quota for a new Instance, Datum doesn't start it. The Instance shows `Pending (quota exceeded)` in `datumctl compute instances` and reports `QuotaExceeded`, and the workload reports `QuotaNotGranted`. To let the Instance start, do one of the following: * Lower the number of Instances with `datumctl compute scale WORKLOAD_NAME --min=COUNT`. * Delete workloads that you no longer need. * Ask for more quota by contacting [support@datum.net](mailto:support@datum.net). If an Instance reports `QuotaNoBudget`, or another reason that starts with `Quota`, your project's quota isn't set up correctly. The workload exists, but its Instances don't start. Contact support, and Datum fixes the problem. ## Limits The following table lists fixed limits: | Item | Limit | | - | - | | Instance type | `datumcloud/d1-standard-2` only: 1 vCPU and 2 GiB of memory | | Instances in each location of a [placement](/docs/compute/placement-and-scaling) (`minReplicas`) | 1 to 1,000 | | Network interfaces for each Instance | Exactly 1 | | Networks for each workload | 1 | | Size of each referenced ConfigMap or Secret | 256 KiB | | Total size of referenced ConfigMaps and Secrets for a workload in one location | 1 MiB | | ALBs attached to each workload with `datumctl compute deploy` | 1, on one port | | Unikernel image size | The unpacked image must fit in the Instance's 2 GiB of memory, along with the app. | | Open shell sessions per Instance | 3 | | Open shell sessions per project | 10 by default. Datum can grant a project more. | ## What's next * To deploy your first workload, see [Deploy an app](/docs/getting-started/first-workload). * To review what Compute doesn't support yet, see [Limitations and roadmap](/docs/compute/limitations-and-roadmap). * To fix quota errors and other problems, see [Troubleshooting](/docs/compute/troubleshooting). # Monitor Compute workloads Source: https://datum.net/docs/compute/observability Use metrics and logs to understand your Compute workloads and Instances. ## Metrics The [Cloud Portal](https://cloud.datum.net) shows CPU and memory usage on Compute workload and Instance pages. Open the **Metrics** tab for a longer view. Datum also publishes the measurements to your project's metrics store so you can query them or [export them to Grafana Cloud](/docs/operations/metrics-export). ![The Metrics tab of a workload with totals for CPU, memory, requests, p99 latency, and errors, and CPU and memory charts with one line for each Instance](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/compute-workload-metrics.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=750773e3d577bb5a2b6b68cc43963c1c) A workload's Metrics tab, with CPU and memory for each Instance ### CPU and memory series | Metric | Type | Meaning | | - | - | - | | `datum_compute_instance_cpu_usage_seconds_total` | Counter | CPU time used, in seconds. Apply `rate()` to see CPU usage in cores. | | `datum_compute_instance_memory_working_set_bytes` | Gauge | Memory working set, in bytes. | Each series measures one container in an Instance. These measurements don't represent total virtual machine memory or include a separate virtual machine overhead measurement. An Instance's quota and configured size also differ from measured usage. Use these labels to select the series you need: | Label | Meaning | | - | - | | `resourcemanager_datumapis_com_project_name` | Project ID. | | `resource_name` | Instance name, as shown by `datumctl compute instances`. This is not the workload name. | | `instance_container` | Container name within the Instance. Use this label for a per-container breakdown. The metrics store doesn't expose the raw Prometheus `container` label. | | `region` | Location where the Instance runs, such as `us-central-1`. | | `runtime_class` | `general-purpose` on standard container Instances. | ### Query an Instance Replace `PROJECT_ID` and `INSTANCE_NAME` with your project ID and an Instance name returned by `datumctl compute instances --workload=WORKLOAD_NAME`. Run these [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/) queries in your metrics tool: ```promql theme={null} sum(rate(datum_compute_instance_cpu_usage_seconds_total{resourcemanager_datumapis_com_project_name="PROJECT_ID",resource_name="INSTANCE_NAME"}[5m])) ``` The CPU result is in cores. For example, `0.5` means approximately half of one CPU core over the five-minute window. ```promql theme={null} sum(datum_compute_instance_memory_working_set_bytes{resourcemanager_datumapis_com_project_name="PROJECT_ID",resource_name="INSTANCE_NAME"}) ``` The memory result is in bytes. These queries add the container measurements for the selected Instance. Keep the project selector because Instance names alone aren't unique across projects. To see each container's contribution, retain `instance_container` in the result: ```promql theme={null} sum by (instance_container) (rate(datum_compute_instance_cpu_usage_seconds_total{resourcemanager_datumapis_com_project_name="PROJECT_ID",resource_name="INSTANCE_NAME"}[5m])) sum by (instance_container) (datum_compute_instance_memory_working_set_bytes{resourcemanager_datumapis_com_project_name="PROJECT_ID",resource_name="INSTANCE_NAME"}) ``` If an Instance has more than one container, each has its own series. Sum the series once for an Instance total; don't add the container breakdown to that total again. To graph a whole workload, get its Instance names with `datumctl compute instances --workload=WORKLOAD_NAME`, then select those names in `resource_name`. The metrics don't carry a workload-name label. ### Workload and Instance status series Datum also publishes the state of your workloads and Instances, so you can alert when a workload falls short of the Instances it wants. These are the same numbers that `datumctl compute workloads` shows: ```text theme={null} $ datumctl compute workloads NAME LOCATIONS HEALTH READY UP-TO-DATE PLACEMENTS api us-east-1, us-central-1, us-west-1 Available 6/6 6/6 us-east, us-central, us-west api-postgres us-east-1 Available 1/1 1/1 us-east ``` | Metric | What it reports | | - | - | | `datum_cloud_compute_workload_status_replicas_desired` | Instances the workload wants. | | `datum_cloud_compute_workload_status_replicas_ready` | Instances that are ready. | | `datum_cloud_compute_workload_status_replicas_updated` | Instances running the workload's latest template. During a rollout, this climbs to the desired count. | | `datum_cloud_compute_workload_placement_status_replicas_desired` | Instances a placement wants, with a `placement` label. | | `datum_cloud_compute_workload_placement_status_replicas_ready` | Instances that are ready in a placement, with a `placement` label. | | `datum_cloud_compute_workload_status_condition` | `1` when a workload condition is true. The `condition` label is the condition, such as `Available`, and `reason` explains it. | | `datum_cloud_compute_instance_status_condition` | `1` when an Instance condition is true, with `condition` and `reason` labels. | | `datum_cloud_compute_instance_info` | One series for each Instance, with its `location` and `runtime_class`. | On these series, `resource_name` is the workload or Instance name, and `resourcemanager_datumapis_com_project_name` is your project ID. To find placements that are short of Instances, replace `PROJECT_ID`: ```promql theme={null} datum_cloud_compute_workload_placement_status_replicas_desired{resourcemanager_datumapis_com_project_name="PROJECT_ID"} - datum_cloud_compute_workload_placement_status_replicas_ready{resourcemanager_datumapis_com_project_name="PROJECT_ID"} > 0 ``` ### Traffic to published workloads A workload published through an ALB has the same traffic, latency, and firewall metrics as any other load balancer. Select them with `gateway_name` set to the load balancer's name, which `datumctl alb list` shows. See [ALB metrics](/docs/alb/observability#metrics). ## Logs Open a workload's **Logs** tab in the Cloud Portal to see output from its Instances. If the workload is published through an ALB, the same view also includes ALB access logs. Open an Instance's **Logs** tab to focus on that Instance's output and related ALB requests. The Overview pages show recent logs, and the Logs tabs provide a larger explorer with time and search controls. ![The Logs tab of a workload with filters for source, method, status code, host, and Instance, and a table mixing ALB requests and Instance output](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/compute-workload-logs.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=3d9c39bc6d8a1fee7f6d32cd22ef7f9c) A workload's Logs tab, with Instance output and ALB access logs together Instance output appears after an Instance starts writing to standard output. ALB access logs require a published workload and requests to its public endpoint. If you don't see expected entries, check that the Instance is running, select a time range that includes the activity, and confirm that requests reached the ALB. To read logs from a terminal, use `datumctl compute logs`. Each line starts with the Instance it came from: ```text theme={null} $ datumctl compute logs api --since 10m --tail 3 [us-west-us-west-1-0/api] Oct 20 01:32:26.375 "POST /submit HTTP/1.1" 303 223 "-" "curl/8.18.0" [us-central-us-central-1-0/api] Oct 20 01:32:35.966 INFO app: processed order 1042 [us-west-us-west-1-1/api] Oct 20 01:32:42.376 "GET /orders HTTP/1.1" 200 2185 "-" "curl/8.18.0" ``` Useful flags: * `-f` follows new lines as they arrive. * `--location us-east-1` and `--instance NAME` narrow the logs to some Instances. * `--search error` shows only lines that contain the text. * `--previous` shows the logs of each Instance's previous generation, such as before a crash or restart. * `--alb` shows the access logs of the workload's load balancer instead. For more, see [View Compute logs with datumctl](/docs/datumctl/compute/viewing-logs). To find who changed a workload's configuration, see [Activity logs](/docs/operations/activity-logs). Activity logs record resource changes; the Logs tab shows output from running Instances and requests to the ALB. # Compute operations Source: https://datum.net/docs/compute/operations Check the status of Compute workloads and Instances, update them, and delete them. This page shows how to check on a running workload, change it, and delete it. To find out why an Instance isn't running, see [Troubleshooting](/docs/compute/troubleshooting). > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Before you begin * Select a project, install the `compute` plugin, and get access to Compute. For more information, see [Compute commands](/docs/datumctl/compute/overview). ## Check workload status To list the workloads in your project, run the following command: ```bash theme={null} datumctl compute workloads ``` The `HEALTH` column shows `Available`, `Degraded`, `Unavailable`, or `Unknown`. The `READY` column shows how many Instances are ready out of how many the workload wants, and `UP-TO-DATE` shows how many run the latest template. To filter the list, add `--health` with a health value in lowercase, such as `--health=degraded`, or `--location` with a location, such as `--location=us-central-1`. To see the configuration, ALB hostname, and per-location health of one workload, run the following command: ```bash theme={null} datumctl compute workloads describe WORKLOAD_NAME ``` Replace `WORKLOAD_NAME` with the name of your workload. In the [Cloud Portal](https://cloud.datum.net), **Compute** > **Workloads** lists the same workloads, with each one's health, recent CPU and memory, network, and locations. ![The Workloads page with totals for workloads, Instances, healthy workloads, and requests, and a table of workloads with status, CPU and memory sparklines, network, and locations](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/compute-workloads-list.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=e8eea062cc281be1fdcf206b21313968) Workloads in the Cloud Portal The workload's `Available` condition also has a reason, which you can see with `datumctl get workloads`. When a workload isn't available, the reason points to the cause, and the Instance has the details. ## Check Instances To list the Instances of a workload, run the following command: ```bash theme={null} datumctl compute instances --workload=WORKLOAD_NAME ``` To list Instances in one location, add `--location` with a location, such as `--location=us-central-1`. To add the instance type column, add `-o wide`. The `STATUS` column describes each Instance in plain language: | Status | Meaning | | - | - | | `Available` | The Instance is running. | | `Starting` | Datum is starting the Instance. | | `Pending (quota exceeded)` | Your project doesn't have enough compute quota for the Instance. | | `Pending (REASON)` | The Instance is waiting on something else, named by `REASON`. | | `Failed (image unavailable)` | Datum couldn't pull the image. | | `Failed (crashing)` | The program starts and then exits, repeatedly. | | `Failed (configuration error)` | Datum refused the Instance's configuration before the program ran. | To see an Instance's conditions, network addresses, and a plain-language explanation of any failure, run the following command: ```bash theme={null} datumctl compute instances describe INSTANCE_NAME ``` Replace `INSTANCE_NAME` with a name from the `NAME` column of `datumctl compute instances`. To open an interactive shell or run a command in a running Instance, see [Instance shell sessions](/docs/compute/instance-shell). To inspect CPU and memory usage or read Instance output, see [Observability](/docs/compute/observability). ## Update a workload To change the image or placement of a workload that you deployed with flags, run `datumctl compute deploy` again with the same name and the new values. For example, the following command deploys a new image version: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME --image=IMAGE --location=LOCATION --min=COUNT ``` Replace the following: * `WORKLOAD_NAME`: the name of your workload. * `IMAGE`: the new image reference. * `LOCATION`: the locations that the workload runs in. * `COUNT`: the number of Instances to run in each location. > [!WARNING] > > A flag-based `datumctl compute deploy` replaces the workload's whole template and placements with what the flags describe. It keeps environment variables, ConfigMap and Secret imports, volumes, and labels. It removes registry credentials, custom commands, and extra placements that you set in a manifest, and it sets the Instance count to the `--min` value, which defaults to `1`. Once you manage a workload with a manifest, update it with `datumctl compute deploy -f FILE`. To update a workload from a manifest, edit the manifest and run `datumctl compute deploy -f FILE`, where `FILE` is the path to your manifest. The command shows the changes and asks for confirmation before it applies them. > [!TIP] > > Push each new image version with a new tag, or reference it by digest, such as `ghcr.io/example/api@sha256:...`. Hosts cache images by tag, so pushing a new image under a tag you already deployed might not reach every Instance. Datum rolls out template changes one Instance at a time in each location. For more information, see [How rollouts work](/docs/compute/placement-and-scaling#how-rollouts-work). To restart Instances without changing the template, see [Restart Instances](/docs/compute/placement-and-scaling#restart-instances). You can't change the following settings after you create a workload: * The workload name * The [runtime class](/docs/compute/runtimes/overview) * The network, and the network interface's `name`, `ipFamilies`, `addresses`, and `reclaimPolicy` To change one of these settings, deploy a new workload with a different name, and then delete the old one. ## Delete a workload Deleting a workload with `datumctl compute destroy` also deletes its Instances, its ALB, and the network service that selects it, even if you created the network service separately. > [!WARNING] > > You can't undo the deletion. Anything that an Instance stored on its own filesystem is lost. To delete a workload, follow these steps: 1. Run the following command: ```bash theme={null} datumctl compute destroy WORKLOAD_NAME ``` Replace `WORKLOAD_NAME` with the name of your workload. 2. Review the summary of placements, locations, Instances, and the ALB's hostnames that the command prints. 3. Enter `y` to confirm. The command prints `workload/WORKLOAD_NAME deleted.` To skip the confirmation in scripts, add `-y`. If you delete a workload with `datumctl delete workload`, Datum deletes the workload and its Instances, but leaves in place any ALB that `datumctl compute deploy` created. Use `datumctl compute destroy` to remove both. ## What's next * To open a shell in a running Instance, see [Instance shell](/docs/compute/instance-shell). * To fix common problems, see [Troubleshooting](/docs/compute/troubleshooting). * For command details, see [Deploying workloads](/docs/datumctl/compute/deploying-workloads), [Operations](/docs/datumctl/compute/scaling-and-operations), and [Destroying workloads](/docs/datumctl/compute/destroying-workloads). # Compute Source: https://datum.net/docs/compute/overview Run your apps as isolated Instances in Datum Cloud locations, from a standard container image or a fast-starting unikernel. > [!NOTE] > > **Working with this product via an AI agent?** [Datum MCP](/docs/agents/datum-mcp) gives agents live access to workloads and Instances through its `compute` toolset, including Compute's own diagnostic tools. For an open-ended question such as "why isn't my workload available?", agents can ask [Patch](/docs/datumctl/ai/assistant) with `datumctl assistant chat`. Datum Cloud Compute runs your app close to your users. You describe what to run and where, and Datum starts isolated Instances in the locations you choose, connects them to a private network, and can serve them publicly through Datum's Application Load Balancer (ALB). > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. Your project needs approved access to Compute before you can deploy, and Datum reviews access requests manually. For details, see [Limits and quotas](/docs/compute/limits-and-quotas). Every Instance runs in its own virtual machine with its own kernel. No kernel is shared between tenants. Compute is a good fit if you want to: * Run an HTTP service or API in more than one location without operating servers. * Deploy a container image you already build, with virtual machine isolation. * Start small, purpose-built services in a fraction of a second as unikernels. ## How it works You create a *workload*. A workload holds a template for the Instances you want (image, command, environment variables, and ports) and one or more *placements* that say where to run them and how many to run in each location. For each location in a placement, Datum starts the requested number of Instances there. Every Instance attaches to a [network](/docs/galactic-vpc/overview) in your project and receives a private IPv6 address. To reach a workload from the internet, attach Datum's [Application Load Balancer (ALB)](/docs/alb/overview) to it. The following diagram shows a workload with one placement in two locations: ```text theme={null} Workload ├── Placement "default" (us-central-1, us-east-1; 2 Instances per location) │ ├── us-central-1: Instance 0, Instance 1 │ └── us-east-1: Instance 0, Instance 1 └── Network "default" → private IPv6 address per Instance ``` Each Instance reports its own status, and the workload summarizes the status of its Instances. When something goes wrong, the Instance has the most specific reason. ## Key concepts | Concept | Description | | - | - | | Workload | The resource you create. It defines the Instance template and the placements. | | Placement | A named set of locations and the minimum number of Instances to run in each one. | | Location | A Datum Cloud region where Instances run, such as `us-central-1` (Dallas). | | Instance | One running copy of your workload in one location. Each Instance has 1 vCPU and 2 GiB of memory. | | Runtime class | The execution tier for a workload's Instances: `general-purpose` or `unikernel`. You choose it when you create the workload. | | Network | The private network your Instances attach to. `datumctl compute deploy` uses a network named `default`. | | ALB | Datum's [Application Load Balancer](/docs/alb/overview), which serves public HTTP and HTTPS traffic to your workload's Instances. | ## Runtime classes Compute offers two runtime classes. Both give each Instance its own kernel. * **General purpose (`general-purpose`):** Runs standard Linux container images without changes. Instances take several seconds to start. * **Unikernel (`unikernel`):** Runs a specially packaged image as a unikernel that starts in a fraction of a second, with very little overhead per Instance. You package the image with `datumctl compute build`. If you don't name a runtime class, Datum uses `unikernel`. For a detailed comparison, see [Choose a runtime class](/docs/compute/runtimes/overview). ## Ways to work with Compute * **`datumctl compute`:** A `datumctl` plugin that deploys, publishes, scales, and deletes workloads with a single command each. The guides in this section use it. For the command reference, see [Compute commands](/docs/datumctl/compute/overview). * **Manifests:** Workloads are Kubernetes-style resources. You can write them in YAML and manage them with `datumctl apply`, `datumctl get`, and `datumctl describe`, or with GitOps tooling. For more information, see [Kubernetes](/docs/platform/kubernetes). ## What's next > [card] [Deploy an app](/docs/getting-started/first-workload) > > Deploy draw\.io with one command and open it through Datum's ALB. > [card] [Choose a runtime class](/docs/compute/runtimes/overview) > > Compare the general-purpose and unikernel tiers. > [card] [Run a container image](/docs/compute/runtimes/general-compute) > > Run a standard Linux container on general-purpose Instances. > [card] [Build and deploy a unikernel](/docs/compute/runtimes/unikernels) > > Package your app as a unikernel with `datumctl compute build`. > [card] [Configure a workload](/docs/compute/configuration) > > Set environment variables and mount ConfigMaps and Secrets. > [card] [Placement and scaling](/docs/compute/placement-and-scaling) > > Choose locations, set Instance counts, and roll out changes. > [card] [Publish a workload](/docs/compute/compute-and-alb) > > Serve a workload with Datum's ALB, on a generated or custom hostname. > [card] [Operations](/docs/compute/operations) > > Check status, update, and delete workloads. > [card] [Troubleshooting](/docs/compute/troubleshooting) > > Fix common errors and Instance failures. > [card] [Observability](/docs/compute/observability) > > View CPU and memory metrics, Instance output, and ALB access logs. > [card] [Instance shell](/docs/compute/instance-shell) > > Open an interactive shell from the Cloud Portal or run a command with `datumctl`. > [card] [Limits and quotas](/docs/compute/limits-and-quotas) > > Request access, check quota, and review fixed limits. > [card] [Limitations and roadmap](/docs/compute/limitations-and-roadmap) > > Review what the preview doesn't support yet. > [card] [gVPC](/docs/galactic-vpc/overview) > > Create private networks and attach workloads to them. > [card] [Private DNS](/docs/compute/private-dns) > > Reach Instances by private name instead of IP address. # Placement and scaling Source: https://datum.net/docs/compute/placement-and-scaling Choose the locations your workload runs in, set how many Instances run in each, and understand how Datum rolls out changes. A workload's placements decide where its Instances run and how many run in each location. This page shows how to choose locations, change the number of Instances, and roll out changes safely. > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Before you begin * Select a project, install the `compute` plugin, and get access to Compute. For more information, see [Compute commands](/docs/datumctl/compute/overview). ## Locations Compute runs in the following locations: | Location | City | City code | Runtime classes | | - | - | - | - | | `us-central-1` | Dallas | `DFW` | `general-purpose`, `unikernel` | | `us-east-1` | Ashburn | `IAD` | `general-purpose`, `unikernel` | | `us-west-1` | San Jose | `SJC` | `general-purpose`, `unikernel` | If you name a location where Compute isn't offered to your project, Datum rejects the request, and the error lists the locations that you can use. For more information about Datum locations and region codes, see [Locations](/docs/locations). ## Choose locations `datumctl compute deploy` accepts exactly one of the following flags: * `--location`: one or more location names, separated by commas, such as `--location=us-central-1,us-east-1`. * `--city`: every location in one or more cities, such as `--city=DFW,IAD`. * `--location-selector`: every location whose labels match a selector, such as `--location-selector='topology.datum.net/region=us-east-1'`. For example, the following command runs two Instances in Dallas and two in Ashburn: ```bash theme={null} datumctl compute deploy api \ --image=ghcr.io/example/api:1.4.2 \ --runtime-class=general-purpose \ --location=us-central-1,us-east-1 \ --min=2 ``` The flags put all locations in a single placement named `default`. On an existing workload, a flag-based deploy replaces every placement and some other settings. For more information, see [Update a workload](/docs/compute/operations#update-a-workload). ### Choose locations in a manifest In a manifest, each entry in `spec.placements` has a `name`, a `scaleSettings` block, and exactly one of the following fields: * `locations`: a list of location names. * `locationSelector`: a label selector on location labels. Use `topology.datum.net/city-code` to select by city code, or `topology.datum.net/region` to select by region. Datum reevaluates the selector when Datum adds or removes a location. A workload can have more than one placement, for example to run a different number of Instances in different locations. The following manifest runs two Instances in Dallas and one Instance in each of Ashburn and San Jose: ```yaml theme={null} apiVersion: compute.datumapis.com/v1alpha kind: Workload metadata: name: api spec: template: spec: runtime: class: general-purpose resources: instanceType: datumcloud/d1-standard-2 sandbox: containers: - name: api image: ghcr.io/example/api:1.4.2 ports: - name: http port: 8080 networkInterfaces: - network: name: default placements: - name: dallas locations: - name: us-central-1 scaleSettings: minReplicas: 2 - name: coasts locationSelector: matchExpressions: - key: topology.datum.net/city-code operator: In values: ["IAD", "SJC"] scaleSettings: minReplicas: 1 ``` If a selector matches no location where Compute is offered, Datum rejects the workload when you apply it. ## Set the number of Instances `minReplicas` is the number of Instances that Datum runs in each location of a placement. A placement with `minReplicas: 2` across two locations runs four Instances. The value must be between 1 and 1,000. To change the number of Instances for every placement of a workload, run the following command: ```bash theme={null} datumctl compute scale WORKLOAD_NAME --min=COUNT ``` Replace the following: * `WORKLOAD_NAME`: the name of your workload. * `COUNT`: the number of Instances to run in each location. To set different counts for different placements, edit `scaleSettings.minReplicas` for each placement in the manifest and apply it with `datumctl compute deploy -f workload.yaml`. Each Instance counts against your project's compute quota. If a new Instance doesn't start because the quota is used up, it reports `QuotaExceeded`. For more information, see [Limits and quotas](/docs/compute/limits-and-quotas). Workloads don't scale automatically, and they can't scale to zero. The number of Instances stays at `minReplicas` until you change it. Don't set `scaleSettings.maxReplicas` or `scaleSettings.metrics`, because autoscaling isn't supported. ## How rollouts work When you change a workload's template, such as its image, command, or environment variables, Datum rolls out the change to every location. In each location, Datum does the following: 1. Deletes one Instance, starting with the highest-numbered one. 2. Creates a replacement from the new template. 3. Waits until the replacement is ready before it moves on to the next Instance. Datum replaces Instances instead of updating them in place, so each location has one fewer ready Instance while its replacement starts. If a location runs only one Instance, that location has no ready Instance during the rollout. To keep serving during rollouts, run at least two Instances in each location. If a replacement Instance never becomes ready, the rollout in that location stops and waits. The remaining old Instances keep running. To watch a rollout, run the following command: ```bash theme={null} datumctl compute rollout WORKLOAD_NAME ``` Replace `WORKLOAD_NAME` with the name of your workload. The command shows each placement and location with the phase `Pending`, `Updating`, `Done`, or `Blocked`. A location is `Blocked` when it makes no progress for more than 30 seconds, and the command prints the reason. To stop watching, press Control+C. The rollout continues. ## Restart Instances A restart replaces every Instance with the same template, one at a time in each location, in the same way as a rollout. Restart a workload to pick up a changed ConfigMap or Secret, or to recover from a bad state. To restart a workload, run the following command: ```bash theme={null} datumctl compute restart WORKLOAD_NAME ``` Replace `WORKLOAD_NAME` with the name of your workload. `restart` replaces every Instance of the workload in every location. ## What's next * To route each request to the nearest location, see [Publish a workload](/docs/compute/compute-and-alb). * To check status and fix problems, see [Compute operations](/docs/compute/operations) and [Troubleshooting](/docs/compute/troubleshooting). * For command details, see [Operations](/docs/datumctl/compute/scaling-and-operations). # Private DNS for Compute Source: https://datum.net/docs/compute/private-dns How Compute publishes stable private names for instances attached to a Datum VPC. Compute integrates with [Private DNS](/docs/domain-dns/private-dns) to publish a predictable name for an instance in each VPC to which it is attached. The name resolves to the instance's programmed private IPv4 and IPv6 addresses and is visible only within the associated VPC's DNS context. > [!NOTE] > > The Compute integration is under active development and is not yet a stable, > generally available interface. ## Instance names Compute publishes instance identity names beneath the `instances` subtree of the VPC's managed namespace. A name has this general form: ```text theme={null} .instances. ``` Compute normalizes the instance name into a valid DNS label when necessary. If two instance names normalize to the same DNS name, Private DNS keeps the first claim and reports a name conflict for the other instance. Treat the complete name reported by the service as an output. The managed suffix is a platform allocation and should not be constructed or parsed by applications. The allocated hostname and its publication state are reported for each network in `Instance.status.dns`. ## Published addresses Compute publishes: * `A` records for private IPv4 addresses. * `AAAA` records for private IPv6 addresses. An address is eligible only while the instance is ready, its runtime is ready, and its network interface reports that the address is both allocated and programmed. Public addresses and delegated prefixes are not published as instance identity records. Instance readiness covers the managed runtime and network attachment. Use the application's own health checks or a service-specific discovery mechanism when traffic should be sent only to healthy application endpoints. ## Lifecycle behavior Compute continuously reconciles the DNS records from current instance and network state: * Address changes update the records behind the stable identity name. * An unavailable or deprogrammed interface withdraws its addresses. * Deleting the instance removes its published records. * Resource lifetime checks prevent an old instance from publishing addresses after it has been replaced. * An instance attached to more than one VPC receives independently isolated DNS state for each VPC. Publication is asynchronous. A running instance can become ready before its name is visible in every resolver, and clients can retain an older answer until the DNS TTL expires. ## Network access Resolving an instance name does not grant access to the instance. The client must still have a valid route and be permitted by the applicable network policy. Clients outside the associated VPC context cannot resolve the private instance name through public DNS. # Run a container image Source: https://datum.net/docs/compute/runtimes/general-compute Run a standard Linux container image on general-purpose Instances, each isolated in its own lightweight virtual machine. The `general-purpose` [runtime class](/docs/compute/runtimes/overview) runs standard Linux container images without changes. Each Instance runs in its own lightweight virtual machine with its own kernel, so the image gets the compatibility of a container and the isolation of a virtual machine. > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Before you begin * Select a project, install the `compute` plugin, and get access to Compute. For more information, see [Compute commands](/docs/datumctl/compute/overview). * Push your image to a registry that Compute can reach, or use a public image. ## Image requirements A container image runs on `general-purpose` Instances if it meets the following requirements: * **The image reference includes a registry host:** Use `docker.io/library/redis:7` or `ghcr.io/example/api:1.4.2`, not `redis:7`. Datum rejects references without a registry host. * **The app listens on IPv6:** Instances have IPv6 addresses only. An app that listens only on an IPv4 address, such as `0.0.0.0`, is unreachable. Configure it to listen on `::` (all IPv6 addresses), for example `[::]:8080`. * **The app serves plain HTTP if you publish it:** Datum terminates TLS for public URLs and connects to your Instances over plain HTTP. For more information, see [Publish a workload](/docs/compute/compute-and-alb). * **Private images have credentials:** To pull from a private registry, add registry credentials to the workload. For more information, see [Use images from a private registry](/docs/compute/configuration#use-images-from-a-private-registry). You don't need `datumctl compute build` for `general-purpose` workloads. Build and push your image with the tools that you already use. ## Deploy a container image To deploy an image and publish it on a public URL, run the following command: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME \ --image=IMAGE \ --runtime-class=general-purpose \ --location=LOCATION \ --http-port=PORT ``` Replace the following: * `WORKLOAD_NAME`: a name for your workload. * `IMAGE`: the full image reference, including the registry host. * `LOCATION`: one or more locations, separated by commas, such as `us-central-1,us-east-1`. * `PORT`: the port that your container serves HTTP on. To run the workload without a public URL, omit `--http-port`. Always pass `--runtime-class=general-purpose` when you create the workload. Without it, Datum uses the `unikernel` class, and a standard image doesn't boot. ## Deploy from a manifest Use a manifest when you need settings that the `deploy` flags don't expose, such as a custom command, a second placement, or a variable set from one key of a Secret. A flag-based deploy saves its result to `workload.yaml`, which you can edit as a starting point. Each flag-based deploy overwrites `workload.yaml` in the current directory, so keep edited manifests under another name. The following manifest runs Redis in Dallas with a custom command: ```yaml theme={null} apiVersion: compute.datumapis.com/v1alpha kind: Workload metadata: name: cache spec: template: spec: runtime: class: general-purpose resources: instanceType: datumcloud/d1-standard-2 sandbox: containers: - name: redis image: docker.io/library/redis:7 command: ["redis-server"] args: ["--bind", "::", "--protected-mode", "no"] ports: - name: redis port: 6379 networkInterfaces: - network: name: default placements: - name: dallas locations: - name: us-central-1 scaleSettings: minReplicas: 1 ``` The arguments make Redis listen on IPv6 and accept connections from other Instances on the network without a password. Set a password for anything beyond testing. The manifest sets the following fields. Paths are relative to `spec.template.spec`. * `runtime.class`: the runtime class. Set it to `general-purpose`. * `runtime.resources.instanceType`: the Instance size. `datumcloud/d1-standard-2` (1 vCPU, 2 GiB) is the only type. * `sandbox.containers[].command` and `args`: override the image's entrypoint and command, in the same way as in Kubernetes. * `sandbox.containers[].ports`: the ports the container listens on. A port on its own doesn't make the workload reachable from the internet. * `networkInterfaces`: the network the Instances attach to. Every workload needs exactly one network interface. To apply the manifest, run the following command: ```bash theme={null} datumctl compute deploy -f cache.yaml ``` The command shows what changes, asks for confirmation, and then waits for the rollout. You can also apply the manifest with `datumctl apply -f cache.yaml`, which doesn't wait for the rollout. ## Control Linux capabilities Every container starts with all Linux capabilities dropped. When a container doesn't set a security context, Datum applies the class default and writes it into the workload, so `datumctl get workload cache -o yaml` shows exactly what runs for the `cache` workload. The default does the following: * Adds back `CHOWN`, `DAC_OVERRIDE`, `FOWNER`, `FSETID`, `KILL`, `NET_BIND_SERVICE`, `SETFCAP`, `SETGID`, `SETPCAP`, and `SETUID`. * Sets `allowPrivilegeEscalation` to `false`. * Sets `seccompProfile.type` to `RuntimeDefault`. The default set is enough for common images such as NGINX, PostgreSQL, and images that switch users at startup. If your container needs a different set, set `securityContext.capabilities` on the container. The following example grants only `NET_BIND_SERVICE` and `NET_ADMIN`: ```yaml theme={null} containers: - name: app image: ghcr.io/example/app:1.0 securityContext: capabilities: add: - NET_BIND_SERVICE - NET_ADMIN ``` Keep the following rules in mind: * A `capabilities` list that you set replaces the class default. The default isn't merged in, so list every capability your container needs. * Use capability names without the `CAP_` prefix. `ALL` isn't accepted in `add`. * `general-purpose` Instances can add any Linux capability, including `SYS_ADMIN`, because the capability applies only inside the Instance's own virtual machine. * `allowPrivilegeEscalation` and `seccompProfile` default independently of `capabilities`. `seccompProfile.type` accepts `RuntimeDefault` or `Unconfined`. * Datum applies the default when you create a workload. A later change to the class default doesn't affect existing workloads. ### Images that use sudo or setuid Because `allowPrivilegeEscalation` defaults to `false`, a process can't gain privileges after it starts. `sudo`, `su`, and other setuid programs fail inside the container, even though the capabilities are granted. This often affects vendor images that run as a non-root user and use `sudo` in an entrypoint script, for example to fix file ownership at startup. To fix it, do one of the following: * **Preferred:** do the privileged work when you build the image, such as running `chown` in the `Containerfile`, so the container doesn't need `sudo` at runtime. * Set `allowPrivilegeEscalation: true` in the container's `securityContext`. It only affects the Instance's own virtual machine. To catch these problems before you deploy, run the image locally with the same restrictions, and without a volume, which can hide ownership problems: ```bash theme={null} podman run --rm --security-opt no-new-privileges --cap-drop ALL \ --cap-add CHOWN,DAC_OVERRIDE,FOWNER,FSETID,KILL,NET_BIND_SERVICE,SETFCAP,SETGID,SETPCAP,SETUID \ IMAGE ``` ## Limitations In addition to the [limitations that apply to all of Compute](/docs/compute/limitations-and-roadmap), the `general-purpose` runtime class has the following limitations: * Instances take several seconds to start, mostly spent booting a kernel and pulling the image. * Host namespaces, host ports, host paths, and privileged host access aren't available. * The Instance's kernel can't load kernel modules. Features that need a module that isn't built in, such as in-kernel WireGuard, VRFs, VLAN interfaces, or GRE tunnels, aren't available. Userspace alternatives, such as `wireguard-go`, work. ## What's next * To set environment variables and use ConfigMaps and Secrets, see [Configure a workload](/docs/compute/configuration). * To choose where to run and how many Instances to run, see [Placement and scaling](/docs/compute/placement-and-scaling). * To serve the workload on a public URL, see [Publish a workload](/docs/compute/compute-and-alb). # Choose a runtime class Source: https://datum.net/docs/compute/runtimes/overview Compare the general-purpose and unikernel runtime classes, and learn how to select one for a workload. A runtime class is the execution tier that a workload's Instances run in. Compute offers two classes, and each one trades image compatibility against startup speed and overhead. You choose the class when you create a workload, and you can't change it later. > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Compare runtime classes The following table compares the two classes: | Feature | General purpose | Unikernel | | - | - | - | | Class name | `general-purpose` | `unikernel` (the default) | | Isolation | Each Instance runs in its own lightweight virtual machine with its own kernel. | Each Instance runs as its own unikernel in its own virtual machine. The Instance carries only the operating system code that its app uses. | | Images | Standard Linux container images, without changes. Includes images that need a full filesystem, dynamic linking, or credentials to pull. | Images packaged with `datumctl compute build`. A plain `docker build` image doesn't boot. | | Startup time | Several seconds, mostly spent booting a kernel and pulling the image. | A fraction of a second after the image is pulled. | | Overhead per Instance | Higher | Very low | | Root filesystem | The image filesystem. | Held in memory, so the unpacked image size counts against the Instance's 2 GiB of memory. | | Environment variables from a whole ConfigMap or Secret (`envFrom`) | Supported | Not supported. Use `env[].valueFrom` for each key instead. | | ConfigMap and Secret volumes | Supported | Supported | | Private registry credentials (`imagePullSecrets`) | Supported | Supported | | Added Linux capabilities | Supported | Not supported | | Instance size | 1 vCPU, 2 GiB | 1 vCPU, 2 GiB | ## When to use each class Choose `general-purpose` in the following cases: * You want to run an existing container image without repackaging it. * Your app needs `envFrom`, extra Linux capabilities, setuid binaries, or a large filesystem. Choose `unikernel` in the following cases: * Startup time and per-Instance overhead matter, such as for bursty or short-lived services. * Your app is a self-contained binary, such as a Go or Rust program, or a small interpreted app whose runtime and libraries fit in memory. ## Select a runtime class If you don't set a runtime class, Datum uses `unikernel`. To run a standard container image, set the class to `general-purpose` explicitly. To select a class with `datumctl`, pass the `--runtime-class` flag when you create the workload: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME \ --image=IMAGE \ --location=LOCATION \ --runtime-class=RUNTIME_CLASS ``` Replace the following: * `WORKLOAD_NAME`: a name for your workload. * `IMAGE`: the full image reference, including the registry host, such as `ghcr.io/example/app:1.0`. * `LOCATION`: the location to run in, such as `us-central-1`. * `RUNTIME_CLASS`: `general-purpose` or `unikernel`. To select a class in a manifest, set `spec.template.spec.runtime.class`: ```yaml theme={null} apiVersion: compute.datumapis.com/v1alpha kind: Workload metadata: name: web spec: template: spec: runtime: class: general-purpose resources: instanceType: datumcloud/d1-standard-2 sandbox: containers: - name: web image: docker.io/traefik/whoami:v1.11 ports: - name: http port: 80 networkInterfaces: - network: name: default placements: - name: dallas locations: - name: us-central-1 scaleSettings: minReplicas: 1 ``` When you apply a workload, Datum checks the spec against the class. If the workload uses a feature that the class doesn't offer, Datum rejects the request, and the error names the class and the feature. For example, adding a disk volume to a `general-purpose` workload returns an error that says disk-backed volumes aren't supported by the `general-purpose` runtime class. ## View the runtime class catalog The classes are read-only resources in your project. To list them, run the following command: ```bash theme={null} datumctl get runtimeclasses ``` The output includes each class's display name, isolation boundary, whether it's the default, and whether it's available. ## Change a workload's runtime class A workload keeps its runtime class for its whole life. If you try to change the class, `datumctl compute deploy` and the API reject the change. To move a workload to another class, follow these steps: 1. Deploy a new workload with a different name and the class that you want. 2. To confirm that the new workload is available, run `datumctl compute workloads` and check that its `HEALTH` is `Available`. 3. Delete the old workload with `datumctl compute destroy OLD_WORKLOAD_NAME`. Replace `OLD_WORKLOAD_NAME` with the name of the workload that you're replacing. ## Where each class runs Today, `us-central-1`, `us-east-1`, and `us-west-1` serve both classes. If you place a workload in a location that doesn't serve its class, Datum creates no Instances there and the workload reports the reason `RuntimeClassNotServed`. To fix it, choose another location or deploy a new workload with a class that the location serves. For the classes that each location serves, see [Locations](/docs/compute/placement-and-scaling#locations). ## What's next * To run a standard container image, see [Run a container image](/docs/compute/runtimes/general-compute). * To package your own app as a unikernel, see [Build and deploy a unikernel](/docs/compute/runtimes/unikernels). # Build and deploy a unikernel Source: https://datum.net/docs/compute/runtimes/unikernels Package your app as a unikernel image with datumctl compute build, check it for compatibility, and deploy it. The `unikernel` [runtime class](/docs/compute/runtimes/overview) runs each Instance as a unikernel: a single-purpose virtual machine that carries only the operating system code your app uses. Unikernel Instances start in a fraction of a second and use very little overhead. A unikernel needs a specially packaged image. You write an ordinary Dockerfile, and `datumctl compute build` turns it into an image that Compute can boot. A plain `docker build` image doesn't boot as a unikernel. > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Before you begin * Select a project, install the `compute` plugin, and get access to Compute. For more information, see [Compute commands](/docs/datumctl/compute/overview). * Run Docker with BuildKit enabled on your machine. Docker Desktop and recent Docker Engine releases enable BuildKit by default. You can also point the `BUILDKIT_HOST` environment variable at a standalone BuildKit daemon. * Have a container registry that you can push to, such as GitHub Container Registry or Docker Hub. `datumctl compute build` uses the registry credentials from your local Docker configuration, so run `docker login` for the registry first. * Have an app that listens on IPv6. Instances have IPv6 addresses only, so an app that listens only on an IPv4 address such as `0.0.0.0` is unreachable. Configure it to listen on `::` (all IPv6 addresses) instead, for example `[::]:8080`. ## How a unikernel image works `datumctl compute build` builds the final stage of your Dockerfile and packages its filesystem as the unikernel's root filesystem. At boot, Compute supplies the kernel and starts your image's entrypoint with its command and arguments. Keep the following requirements in mind: * **x86\_64 only:** The build always targets `linux/amd64`. * **Everything your program loads must be in the image:** Include the program, its dynamic loader, and every shared library it links against. For an interpreted language, include the interpreter and its libraries. * **The root filesystem is held in memory:** The unpacked image size counts against the Instance's 2 GiB of memory. Keep images small. * **No shell unless you include one:** If your entrypoint is a shell script or uses the shell form of `CMD`, the image needs that shell. Multi-stage Dockerfiles work well: build in a full toolchain image, and copy only the program and the files that it needs into a minimal final stage. ## Write a Dockerfile `datumctl compute build` uses `Dockerfile.datum` in the build directory if the file exists, and `Dockerfile` otherwise. A separate `Dockerfile.datum` lets you keep a Compute-specific build next to your regular one. The following `Dockerfile.datum` builds a Go web server and copies only the binary into an empty final stage: ```dockerfile theme={null} FROM golang:1.25 AS build WORKDIR /src COPY . . RUN CGO_ENABLED=0 go build -o /server . FROM scratch COPY --from=build /server /server ENTRYPOINT ["/server"] ``` In Go, `http.ListenAndServe(":8080", handler)` listens on all IPv6 and IPv4 addresses. ## Check the image for compatibility Before you deploy, check that the image can boot as a unikernel. To build the image locally and analyze it, run the following command in your app's directory: ```bash theme={null} datumctl compute build --analyze . ``` The analysis reports problems such as the following: * The program is built for an architecture other than x86\_64. * A shared library or the dynamic loader is missing from the final stage. * The startup command needs a shell or a script interpreter that the image doesn't include. * The entrypoint file isn't executable. * A glibc program that resolves host names or users is missing the NSS modules that it loads at runtime. When a problem has a mechanical fix, the report shows the Dockerfile change. The following example shows a missing library: ```text theme={null} error[missing-libs]: /app/server requires a runtime file that is not present in the image --> Dockerfile:8 | copy missing runtime files into the final image | - COPY --from=build /app /app | + COPY --from=build /app /app | + COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 ``` To apply these fixes to your Dockerfile and rebuild, run the following command: ```bash theme={null} datumctl compute build --fix . ``` `--fix` edits the Dockerfile in place with exact-line changes. The analysis can't detect libraries that your program loads at runtime, such as plugins loaded with `dlopen`. Copy those files into the image yourself. ## Build and deploy in one step To build the image, push it, and deploy it as a workload with a public URL, run the following command in your app's directory: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME \ --build \ --image=IMAGE \ --runtime-class=unikernel \ --location=us-central-1 \ --http-port=PORT ``` Replace the following: * `WORKLOAD_NAME`: a name for your workload, such as `hello`. * `IMAGE`: the image reference to push to and deploy, including the registry host, such as `ghcr.io/example/hello:1.0`. * `PORT`: the port that your app serves HTTP on, such as `8080`. With `--build`, the command does the following: 1. Builds the image from the current directory. To build from another directory, pass it as `--build=DIRECTORY`, where `DIRECTORY` is the path to your app. 2. Applies the same fixes as `datumctl compute build --fix`, which can rewrite your Dockerfile. 3. Pushes the image to the registry without asking for confirmation. 4. Deploys the image pinned by its digest, not by the tag. `--runtime-class=unikernel` is the default, so you can omit it. Naming the class makes the choice clear to anyone who reads the command later. The deployed workload pulls the image anonymously. If the image is in a private registry, give the workload registry credentials. For more information, see [Use images from a private registry](/docs/compute/configuration#use-images-from-a-private-registry). ## Build and deploy in separate steps Separate steps are useful in CI, or when you need build arguments or a specific Dockerfile stage. To build and deploy in separate steps, follow these steps: 1. To build the image and push it to your registry, run the following command: ```bash theme={null} datumctl compute build --push --output IMAGE . ``` To pass build arguments, add `--build-arg NAME=VALUE`, where `NAME` and `VALUE` are the argument and its value. To build a stage other than the last one, add `--target STAGE`, where `STAGE` is the stage name. 2. To deploy the image, run the following command: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME \ --image=IMAGE \ --runtime-class=unikernel \ --location=us-central-1 \ --http-port=PORT ``` Replace the following: * `IMAGE`: the image reference to push to and deploy, including the registry host, such as `ghcr.io/example/hello:1.0`. * `WORKLOAD_NAME`: a name for your workload. * `PORT`: the port that your app serves HTTP on. To check an image that's already in a registry, run `datumctl compute build inspect IMAGE`. For all build options, see [Building images](/docs/datumctl/compute/building-images). ## Package apps in other languages Any x86\_64 Linux program can run as a unikernel if the image contains everything that the program loads. The approach depends on how the program is built: * **Compiled languages, such as Go and Rust:** Copy the compiled binary into a minimal final stage. A statically linked binary needs no other files. * **Dynamically linked programs:** Copy the dynamic loader and each shared library that the program links against. `datumctl compute build --analyze` lists missing libraries, and `--fix` adds `COPY` lines for them. * **Interpreted languages, such as Node.js and Python:** Copy the interpreter, its shared libraries, and your app into the final stage. Copy only the files that the interpreter needs, not whole system library directories, because the image must fit in memory. ## Limitations In addition to the [limitations that apply to all of Compute](/docs/compute/limitations-and-roadmap), the `unikernel` runtime class has the following limitations: * Images must be packaged with `datumctl compute build`, and must target x86\_64. * The root filesystem is held in memory. The unpacked image counts against the Instance's 2 GiB of memory. * Environment variables from a whole ConfigMap or Secret (`envFrom`) aren't supported. Use `env[].valueFrom` for each key. * Linux capabilities can't be added, and container security context settings have no effect. * Crash details are limited. An Instance whose program exits at startup can stay `Starting` instead of reporting a crash. If an Instance stays `Starting` for more than a few minutes, run `datumctl compute build --analyze` on the image. ## What's next * To set environment variables and use ConfigMaps and Secrets, see [Configure a workload](/docs/compute/configuration). * To serve the workload on a public URL or your own domain, see [Publish a workload](/docs/compute/compute-and-alb). * To find out why an Instance doesn't start, see [Compute troubleshooting](/docs/compute/troubleshooting). # Compute troubleshooting Source: https://datum.net/docs/compute/troubleshooting Find out why a Compute workload or Instance isn't running, and fix common errors. This page lists common Compute errors and states, what they mean, and how to fix them. To check on workloads and Instances first, see [Operations](/docs/compute/operations). > [!NOTE] > > Compute is in preview, and the `v1alpha` API can change. ## Common problems The following table lists common errors and states, what they mean, and how to fix them. Errors that Datum returns when you apply a workload appear in the command output. Reasons appear in `datumctl get workloads`, `datumctl get instances`, and `datumctl compute instances describe`. | Error or reason | Meaning | What to do | | - | - | - | | `must include a registry` | The image reference has no registry host. | Use a full reference, such as `docker.io/library/redis:7`. | | `... are not supported by the "CLASS" runtime class` | The workload uses a feature that its runtime class doesn't offer, such as `envFrom` or added capabilities on `unikernel`. | Remove the feature, or deploy a new workload with a class that offers it. See [Choose a runtime class](/docs/compute/runtimes/overview). | | `resources: Forbidden: not implemented` | The workload sets CPU or memory for a container. | Remove `resources`, and set `runtime.resources.instanceType` to `datumcloud/d1-standard-2`. | | `permission to use the network was denied` | You don't have permission to attach workloads to the network. | Ask a project administrator for the **Network Admin** role. | | `permission to get ... was denied` | You can't read a ConfigMap or Secret that the workload references. | Ask for read access to the object, or reference another object. | | `volume must be attached at least 1 time` | The workload declares a volume that no container mounts. | Add a `volumeAttachments` entry with a `mountPath`, or remove the volume. | | `NetworkNotFound` | The network named in `networkInterfaces` doesn't exist. | Create the network. See [gVPC configuration](/docs/galactic-vpc/configuration). | | `RuntimeClassNotServed` | A location in the [placement](/docs/compute/placement-and-scaling) doesn't serve the workload's runtime class. | Choose another location, or deploy a new workload with a class that the location serves. | | `NoMatchingLocations` | A placement's selector matches no location. | Fix the selector, or list locations by name. | | `QuotaExceeded` or `QuotaNotGranted` | Your project's compute quota is used up. | Run fewer Instances, delete unused workloads, or ask for more quota. See [Limits and quotas](/docs/compute/limits-and-quotas). | | `QuotaNoBudget`, or another reason that starts with `Quota` | Compute quota isn't set up correctly for your project. | Contact [support@datum.net](mailto:support@datum.net). Datum fixes this. | | `ReferencedDataNotReady` with `SourceNotFound` | A referenced ConfigMap or Secret doesn't exist. | Create the object, or fix its name in the workload. | | `ReferencedDataNotReady` with `SourceTooLarge` | A referenced ConfigMap or Secret is larger than 256 KiB, or all of them together exceed 1 MiB. | Make the objects smaller. | | `ReferencedDataNotReady` with `SourceUnauthorized` | Datum can't read a referenced ConfigMap or Secret. | Check that the object is in the same project, and contact support if the problem continues. | | `ImageUnavailable` | Datum couldn't pull the image. | Check the image reference and tag. For a private registry, check the `imagePullSecrets` Secret. | | A standard container image, such as nginx, won't start | You didn't set a runtime class, so the workload uses the default `unikernel` class, which doesn't boot standard container images. | Deploy a new workload with `--runtime-class=general-purpose`. You can't change a workload's runtime class. See [Run a container image](/docs/compute/runtimes/general-compute). | | `InstanceCrashing` | The program starts and exits repeatedly. | Check the command, arguments, environment variables, and mounted files. For a `unikernel` workload, rebuild with `datumctl compute build --analyze`. | | `ConfigurationError` | Datum refused the configuration before the program ran. | Check the environment variables and volume references. | | The Instance stays `Starting` for more than a few minutes | The first image pull can take a while. A program that exits at startup can also stay in this state. | Check the image and configuration. For a `unikernel` workload, run `datumctl compute build --analyze`. If the problem continues, contact support. | | The Instance is `Available`, but the ALB's hostname returns an error | The app isn't reachable on the published port, or the container isn't running even though the Instance is `Available`. | Make sure that the app listens on IPv6 (`::`), serves plain HTTP, and uses the port that you passed to `--http-port`. Check the logs with `datumctl compute logs WORKLOAD_NAME`. If the container exits at startup, check its `securityContext`. See [Images that use sudo or setuid](/docs/compute/runtimes/general-compute#images-that-use-sudo-or-setuid). | | The ALB returns `503` for a minute or two after the Instance becomes `Available` | The Instance is running, but the app inside it is still starting. | Wait, or give the app a faster startup. Apps such as JVMs can take a minute or more to listen. | | `sudo` or another setuid program fails with a permission error | `allowPrivilegeEscalation` defaults to `false`. | Do the privileged work at build time, or set `allowPrivilegeEscalation: true`. See [Images that use sudo or setuid](/docs/compute/runtimes/general-compute#images-that-use-sudo-or-setuid). | | A new image version doesn't show up after a rollout | The image was pushed again under the same tag. | Push each version with a new tag, or reference the image by digest. | | `curl` can't resolve the ALB's hostname | The new hostname hasn't propagated. | Wait a few minutes and try again. | ## See why an Instance won't start When an Instance can't start, Datum records why. In the [Cloud Portal](https://cloud.datum.net), open the workload and select the **Activity** tab. Each Instance's start, stop, and failure appears with a plain-language reason. In this example, two Instances couldn't start because their location refused the Linux capabilities the containers requested, and the platform keeps retrying. The fix is in the workload's configuration, not in the Instance. ![The Activity tab of a workload, listing Instances that started running, Instances that are shutting down to be replaced, and two Instances that could not start because the location refused the Linux capabilities their containers request](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/compute-workload-activity-config-error.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=c211ee6de96ad27ab3617407207de6e9) A workload's Activity tab, showing Instances that couldn't start because of their configuration To see the same reason from a terminal, run `datumctl compute instances describe INSTANCE_NAME`, or list recent events with `datumctl activity events`. To see a history of changes to your workloads, use `datumctl activity`. For more information, see [Querying activity](/docs/datumctl/activity/overview). If you can't resolve a problem, contact [support@datum.net](mailto:support@datum.net) with the workload name, the project ID, and the output of `datumctl compute instances describe INSTANCE_NAME`. ## Debug inside an Instance To open an interactive shell or run a command in a running `general-purpose` Instance, see [Instance shell](/docs/compute/instance-shell). A shell needs the Instance to be running and the image to include `sh`. If the container exits too quickly to open a shell (`InstanceNotRunning`), read its logs instead. To read Instance output and CPU and memory usage, see [Observability](/docs/compute/observability). ## What's next * To check quota and review limits, see [Limits and quotas](/docs/compute/limits-and-quotas). * To review what Compute doesn't support yet, see [Limitations and roadmap](/docs/compute/limitations-and-roadmap). # Connectors limitations and roadmap Source: https://datum.net/docs/connectors/limitations-and-roadmap Review what Connectors don't support yet and what Datum is building next. [Connectors](/docs/connectors/overview) are still being built. Today, the only Connector is the Iroh-powered [Tunnel](/docs/connectors/tunnels), and its first use case is the [Desktop Apps](/docs/desktop-apps). ## Planned capabilities * **"Anywhere to anywhere" tunnels**, beyond exposing a local environment to the internet. * **Developer connections**, such as Tailscale tailnets and WireGuard VPNs. * **Telco connections**, such as AWS Direct Connect, Equinix Fabric, and Megaport onramps. For what Datum is building next across the platform, see the [Platform roadmap](/docs/platform/roadmap). > [!NOTE] > > Plans can change. This list isn't a commitment to deliver any feature or to meet any timeline. # Connectors limits and quotas Source: https://datum.net/docs/connectors/limits-and-quotas The limits and quotas that apply to Connectors and Tunnels. Documentation is coming soon. > [!NOTE] > > **Coming soon.** Datum is still building [Connectors](/docs/connectors/overview), and documentation for this topic is on the way. ## Follow the roadmap > [card] [Connectors limitations and roadmap](/docs/connectors/limitations-and-roadmap) > > See what Connectors doesn't support yet and what Datum is building next. # Connectors observability Source: https://datum.net/docs/connectors/observability See the traffic that reaches your Connectors through a load balancer, and the metrics and logs Datum publishes for it. This page covers how to see the traffic your [Connectors](/docs/connectors/overview) carry. Datum is still building Connectors, and more detail is on the way. ## Traffic through a load balancer Today, a Connector receives traffic through an [Application Load Balancer](/docs/alb/overview) whose route uses a `connector` backend. Every request it carries is measured and logged by that load balancer, so you observe a Connector through the load balancers that use it. In the [Cloud Portal](https://cloud.datum.net), each Connector shows a request-rate chart that adds up the traffic from every load balancer that routes to it. ### Metrics Use the [ALB traffic and latency metrics](/docs/alb/observability#traffic-and-latency), selected by the name of each load balancer that routes to your Connector. For example, to see requests per second through one load balancer, replace `PROJECT_ID` and `ALB_NAME`: ```promql theme={null} sum(rate(envoy_vhost_vcluster_upstream_rq{resourcemanager_datumapis_com_project_name="PROJECT_ID", gateway_name="ALB_NAME"}[5m])) ``` To add up several load balancers, select them with a regular expression, such as `gateway_name=~"alb-one|alb-two"`. To query these metrics in your own tools, [export them](/docs/operations/metrics-export). ### Access logs To see the requests that reached your Connector, read the access logs of the load balancer that routes to it: ```bash theme={null} datumctl alb logs ALB_NAME --since 1h ``` A `502` or `503` with nothing reaching your machine can mean the tunnel isn't connected. Check that the Connector is running on your machine. See [ALB access logs](/docs/alb/observability#access-logs). ## Follow the roadmap > [card] [Connectors limitations and roadmap](/docs/connectors/limitations-and-roadmap) > > See what Connectors doesn't support yet and what Datum is building next. # Connectors operations Source: https://datum.net/docs/connectors/operations How to create, inspect, and manage Connectors and Tunnels. Documentation is coming soon. > [!NOTE] > > **Coming soon.** Datum is still building [Connectors](/docs/connectors/overview), and documentation for this topic is on the way. ## In the meantime * To create and remove tunnels from your desktop, see [Desktop Apps](/docs/desktop-apps). ## Follow the roadmap > [card] [Connectors limitations and roadmap](/docs/connectors/limitations-and-roadmap) > > See what Connectors doesn't support yet and what Datum is building next. # Connectors Source: https://datum.net/docs/connectors/overview Connectors orchestrate connections between your infrastructure and Datum's edge cloud, starting with QUIC tunnels powered by Iroh. Connectors are how you bring connections from outside Datum into Datum's edge cloud. Datum is still building Connectors, and documentation will grow with the product. ## Available today * **[Tunnels](/docs/connectors/tunnels):** flexible QUIC tunnels powered by the [Iroh](/docs/platform/iroh-relays) protocol. The first use case is the [Desktop Apps](/docs/desktop-apps), which securely expose a local environment to the internet through an [Application Load Balancer](/docs/alb/overview). ## Planned * **Developer connections:** such as Tailscale tailnets and WireGuard VPNs. * **Telco connections:** such as AWS Direct Connect, Equinix Fabric, and Megaport onramps. * **"Anywhere to anywhere" tunnels.** See [Limitations and roadmap](/docs/connectors/limitations-and-roadmap) for details. # Connectors troubleshooting Source: https://datum.net/docs/connectors/troubleshooting How to find and fix problems with Connectors and Tunnels. Documentation is coming soon. > [!NOTE] > > **Coming soon.** Datum is still building [Connectors](/docs/connectors/overview), and documentation for this topic is on the way. ## In the meantime * For help, ask in [Discord](https://link.datum.net/discord) or contact [support@datum.net](mailto:support@datum.net). ## Follow the roadmap > [card] [Connectors limitations and roadmap](/docs/connectors/limitations-and-roadmap) > > See what Connectors doesn't support yet and what Datum is building next. # Tunnels Source: https://datum.net/docs/connectors/tunnels Datum's first Connector is a flexible tunnel powered by the Iroh protocol and built on QUIC. Iroh is a modular, Rust-based networking stack that establishes fast, direct, and secure peer-to-peer (P2P) connections between devices. Instead of relying on IP addresses, devices "dial" each other directly using public keys, bypassing NATs and firewalls. Our first use case for Iroh Tunnels is with our [desktop apps](/docs/desktop-apps), which help you securely expose localhost to the internet. Future iterations will include "anywhere to anywhere" Tunnels. ## How a tunnel connects * Each tunnel endpoint keeps one outbound connection to its nearest Datum relay, its *home relay*, over HTTPS on port 443. The endpoint needs no public address and no open inbound port. * Traffic starts through the relay, then moves to a direct UDP path when both sides can holepunch one. The relay stays available as a fallback. * The connection is QUIC, encrypted end to end with TLS 1.3, so a relay forwards only packets it can't read. For details, see [Iroh relays](/docs/platform/iroh-relays). # Audit logs & events Source: https://datum.net/docs/datumctl/activity/audit-and-events Investigate who did what to which resource with datumctl activity audit, and surface resource lifecycle events and warnings with datumctl activity events. When you need to answer a pointed question — "who deleted that DNS zone last night?" or "what went wrong at 14:20?" — `datumctl activity` gives you two purpose-built investigation tools: * **`datumctl activity audit`** records the actions people and machines take against your resources: who did what, when, and to which resource. This is your compliance and forensics trail. * **`datumctl activity events`** surfaces the lifecycle events a resource emits as it moves through its states — the warnings and status changes you reach for during an incident. Both commands share one query model — time ranges, server-side filters, pagination, and machine-readable output — so once you learn one, the other feels familiar. > [!NOTE] > > `datumctl activity` is scoped like every other command: it acts on your active organization or project, and honors the `--project` and `--organization` flags. See [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how scope is resolved. ## The shared query model Every `activity` subcommand accepts the same flags for narrowing a time window, filtering server-side, paging through large result sets, and choosing an output format. ### Time range Both commands default to the **last 24 hours**. Widen or narrow the window with `--start-time` and `--end-time`. Two formats are accepted: | Format | Example | Notes | | - | - | - | | Relative | `now-7d`, `now-24h`, `now-30m` | Units: `s`, `m`, `h`, `d`, `w`. `--end-time` defaults to `now`. | | Absolute | `2026-01-01T00:00:00Z` | RFC3339 with a timezone. | ```bash theme={null} # Everything in the last week datumctl activity audit --start-time now-7d # A precise window around an incident datumctl activity events \ --start-time 2026-06-30T14:00:00Z \ --end-time 2026-06-30T14:30:00Z ``` ### Pagination Results are paged. By default you get one page of up to 25 rows. | Flag | Default | Description | | - | - | - | | `--limit` | `25` | Rows per page, between 1 and 1000. | | `--all-pages` | off | Fetch every page automatically — ideal for an export. | | `--continue-after` | — | Resume from a cursor returned by a previous query. | > [!NOTE] > > `--all-pages` and `--continue-after` are mutually exclusive. Use `--all-pages` when you want the whole result set in one go; use `--continue-after` when you are stepping through pages yourself (for example, in an automation loop that checkpoints its position). ### Output format The default output is a human-readable table. Pass `-o` to get machine-readable output, and `--no-headers` to drop the table header row. | `-o` value | Description | | - | - | | *(none)* | Formatted table (the default). | | `json` | A single JSON document — best for piping into `jq`. | | `yaml` | A YAML document. | | `jsonpath` | Extract specific fields, e.g. `-o jsonpath='{.items[*].objectRef.name}'`. | > [!TIP] > > For pipelines, exports, and error handling patterns that apply across the CLI, see [Output formats & scripting](/docs/datumctl/output-and-scripting). *** ## Audit — who did what, when, to which resource > [!NOTE] > > **The question:** *"Prove who did what, when, and to which resource — and hand me an exportable record."* `datumctl activity audit` answers that question. Each entry records an action (a verb) taken by a user against a resource, together with a timestamp and the response status. It is the tool for security reviews, compliance evidence, and after-the-fact forensics. The default table shows the timestamp, verb, user, affected resource, and status code: ```bash theme={null} datumctl activity audit ``` ### Narrowing the trail Four shorthand flags let you zero in without writing a filter expression. They combine with **AND** logic — every condition must match. | Flag | Filters by | Example | | - | - | - | | `--user` | The identity that performed the action | `--user alice@example.com` | | `--verb` | The action taken | `--verb delete` | | `--resource` | The resource type acted on | `--resource dnszones` | | `--namespace` | The target namespace | `--namespace default` | For anything the shorthand flags do not cover, `--filter` takes a server-side [Common Expression Language (CEL)](https://cel.dev/) expression that is applied on top of them — for example `--filter "responseStatus.code >= 400"` to find failed requests. > [!NOTE] > > Not sure which values exist in your data? The audit command can suggest them. `datumctl activity audit --suggest user.username` lists the users with recorded activity in the window, which is a fast way to discover exactly what to pass to `--user`. ### Scenario: who deleted resources last week? A security auditor wants every deletion across the project over the past seven days, with the responsible identity: ```bash theme={null} datumctl activity audit --start-time now-7d --verb delete ``` To focus on one person's activity against sensitive resources: ```bash theme={null} datumctl activity audit \ --start-time now-30d \ --user alice@example.com \ --resource secrets ``` ### Scenario: export an immutable record Auditors and SIEM pipelines need the full record, not a single page. Combine `--all-pages` with `-o json` to capture everything in the window as one JSON document: ```bash theme={null} # Export a full 30-day audit record for archival or ingestion datumctl activity audit \ --start-time now-30d \ --all-pages \ -o json > audit-record.json ``` That file is a stable, machine-readable artifact you can attach to a compliance report or feed into a log store. > [!TIP] > > Pull just the fields you need with a JSONPath expression instead of post-processing: > > ```bash theme={null} > datumctl activity audit --start-time now-7d --verb delete \ > -o jsonpath='{.items[*].objectRef.name}' > ``` ### Scenario: stream audit records into a logging pipeline Automation and SIEM integrators typically want to ingest audit records incrementally and checkpoint their position rather than re-pull the whole window each run. Page through the results yourself with `--continue-after`, using the cursor the CLI reports after each page: ```bash theme={null} # First page — capture the continue cursor from the output datumctl activity audit --start-time now-24h --limit 1000 -o json # Next run resumes from where you left off datumctl activity audit --start-time now-24h --limit 1000 \ --continue-after "" -o json ``` > [!TIP] > > For a robust wrapper — non-interactive auth, pinning `--organization`/`--project`, structured errors, and branching on exit codes — see [Output formats & scripting](/docs/datumctl/output-and-scripting). *** ## Events — what changed, and were there warnings? > [!NOTE] > > **The question:** *"Something broke at 14:20 — what was happening to my resources just before, and were there any warnings?"* `datumctl activity events` surfaces the lifecycle events resources emit as they change state — the same events you would see reported against a resource, retained far longer than the live event window so you can investigate after the fact. Each event carries a type (`Normal` or `Warning`), a reason, the object it regards, and a message. The default table shows the last-seen time, type, reason, regarding object, and message: ```bash theme={null} datumctl activity events ``` ### The on-call reflex: filter to warnings When you are paged, the fastest signal is the set of warning events. `--type Warning` cuts straight to problems: ```bash theme={null} datumctl activity events --type Warning --start-time now-1h ``` ### Zeroing in during an incident Combine these flags to narrow from "everything" down to the resource you suspect: | Flag | Filters by | Example | | - | - | - | | `--type` | Event severity: `Normal` or `Warning` | `--type Warning` | | `--reason` | The machine reason for the event | `--reason FailedMount` | | `--regarding-kind` | The kind of resource the event is about | `--regarding-kind Workload` | | `--regarding-name` | A specific resource by name | `--regarding-name checkout-api` | | `--namespace` | The namespace of the regarding resource | `--namespace default` | | `--field-selector` | A raw field-selector expression | `--field-selector "type=Warning,reason=Scheduled"` | The shorthand flags are combined into a single field selector for you; `--field-selector` lets you express conditions the shorthands do not cover. Comma-separated conditions must all match. > [!NOTE] > > Unsure which reasons appear in your data? `datumctl activity events --suggest reason` lists the event reasons seen in the window, so you know exactly what to pass to `--reason`. ### Scenario: what happened to one resource? An on-call responder suspects a specific resource is the culprit. Pull every event about it in the last hour: ```bash theme={null} datumctl activity events \ --regarding-kind Workload \ --regarding-name checkout-api \ --start-time now-1h ``` Then narrow to only the warnings for that resource over a longer window to see whether the trouble started earlier: ```bash theme={null} datumctl activity events \ --regarding-kind Workload \ --regarding-name checkout-api \ --type Warning \ --start-time now-24h ``` ### Scenario: hunt a recurring failure by reason If you already know the failure mode, filter by its reason across the whole project to see how widespread it is: ```bash theme={null} datumctl activity events --reason FailedMount --start-time now-7d ``` > [!WARNING] > > Events answer "what happened to a resource," while audit answers "who acted on a resource." During an incident, reach for `events --type Warning` to find the symptom, then switch to `audit` to find the change — a `--verb update` or `--verb delete` around the same timestamp — that likely caused it. *** ## Choosing between audit and events > [card] Reach for audit when… > > You need to know **who** took an action, when, and against which resource — deletions, updates, failed requests, or a full compliance export. > [card] Reach for events when… > > You need to know **what happened** to a resource — warnings, lifecycle changes, and the symptoms of an incident as the platform reports them. ## Related * [Querying activity](/docs/datumctl/activity/overview) — the hub for the whole `activity` command family, including the persona routing table and the shared query model. * [Change history & feed](/docs/datumctl/activity/change-history-and-feed) — the `feed` and `history` subcommands, for a merged, human-readable activity stream and per-resource spec diffs. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — `jq`/JSONPath patterns, structured errors, and exit-code handling for wrapping these commands in pipelines and SIEM ingestion. * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — how the active organization and project are resolved, and how to pin them explicitly with `--organization` and `--project`. * [Discovering resources & schemas](/docs/datumctl/discovering-resources) — find the resource-type and kind names to pass to `--resource` and `--regarding-kind`. # Change history & feed Source: https://datum.net/docs/datumctl/activity/change-history-and-feed Trace what changed on a single resource with datumctl activity history, and watch a merged, human-readable timeline across resource types with datumctl activity feed. Two `datumctl activity` subcommands answer the "what just happened?" questions that come up when you own a resource or operate a whole project: * **`datumctl activity history`** — zoom in on one resource. What changed, by whom, and what did the spec look like before? * **`datumctl activity feed`** — zoom out to a merged, human-readable timeline across every resource type, so you can spot what moved across the project and separate human changes from automated ones. Both honor the active context and the `--project` / `--organization` flags like every other command — see [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how scope is resolved. > [!NOTE] > > `history` and `feed` are two of the four `datumctl activity` subcommands. When you need the raw record of *who did what* for compliance, or lifecycle events with warnings, reach for [audit and resource events](/docs/datumctl/activity/audit-and-events) instead. ## History: what changed on this resource? > *"Something on my project looks different. What changed, who changed it, and what did the spec look like before?"* If you own a resource — an app developer or resource owner watching your own project, or an on-call responder pinning down the exact change behind an incident — `datumctl activity history` gives you a chronological log of modifications to a single resource's spec. You pass the resource type and name positionally: ```bash theme={null} # The change history for one project, oldest to newest datumctl activity history projects your-project-id ``` ### See what changed with --diff The table tells you *when* a resource changed and *who* changed it. To see *what* changed, add `--diff` — it renders a unified diff of the spec between each consecutive version, so you read the change the way you'd read a code review. ```bash theme={null} # Show the spec diff between each version of the project datumctl activity history projects your-project-id --diff ``` > [!NOTE] > > `--diff` compares consecutive versions and strips noisy bookkeeping fields so you see the meaningful spec changes, not churn in server-managed metadata. By default `history` looks back **30 days** (`now-30d`) and returns up to 100 versions per page. Widen or narrow the window with `--start-time` / `--end-time`, which accept both relative expressions and absolute RFC3339 timestamps: ```bash theme={null} # Only changes in the last 24 hours — narrowing to an incident window datumctl activity history projects your-project-id --diff --start-time now-24h # An explicit calendar range datumctl activity history projects your-project-id \ --start-time 2026-06-01T00:00:00Z --end-time 2026-06-30T23:59:59Z ``` You can also render history in a machine format for tooling with `-o json` or `-o yaml`. > [!TIP] > > **Incident investigation.** When a resource starts misbehaving, `datumctl activity history --diff --start-time now-1h` shows you the exact spec change that landed just before the symptoms — often the fastest path from "it broke" to "here's the line that changed." ## Feed: what's happening across the project? > *"What is going on across this project right now, and which changes came from a person versus an automated system?"* If you run the platform — a platform administrator or an SRE on call — `datumctl activity feed` merges audit events, resource events, and change history into one human-readable timeline across every resource type. Each row shows a timestamp, the actor, the change source, and a plain-language summary. ```bash theme={null} # The combined activity feed for the active project datumctl activity feed ``` By default the feed covers the last **24 hours** (`now-24h`). Adjust the window the same way as history: ```bash theme={null} # Everything in the last hour — a fast timeline during an incident datumctl activity feed --start-time now-1h ``` ### Separate human changes from system changes The single most useful triage filter is `--change-source`. It accepts exactly two values, `human` and `system`, letting you answer "did a person do this, or did an automated controller?" at a glance. ```bash theme={null} # Only changes a person initiated datumctl activity feed --change-source human # Only changes made by automated systems / controllers datumctl activity feed --change-source system ``` ### Narrow the feed Layer these filters to focus the timeline: | Flag | Narrows the feed to | | - | - | | `--actor` | A specific actor (for example a user email or service account) | | `--kind` | A single resource kind | | `--api-group` | A single API group | | `--resource-uid` | The history of one specific resource, by UID | | `--search` | A full-text match against the human-readable summaries | ```bash theme={null} # What has one person been doing? datumctl activity feed --actor alice@example.com # Everything related to a specific resource kind datumctl activity feed --kind Workload # Full-text search across summaries datumctl activity feed --search "created HTTPProxy" ``` For advanced cases, `--filter` takes a server-side [Common Expression Language (CEL)](https://cel.dev/) expression (for example `spec.resource.kind in ['Workload', 'HTTPProxy']`). ### Watch live during an incident Add `--watch` (`-w`) to stream new activity as it happens instead of printing a snapshot and exiting — invaluable when you're actively working an incident and want to see changes land in real time. ```bash theme={null} # Stream human-initiated changes live datumctl activity feed --change-source human --watch ``` > [!WARNING] > > In `--watch` mode, use the field-selector filters (`--actor`, `--kind`, `--change-source`, `--namespace`) to narrow the live stream. The CEL `--filter` expression is **not** supported while watching — combining `--filter` with `--watch` is rejected. **1. Establish the timeline** Start with `datumctl activity feed --start-time now-1h` to see everything that moved around the time of the incident. **2. Split human vs system** Add `--change-source human` or `--change-source system` to decide whether a person or a controller triggered the change. **3. Pin the exact change** Once you've identified the affected resource, switch to `datumctl activity history --diff` to read the precise spec change. **4. Watch for the next move** Leave `datumctl activity feed --watch` running to catch follow-on changes live as you remediate. ## Output and pagination Both subcommands share the standard activity query model. The default is a table; pass `-o json`, `-o yaml`, or a `jsonpath` expression for machine-readable output, and add `--no-headers` to drop the table header. Use `--limit` (default 25 on `feed`, 100 on `history`; max 1000), `--all-pages`, and `--continue-after` to page through large result sets. ```bash theme={null} # Extract just the summaries from the feed as newline-friendly output datumctl activity feed --start-time now-7d \ -o jsonpath='{.items[*].spec.summary}' ``` See [Output formats & scripting](/docs/datumctl/output-and-scripting) for the full set of formats, structured errors, and patterns for wrapping `datumctl` in automation. ## Related * [Querying activity](/docs/datumctl/activity/overview) — the hub for the whole `activity` command family, including the persona routing table and the shared query model. * [Audit logs & events](/docs/datumctl/activity/audit-and-events) — the `audit` and `events` subcommands, for the immutable record of who did what and lifecycle events with warnings. * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — how the active organization and project are resolved before a history or feed query runs. * [Discovering resources & schemas](/docs/datumctl/discovering-resources) — find the `RESOURCE_TYPE` name that `activity history` expects as its first positional argument. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — machine-readable output, JSONPath, and pagination for piping activity into tooling. # Querying activity Source: https://datum.net/docs/datumctl/activity/overview Investigate what changed, who did it, and when — using the datumctl activity command family for audit records, resource events, a merged feed, and per-resource change history. `datumctl activity` answers questions about *what has happened* in your organization or project — as opposed to `get` and `describe`, which show you the current state of a resource. Reach for it when you need to reconstruct a timeline, prove who took an action, understand why something changed, or export an immutable record for compliance. The command is organized into four subcommands. Each one queries the same underlying activity service but shapes the data for a different question: > [card] audit > > **Use this when** you need the authoritative record of *who did what, when, to which resource*. Actions and verbs (create, update, delete) with the acting user attached. > [card] events > > **Use this when** you want lifecycle signals a resource emitted — state changes, warnings, and notable occurrences — with extended retention. > [card] feed > > **Use this when** you want one merged, human-readable stream across audit, resource events, and change history — optionally streaming live. > [card] history > > **Use this when** you want the change log for a single resource, including spec diffs between versions. > [!NOTE] > > Activity is scoped like the rest of the CLI: it honors your active context and the `--organization` / `--project` flags. See [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how scope is resolved. ## Which command answers your question? Start from the question you are trying to answer, then use the command in the second column. | The question you're asking | Reach for | Key flags | | - | - | - | | *Prove who did what, when, to which resource — and export an immutable record.* (security / compliance auditor) | `activity audit` | `--user`, `--verb`, `--resource`, `-n/--namespace`, a time range, `--all-pages`, `-o json` | | *Something broke at 14:20 — what changed just before, and were there warnings?* (SRE / on-call responder) | `activity feed`, then `activity events`, then `activity history` | `feed --start-time now-1h`, `events --type Warning`, `history --diff`, `feed --watch` for live | | *What is happening across the project, and which changes are human vs automated?* (platform administrator) | `activity feed` | `--change-source human` / `--change-source system`, org vs. project scope | | *What changed on my resource, by whom, and what did it look like before?* (application developer / resource owner) | `activity history`, `activity feed` | `history --diff`, `feed --actor ` | | *Stream or export audit records into our logging pipeline.* (automation / SIEM integrator) | `activity audit` | `-o json`, `--all-pages` (or `--continue-after` for cursor paging) | ## The shared query model Every subcommand shares the same querying primitives, so once you learn them for `audit` they carry over to `events`, `feed`, and `history`. ### Time range Bound the query with `--start-time` and `--end-time`. Both accept relative expressions (`now`, `now-1h`, `now-24h`, `now-7d`, `now-30d`) and absolute RFC 3339 timestamps (`2026-01-01T00:00:00Z`). ```bash theme={null} # Everything in the last hour datumctl activity feed --start-time now-1h # A fixed window datumctl activity audit --start-time 2026-01-14T00:00:00Z --end-time 2026-01-15T00:00:00Z ``` > [!NOTE] > > When you omit the time range, `audit`, `events`, and `feed` default to the last 24 hours; `history` defaults to the last 30 days. These are query windows, not retention. How long audit logs and events are kept isn't documented yet. ### Filtering `audit` and `feed` accept a `--filter` [Common Expression Language (CEL)](https://cel.dev/) expression for precise server-side filtering. `events`, `feed`, and `history` also expose typed convenience flags (for example `--type`, `--reason`, `--actor`) so you rarely need to hand-write CEL for common cases. ```bash theme={null} # CEL: one user, one API group datumctl activity audit --project your-project-id \ --filter='user.username == "swells@datum.net" && objectRef.apiGroup == "networking.datumapis.com"' ``` ### Pagination Results are paged. `--limit` sets the page size (1–1000; defaults to 25 for `audit`, `events`, and `feed`, and 100 for `history`). When more results exist, the CLI tells you how to continue: ```bash theme={null} datumctl activity audit --limit 3 # ... # More results available. Use --continue-after '{continue-token}' to get the next page. # Or use --all-pages to fetch all results automatically. ``` * `--all-pages` fetches every page automatically — ideal for a complete export. * `--continue-after ''` fetches the next page from a cursor, giving an integrator explicit control over paging. > [!NOTE] > > `--all-pages` and `--continue-after` are mutually exclusive — pass one or the other. ### Output The default is a compact table. Pass `-o` to get machine-readable output for [`jq`](https://jqlang.org/), JSONPath, or an ingestion pipeline; add `--no-headers` to drop the table header row. ```bash theme={null} # Full records as JSON, ready for jq or a SIEM datumctl activity audit --start-time now-30d --all-pages -o json > audit.json # Extract a single field with JSONPath datumctl activity feed -o jsonpath='{.items[*].verb}' # YAML for reading a whole record datumctl activity events --type Warning -o yaml ``` > [!TIP] > > Building automation around activity? See [Output formats & scripting](/docs/datumctl/output-and-scripting) for the JSON/YAML/JSONPath patterns, exit-code semantics, and structured errors that apply to these commands. ## The four subcommands in depth > [tab] audit > > Records **who did what, when, to which resource**. Every entry ties an action (verb) on a resource to the acting user, making it the authoritative source for compliance. > > Adds `--user`, `--verb` (create, update, delete, patch, get, list, watch), `--resource` (resource type), and `-n/--namespace` on top of the shared model, plus `--filter` for CEL. > > ```bash theme={null} > # Deletes in the last 7 days > datumctl activity audit --verb delete --start-time now-7d > > # One resource type, exported for the record > datumctl activity audit --resource projects --start-time now-30d --all-pages -o json > audit.json > ``` > [tab] events > > Surfaces **resource lifecycle events** — state changes and noteworthy occurrences a resource emitted — with extended retention. > > Adds `--type` (`Normal` or `Warning`), `--reason`, `--regarding-kind`, `--regarding-name`, `--field-selector`, and `-n/--namespace`. > > ```bash theme={null} > # Only warnings in the last day > datumctl activity events --type Warning --start-time now-24h > > # Events about one resource > datumctl activity events --regarding-kind Project --regarding-name your-project-id > ``` > [tab] feed > > A **merged, human-readable stream** that combines audit records, resource events, and change history into a single timeline. The fastest way to see "everything that just happened." > > Adds `--change-source` (`human` or `system`), `--actor`, `--kind`, `--api-group`, `--resource-uid`, `--search` (full-text over summaries), `-n/--namespace`, `--filter` (CEL), and `-w/--watch` to stream new activity live. > > ```bash theme={null} > # A timeline of the last hour > datumctl activity feed --start-time now-1h > > # Only human-initiated changes > datumctl activity feed --change-source human > > # Follow live > datumctl activity feed --watch > ``` > [tab] history > > The **change log for a single resource**, chronologically. Add `--diff` to see the spec changes between consecutive versions. > > Takes positional `RESOURCE_TYPE` and `NAME`; use `-n/--namespace` for namespaced resources. > > ```bash theme={null} > # What changed on this project, and how > datumctl activity history projects your-project-id --diff > ``` ## How activity relates to the rest of datumctl * **vs. `get` / `describe`** — Those show the *current* state of a resource. Activity shows the *history* of actions and changes over time. Use `describe` to see what a resource looks like now; use `history --diff` to see how it got there. * **vs. the console's activity dashboard** — The [interactive console](/docs/datumctl/console) has an activity dashboard that browses this same data visually. The `activity` subcommands are the scriptable, exportable counterpart — the same information, shaped for pipelines and the terminal. ## Next steps Pick the subcommand that matches your question from the routing table above, or dig into the two detail guides: > [card] [Audit logs & events](/docs/datumctl/activity/audit-and-events) > > The `audit` and `events` subcommands — prove who did what for compliance, and surface lifecycle warnings during an incident. > [card] [Change history & feed](/docs/datumctl/activity/change-history-and-feed) > > The `history` and `feed` subcommands — diff a single resource across versions, or watch a merged, human-readable timeline. ## Related * [Audit logs & events](/docs/datumctl/activity/audit-and-events) — deep dive on `audit` and `events`, including compliance exports and incident-response triage. * [Change history & feed](/docs/datumctl/activity/change-history-and-feed) — deep dive on `history` and `feed`, including spec diffs and the live `--watch` timeline. * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — how the active organization and project are resolved before any activity query runs. * [Discovering resources & schemas](/docs/datumctl/discovering-resources) — find the `RESOURCE_TYPE` names that `activity history` expects. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — JSON/YAML/JSONPath, structured errors, and exit codes for wrapping activity queries in automation. * [Interactive console](/docs/datumctl/console) — browse this same activity data visually in the console's activity dashboard. # AI assistant Source: https://datum.net/docs/datumctl/ai/assistant Chat with Patch, the Datum Cloud assistant, from the terminal: ask about your resources in plain language, diagnose problems, and review changes before they're applied. Patch is the Datum Cloud assistant. Ask it about your resources in plain language, such as "why is my workload not available?" or "is my load balancer serving?", and it answers using the same permissions your account already has. There's no API key to bring and nothing to configure. `datumctl assistant` is a [plugin](/docs/datumctl/plugins/using-plugins), so it updates on its own schedule. To install it, run the following command: ```bash theme={null} datumctl plugin install assistant ``` ## Start a conversation > [tab] Full-screen chat > > ```bash theme={null} > datumctl assistant > ``` > > Opens the full-screen chat for your current project. Add `-c` to open your most recent conversation instead of a new one. > [tab] One question > > ```bash theme={null} > datumctl assistant chat "Why is the api-backend workload not available?" > ``` > > Sends one message, streams the answer, and exits. This is the form to use in scripts and pipes. The conversation ID is printed on standard error. > [tab] Pick up where you left off > > ```bash theme={null} > datumctl assistant resume > ``` > > Opens a searchable list of your past conversations. Add `--last` to go straight to the most recent one. To continue a conversation from a script, pass the ID it printed, or `-c` for the most recent one: ```bash theme={null} datumctl assistant chat -c "and the edge-cache one?" datumctl assistant chat "and the edge-cache one?" --context-id CONTEXT_ID ``` For terminals that can't run the full-screen chat, use `datumctl assistant chat --interactive` for a line-based session. ## Choose the project Patch works in your current context. To ask about another project, select it first with `datumctl ctx use`, or pass `--project your-project-id`. See [Contexts and scoping](/docs/datumctl/contexts-and-scoping). ## What Patch can do Patch answers questions about your project and its resources, and uses diagnostic tools from the Datum services your project uses. For example, it can: * **Diagnose a Compute workload** that isn't available, explain a status reason, check quota, and list Instances and instance types. * **Diagnose a load balancer** that isn't serving, and summarize its traffic, certificates, hostnames, and traffic protection. * **Diagnose a network** that isn't ready, and its network services. To see exactly which services and tools Patch can use in your project, run the following command: ```bash theme={null} datumctl assistant card ``` ## Making changes Patch never changes anything on its own. It proposes a change as a plan: the exact manifests it intends to apply, shown before anything happens. Applying a plan is a separate, explicit step, and the service refuses any manifest that differs from the one you were shown. ## Manage conversations Conversations are kept by the assistant service, not on your machine, so you can continue one from anywhere you're logged in. | Command | What it does | | - | - | | `datumctl assistant conversations list` | List the conversations in the project. Add `--archived` to include archived ones. | | `datumctl assistant conversations show CONTEXT_ID` | Print one conversation's transcript. | | `datumctl assistant conversations rename CONTEXT_ID` | Name a conversation. | | `datumctl assistant conversations archive CONTEXT_ID` | Hide a conversation from the list without deleting it. It stays resumable by ID. | | `datumctl assistant conversations delete CONTEXT_ID` | Permanently delete a conversation and its transcript. | ## Use Patch from your own agent Your own AI agent can hand questions to Patch, which brings Datum's own diagnostic tools and guidance: * **From a script or another agent,** run `datumctl assistant chat "QUESTION"`. It streams the answer to standard output and prints the conversation ID on standard error, so the agent can continue with `--context-id`. * **From Claude Code,** the [Datum MCP repository](https://github.com/datum-cloud/datum-mcp) includes a skill that teaches Claude when an open-ended diagnostic question, or a risky change across several resources, is better handed to Patch than done with [Datum MCP](/docs/agents/datum-mcp)'s own tools. * **Over the A2A protocol,** Patch publishes an agent card that describes its endpoint, its skills, and the Datum services it can use in your project. To see it, run `datumctl assistant card`. ## Moving from datumctl ai `datumctl ai` was a local assistant that needed your own Anthropic, OpenAI, or Gemini key. It has been replaced by Patch. The `datumctl ai` command still works and now runs the `assistant` plugin, so existing scripts keep going. Nothing reads your old provider key anymore, but it's still on disk. Delete it: > [tab] macOS / Linux > > ```bash theme={null} > rm ~/.config/datumctl/ai.yaml > ``` > [tab] Windows > > ```powershell theme={null} > del %AppData%\datumctl\ai.yaml > ``` The chat pane in the [interactive console](/docs/datumctl/console) ran on the same local assistant and has also been removed. Use `datumctl assistant` for chat. Chats you held in the console were saved to `~/.datumctl/conversations`, and nothing reads them now either. ## Next steps * [Connecting an AI client with the MCP server](/docs/datumctl/ai/mcp-server) — use Datum from Claude, Cursor, or another MCP client instead of the terminal. * [Agent skills](/docs/agents/skills/index) — teach your own agents how to work with Datum resources. * [`datumctl` overview](/docs/datumctl/overview) — everything else the CLI can do. # MCP server Source: https://datum.net/docs/datumctl/ai/mcp-server Connect Claude Desktop, Cursor, and other MCP-aware AI clients to your Datum Cloud resources with the Datum MCP server. The [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) lets AI clients like Claude Desktop and Cursor talk to your Datum Cloud resources directly, outside the terminal. This page covers the Datum MCP server. For the hosted Datum Docs MCP server, see [Datum Docs MCP](/docs/agents/datum-mcp#datum-docs-mcp). Where the [AI assistant](/docs/datumctl/ai/assistant) is the interactive, terminal-first experience in `datumctl`, the Datum MCP server is the bridge that gives a desktop AI assistant secure, standards-based access to your organizations, projects, domains, proxies, DNS, and more. > [!NOTE] > > The Datum MCP server is distributed as a separate `datum-mcp` binary, not as a > `datumctl` subcommand. Installation, the full tool catalog, and one-click setup > for clients such as Cursor live on the dedicated [Datum MCP](/docs/agents/datum-mcp) page > and upstream in the [Datum MCP repository](https://github.com/datum-cloud/datum-mcp). ## When to use the MCP server Reach for the MCP server when you want an AI client that lives outside your terminal to work with Datum Cloud: * **Claude Desktop** — chat with an assistant that can read and change your Datum resources. * **Cursor** and other IDE-embedded assistants — bring Datum context into your editor while you build. * **Claude Code** — install the `datum-cloud` plugin, which starts the server for you. See [Add it to your AI client](/docs/agents/datum-mcp#add-it-to-your-ai-client). * Any other MCP-aware client. Datum MCP speaks the standard stdio transport, and can also serve streamable HTTP. If you would rather stay in the terminal, `datumctl assistant` answers questions in natural language without a separate MCP client — see the [AI assistant](/docs/datumctl/ai/assistant) guide. ## What it exposes The Datum MCP server gives connected clients tools to inspect and manage your Datum Cloud resources. Clients can understand the current state of your resources, suggest and implement changes, and help diagnose common configuration issues. The catalog covers: * Organizations, projects, and users * Domains, load balancers (HTTP proxies, routes, and gateways), and traffic protection * DNS zones and records * gVPC networks and network services * Compute workloads and Instances * IP address management, IAM, billing, and services * Activity search * Any other Datum resource, through a generic escape hatch For load balancers, networks, Compute, DNS, and activity, the server also includes each service's own diagnostic tools, the same ones Datum's [AI assistant](/docs/datumctl/ai/assistant) uses. To keep the tool list short, the server loads some toolsets on demand during a session. > [!NOTE] > > The tool catalog evolves with the server. See the [Datum MCP](/docs/agents/datum-mcp) > page for the authoritative, up-to-date list and the exact operations each tool > supports. ## Connecting a client Most MCP clients launch the server over stdio. Add an entry like this to your client's MCP configuration: ```json theme={null} { "datum-mcp": { "command": "datum-mcp", "args": [] } } ``` **1. Install the Datum MCP server** Follow the install instructions on the [Datum MCP](/docs/agents/datum-mcp) page. Binaries are available for Linux, macOS, and Windows on x86 and arm64. **2. Add the server to your client** In Claude Code, install the `datum-cloud` plugin from the `datum-cloud/claude-code-plugins` marketplace, which starts the server for you. For other clients, point them at the `datum-mcp` binary using a stdio configuration like the one above. Cursor users can use the one-click install link on the [Datum MCP](/docs/agents/datum-mcp) page. **3. Sign in on first use** The first tool call triggers a browser-based sign-in (see below). Approve it, and your client is connected. ## Authentication On first use, the server runs a browser-based OAuth (PKCE) sign-in, then stores credentials — including a refresh token — securely in the system keychain. Subsequent calls reuse and refresh the token automatically, so you sign in once. This is the same security model `datumctl` uses when you [log in](/docs/datumctl/auth/logging-in): no static API keys. To run Datum MCP without a browser, such as in CI or on a server, set `DATUM_TOKEN` and `DATUM_API_HOSTNAME` instead. With a [service account](/docs/datumctl/auth/service-accounts) token, the server works in that service account's own project. The server implements the latest [MCP Authorization](https://modelcontextprotocol.io/specification/2025-06-18/basic/authorization) specification and supports OAuth 2.1 (PKCE). > [!WARNING] > > The MCP server acts on real Datum Cloud resources with your credentials. > Connect it only to AI clients you trust, and review any changes an assistant > proposes before approving them. ## Next steps * [Datum MCP](/docs/agents/datum-mcp) — set up the Datum MCP server: install, one-click client setup, and the full tool catalog. * [Datum Docs MCP](/docs/agents/datum-mcp#datum-docs-mcp) — give your AI client the docs as well, with no installation. * [AI assistant](/docs/datumctl/ai/assistant) — prefer the terminal? Use `datumctl assistant` instead. * [Logging in](/docs/datumctl/auth/logging-in) — review how Datum Cloud handles credentials before your first connection. # Local API proxy Source: https://datum.net/docs/datumctl/api-proxy Run a local authenticated proxy so any tool on your machine can talk to the Datum Cloud API without handling tokens. `datumctl api proxy` starts a local HTTP proxy that forwards every request to the Datum Cloud API endpoint of your session, adding your credentials automatically and refreshing them as needed. Point any local tool — a dev server, a test harness, `curl` — at the printed URL and it can talk to the platform with no tokens to copy and no expiry to manage. ```bash theme={null} # Start a proxy on a fixed port, then call the API through it datumctl api proxy --port 8001 curl http://127.0.0.1:8001/apis/resourcemanager.miloapis.com/v1alpha1/organizations ``` > [!NOTE] > > Running in CI or an unattended pipeline? Use a [service account](/docs/datumctl/auth/service-accounts) for the session the proxy serves — the proxy carries whatever identity you are logged in as. ## Start the proxy ```bash theme={null} # Start a proxy on a fixed port for a dev server datumctl api proxy --port 8001 # Start on a random free port; the URL is printed on the first stdout line datumctl api proxy # Serve a session other than your active one (see 'datumctl auth list' for names) datumctl api proxy --session sam@datum.net@api.staging.env.datum.net # Suppress per-request log lines datumctl api proxy --quiet ``` By default the proxy picks a random free port, so starting a second proxy never fails. Two things are always printed once the listener is ready: * A human-readable banner on **stderr** names the session, upstream, scope, and listen address, so you can verify at a glance whose credentials the proxy serves. * The bare proxy URL (for example `http://127.0.0.1:52347`) is printed as the **first and only line on stdout**. Scripts and test harnesses can read that one line as their readiness signal. ```go theme={null} cmd := exec.Command("datumctl", "api", "proxy", "--quiet", "--project", testProject) stdout, _ := cmd.StdoutPipe() _ = cmd.Start() apiURL, _ := bufio.NewReader(stdout).ReadString('\n') // first line = ready + address ``` Press `Ctrl+C` to stop the proxy. In-flight requests get a short grace period before their connections are closed. ## Paths and scoping By default the proxy is a pure passthrough of the platform API surface: the same paths that work against the real API endpoint work against the local port, including the scoped organization and project control-plane prefixes. Swapping one base URL gives a tool dev/prod parity with nothing else to change. ```bash theme={null} # Watch DNS zones on a project control plane through the proxy curl "http://127.0.0.1:8001/apis/resourcemanager.miloapis.com/v1alpha1/projects/your-project-id/control-plane/apis/networking.datumapis.com/v1alpha/dnszones?watch=true" ``` With an explicit `--project` or `--organization` flag, the proxy instead serves that single control plane at its root, so URLs lose the long control-plane prefix: ```bash theme={null} datumctl api proxy --port 8001 --project your-project-id curl "http://127.0.0.1:8001/apis/networking.datumapis.com/v1alpha/dnszones?watch=true" ``` > [!NOTE] > > The session and scope are **pinned when the proxy starts** and shown in the banner. Switching your active account with [`datumctl auth switch`](/docs/datumctl/auth/managing-accounts) or your context with `datumctl ctx use` does not affect a running proxy, and the proxy never inherits a scope from your current context — scoping is always an explicit flag. Restart the proxy to pick up a new session or scope. ## Streaming Streaming responses — watch requests, server-sent events, chunked transfer — pass through unbuffered: each event reaches your client the moment the upstream sends it, and the proxy never limits how long a response stays open. This makes the proxy suitable for long-lived watch clients. ## Credentials and token refresh Every proxied request carries a fresh access token for the pinned session; the proxy refreshes the token before it expires and persists refreshed tokens exactly as other `datumctl` commands do. Service-account sessions work the same way. If the session cannot be refreshed — expired, revoked, or logged out with `datumctl logout` — the proxy stays up and answers requests with a synthesized `502 Bad Gateway` carrying a JSON status body, an `X-Datum-Proxy-Error: true` marker header, and a message telling you to run `datumctl login`. The proxy recovers automatically, without a restart, once you log back in to the same session. > [!TIP] > > A `401` or `403` from the platform itself passes through unchanged and without the marker header, so your client can always tell a proxy-local authentication problem from a real platform answer. ## Security model * **Loopback only.** The proxy listens on `127.0.0.1` and there is no flag to bind other addresses. Anything that should be reachable remotely deserves a tunnel whose security model you own. * **Local clients are trusted.** Like other local developer proxies, the proxy does not authenticate local clients: while it runs, any process on your machine can make API calls as the pinned session. The banner names the identity it serves, requests are logged by default, and the credential itself is never exposed — a local client can act through the proxy but cannot take your token with it. * **Host-header validation.** Requests whose `Host` is not `localhost`, `127.0.0.1`, or `[::1]` are rejected with `403`, which defeats DNS-rebinding attacks from web pages. * **No CORS headers.** Browsers refuse scripted cross-origin reads of proxy responses; the proxy is meant for server-side and command-line clients. * **Authorization discipline.** Any `Authorization` header your local client sends is stripped and replaced with the session's real token, and tokens never appear in the request log. ## Request logging One line per request is written to stderr — silence it with `--quiet`: ``` 10:42:03 GET /apis/resourcemanager.miloapis.com/v1alpha1/organizations 200 143ms 8.1kB 10:42:05 GET /apis/…/projects/your-project-id/control-plane/…/portalplugins?watch=true 200 …streaming ``` Streaming responses log once when headers arrive (marked `…streaming`) and again when the stream ends, with total duration and bytes — so an abruptly closed watch is visible. # Logging in Source: https://datum.net/docs/datumctl/auth/logging-in Sign in to Datum Cloud from your terminal with datumctl login, verify who you are, and sign out. `datumctl` signs you in through your browser using [OAuth 2.0](https://oauth.net/2/) and [OpenID Connect (OIDC)](https://openid.net/developers/how-connect-works/) with [PKCE](https://oauth.net/2/pkce/), then stores the resulting credentials securely in your operating system keyring. There are no API keys to create, copy, or rotate — you log in as yourself and `datumctl` refreshes your tokens automatically. This page covers interactive sign-in: the browser flow, the device-code flow for headless shells, and how to check or drop your session. > [!NOTE] > > Running in CI, a build pipeline, or any unattended process? Use a service account instead of an interactive login — see [Service accounts](/docs/datumctl/auth/service-accounts). ## Log in Run: ```bash theme={null} datumctl login ``` `datumctl` opens your browser to complete sign-in. Once you approve, it stores your credentials in your keyring and confirms who you are: ```text theme={null} ✓ Authenticated as Ada Lovelace (ada@example.com) ``` You only need to log in once per machine. `datumctl` keeps your session refreshed, so day-to-day commands just work until you sign out. In the browser, you sign in with a passkey or with your Google or GitHub account. For the steps, see [Sign in](/docs/account/sign-in). ## Login, then pick a context `login` does two things in one step: it authenticates you, and it helps you choose a default **context** — the organization or project your commands target so you do not have to pass `--organization` or `--project` every time. Immediately after sign-in, `datumctl` discovers what you can access and shows a summary: ```text theme={null} Discovering organizations and projects... You have access to 2 organization(s): Acme Corp (4 project(s)) Personal (1 project(s)) ``` It then opens an interactive picker so you can select the organization or project to work in. When you choose, `datumctl` confirms it: ```text theme={null} ✓ Context set to acme-corp/production ``` From then on, commands like `datumctl get projects` act inside that context. To view or change your context later, use `datumctl ctx` and `datumctl ctx use`. > [!NOTE] > > If context discovery fails (for example, on a restricted network), `datumctl` still saves your credentials and tells you to set a context manually with `datumctl ctx use`. Your login is not lost. For the full picture of how scope is resolved and how to move between contexts, see [Contexts & scoping](/docs/datumctl/contexts-and-scoping). ## Headless and remote shells For unattended use such as CI, don't use this flow, because it needs a person to approve a code. Use a [service account](/docs/datumctl/auth/service-accounts) with `datumctl login --credentials` instead. On a machine with no browser where you are present — an SSH session, a container, or a remote development box — add `--no-browser` to authenticate with the device authorization flow: ```bash theme={null} datumctl login --no-browser ``` Instead of launching a browser, `datumctl` prints a URL and a short code. Open the URL on any device where you are signed in, enter the code, and approve. `datumctl` completes sign-in on the remote machine once you confirm. > [!NOTE] > > On headless runners, `datumctl` may fall back to a file-based credential store when no system keyring is available. When it does, it prints a one-line warning so you know where your credentials are kept. ## Sign in to a custom environment By default `datumctl` authenticates against the production Datum Cloud environment. To sign in against a different environment — such as a self-hosted or otherwise custom Datum Cloud deployment — point at its authentication server with `--hostname`: ```bash theme={null} datumctl login --hostname auth.example.com ``` The `--hostname` flag works with the browser flow, `--no-browser`, and service-account logins alike. Each environment's session is stored separately, so you can hold sessions for more than one environment at once and move between them. ## Check who you are To see the account and context a command would use right now: ```bash theme={null} datumctl whoami ``` This prints your user, your active context, and the organization (and project, if set) it resolves to: ```text theme={null} User: Ada Lovelace (ada@example.com) Context: acme-corp/production Organization: Acme Corp (acme-corp) Project: Production (production) ``` If you have set the `DATUM_ORGANIZATION` or `DATUM_PROJECT` environment variables, `whoami` also flags them, since those override the active context for the current shell. > [!NOTE] > > `datumctl auth whoami` is a different command, for kubectl users only: it asks the control plane which identity and groups your kubeconfig credentials resolve to, and needs a context from `datumctl auth update-kubeconfig`. See [kubectl interop](/docs/datumctl/kubectl-interop). To list the datumctl accounts you are signed in with, use `datumctl auth list`. ## Log out To remove a stored session, pass the email address you signed in with: ```bash theme={null} datumctl logout ada@example.com ``` To remove every Datum Cloud session `datumctl` has stored on this machine: ```bash theme={null} datumctl logout --all ``` Logging out deletes the credentials from your keyring. Sign back in at any time with `datumctl login`. > [!NOTE] > > Signed in with more than one account, or need long-lived tokens for scripts and the credential helper? See [Managing accounts & tokens](/docs/datumctl/auth/managing-accounts) for listing accounts, switching the active user, and printing access tokens. ## Related * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — how the account you are signed in as maps to the organizations and projects your commands target. * [Managing accounts & tokens](/docs/datumctl/auth/managing-accounts) — list local sessions, switch the active user, and retrieve access tokens. * [Service accounts](/docs/datumctl/auth/service-accounts) — non-interactive credentials for CI, pipelines, and automation. * [Install](/docs/datumctl/install) — install `datumctl` and run your first command. # Managing accounts & tokens Source: https://datum.net/docs/datumctl/auth/managing-accounts List, switch between, and remove locally authenticated Datum Cloud users, and print access tokens for tooling. `datumctl` can hold sessions for more than one Datum Cloud user at a time. This is useful when you work across several identities — a personal account and a machine account, or accounts on different environments — and want to move between them without logging out and back in. This guide covers listing your local sessions, switching the active one, logging out, and printing access tokens for other tools. > [!NOTE] > > New to the CLI? Start with the [Install](/docs/datumctl/install) and [Logging in](/docs/datumctl/auth/logging-in) guides. This page assumes you have already logged in at least once with `datumctl login`. ## Working with multiple accounts Every time you run `datumctl login`, the resulting credentials are stored as a separate session. One session is always marked **active** — its credentials are used by every command that talks to the Datum Cloud API. The commands below let you see all your sessions and choose which one is active. ### List your sessions ```bash theme={null} datumctl auth list # Alias: datumctl auth ls ``` This prints a table of every locally stored user: | Column | Meaning | | - | - | | `User` | The email address used to log in. Pass this to `datumctl auth switch` or `datumctl logout`. | | `Endpoint` | Shown only when your sessions span more than one API endpoint (for example production and a custom environment). | | `Status` | `Active` marks the account whose credentials are used by default for all subsequent commands. | If you have not logged in yet, the command tells you to run `datumctl login` to get started. ### Switch the active account ```bash theme={null} # Switch to a specific account datumctl auth switch your-email # Or pick from an interactive list datumctl auth switch ``` Pass the email address exactly as it appears in `datumctl auth list`. If you omit the email, an interactive picker lists all your sessions so you can choose one. If a single email matches sessions on more than one endpoint, you are prompted to pick the right one. > [!NOTE] > > Each session remembers the last context (organization or project) you used. Switching accounts also restores that account's context, so your active scope follows the identity you switch to. See [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how the active context shapes what your commands act on. After switching, `datumctl` confirms the active user and shows the restored context (and endpoint, when you have sessions on more than one): ``` ✓ Switched to Ada Lovelace (your-email) Context: my-project (proj-abc123) ``` The account must already be logged in. To add a new one, run `datumctl login` first — see [Logging in](/docs/datumctl/auth/logging-in) for login options, including `--no-browser` for headless environments, or [Service accounts](/docs/datumctl/auth/service-accounts) for machine-identity credentials. ### Log out Logging out removes a user's stored credentials from this machine. ```bash theme={null} # Log out a single account datumctl logout your-email # Log out every account stored on this machine datumctl logout --all ``` You must either name an account or pass `--all` — the two cannot be combined. Use the email exactly as shown in `datumctl auth list`. > [!WARNING] > > `datumctl logout --all` removes every stored session and clears your saved contexts. You will need to run `datumctl login` again before you can run any authenticated command. If you log out the active account while others remain, use `datumctl auth switch` to choose a new active session. ## Printing access tokens for tooling `datumctl auth get-token` prints an access token for the active user. Most people never need it — `datumctl` obtains and refreshes tokens automatically for its own commands. It exists as a **credential helper** so that other tools can borrow your Datum Cloud session instead of managing their own login. ```bash theme={null} # Print a raw bearer token (default output) datumctl auth get-token ``` If the stored token has expired, `datumctl` uses the saved refresh token to fetch a new one before printing, so you always get a valid token. How long an access token stays valid isn't documented yet, so don't plan around a specific lifetime. Because `get-token` returns a valid token each time it runs, call it again when you need one instead of saving its output for a long-running job. Common uses: * **Scripts and direct API calls** — capture the raw token and pass it as a bearer credential to `curl` or another HTTP client. * **Plugins** — [datumctl plugins](/docs/datumctl/plugins/using-plugins) call `get-token` to obtain a fresh token for the active session rather than implementing their own authentication. * **kubectl** — for existing kubectl users, `get-token` also emits a Kubernetes-format credential object. This path is set up for you by `datumctl auth update-kubeconfig`; see [using datumctl with kubectl](/docs/datumctl/kubectl-interop). ### Flags | Flag | Description | | - | - | | `--output`, `-o` | Output format. `token` (default) prints the raw access token; `client.authentication.k8s.io/v1` prints a credential object for kubectl. | | `--session` | Look up a specific session by name instead of the active one. Used by tooling that pins each configuration to its own session. | > [!WARNING] > > A printed token is a live credential for your Datum Cloud session. Do not log it, commit it, or paste it where it might be captured. Prefer piping it directly into the tool that needs it rather than storing it in a file or environment variable. ## Where credentials are stored `datumctl` splits what it stores in two places: * **Secrets** (access and refresh tokens) go into your operating system keyring — Keychain on macOS, Secret Service on Linux, Credential Manager on Windows. * **Session and context metadata** (which accounts exist, which is active, and each account's last context) lives in `~/.datumctl/config`. > [!NOTE] > > When the OS keyring is unavailable — commonly on headless CI runners — `datumctl` falls back to a file at `~/.datumctl/credentials.json`, written with owner-only (`0600`) permissions. The fallback engages automatically; the rest of your workflow is unchanged. This file holds live tokens in plain text, so treat it as sensitive and keep it off shared or version-controlled storage. ## Next steps * [Logging in](/docs/datumctl/auth/logging-in) — the [PKCE](https://oauth.net/2/pkce/) browser flow, `--no-browser` device flow, `whoami`, and signing out. * [Service accounts](/docs/datumctl/auth/service-accounts) — non-interactive machine credentials for CI and automation. * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — how the active account maps to the organizations and projects your commands target. * [Using plugins](/docs/datumctl/plugins/using-plugins) — extend the CLI with commands that reuse your session through the credential helper. * [Using datumctl with kubectl](/docs/datumctl/kubectl-interop) — hand your active session to kubectl (existing kubectl users only). # Service accounts Source: https://datum.net/docs/datumctl/auth/service-accounts Authenticate datumctl non-interactively with a service account credentials file for CI, automation, and headless environments. Service accounts let `datumctl` authenticate without a human at the keyboard. Instead of opening a browser, you point the CLI at a credentials file, and it signs in as the service account's machine identity. Use this for continuous integration jobs, scheduled automation, and any headless runner where an interactive login is not possible. > [!NOTE] > > Service accounts are **created in the Datum Cloud portal**, which is where you generate and download the credentials file. This page covers **using** those credentials with `datumctl`. To create a service account and download its key, see the portal guide on [service accounts](/docs/projects/service-accounts). ## When to use a service account Choose the authentication method that matches who — or what — is running the command: > [card] Interactive login > > A person at a terminal. Run `datumctl login` to authenticate through your browser (or `datumctl login --no-browser` for a device-code flow). Credentials refresh automatically and are tied to your user identity. > [card] Service account > > A pipeline, cron job, or agent with no human present. Run `datumctl login --credentials ` to sign in as a machine identity using a downloaded credentials file. Service accounts are the right choice when: * The command runs unattended, so no one can complete a browser or device prompt. * You want a dedicated, auditable identity for automation rather than a personal account. * You need the same credentials to work reproducibly across many runners. ## Log in with a credentials file Once you have downloaded a credentials JSON file from the portal, authenticate by passing its path to `datumctl login`: ```bash theme={null} datumctl login --credentials ./my-service-account.json ``` `datumctl` reads the file, requests an access token for the service account, and stores the resulting session so subsequent commands are authenticated. After login you can run any command as the service account: ```bash theme={null} datumctl get organizations ``` > [!NOTE] > > Service-account login is non-interactive: it does **not** open a browser and does **not** prompt you to pick a context. Set the organization or project your commands should target explicitly — see [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — or pass `--organization` / `--project` on each command. ### Targeting a custom environment The credentials file is issued for a specific Datum Cloud environment. For a self-hosted or otherwise custom environment, also pass that environment's auth hostname so token discovery points at the right server: ```bash theme={null} datumctl login --credentials ./my-service-account.json --hostname auth.example.com ``` The default `--hostname` is `auth.datum.net` (production). You normally do not need `--api-hostname` — `datumctl` derives the API server from the auth hostname — but you can override it if your environment requires a specific API endpoint. ### Confirming the active identity To verify which identity is active, use: ```bash theme={null} datumctl whoami ``` The service account also appears in the local account list, where its `client_email` is shown as the name: ```bash theme={null} datumctl auth list ``` ## The credentials file The credentials file is a JSON document you download from the portal. It looks like this: ```json theme={null} { "type": "datum_service_account", "client_email": "ci-deployer@example", "client_id": "1234567890", "private_key_id": "abc123", "private_key": "-----BEGIN RSA PRIVATE KEY-----\n...\n-----END RSA PRIVATE KEY-----\n", "token_uri": "https://auth.datum.net/oauth/v2/token", "api_endpoint": "https://api.datum.net", "scope": "openid profile email offline_access" } ``` Key fields: | Field | Purpose | | - | - | | `type` | Must be `datum_service_account`. `datumctl` rejects any other value. | | `client_id` | The service account's identity. **Required.** | | `private_key_id` | Identifies which signing key is in use. **Required.** | | `private_key` | PEM-encoded private key used to sign the login request. **Required.** | | `client_email` | Human-readable identity, shown as the account name in `datumctl auth list`. | | `token_uri`, `api_endpoint`, `scope` | Environment and authorization details carried in the file. | > [!WARNING] > > The credentials file contains a **private key**. Treat it like a password: never commit it to source control, restrict its file permissions, and store it in your CI system's secret manager. Anyone with the file can authenticate as the service account. If login fails, `datumctl` reports exactly what is wrong — a missing required field, an unreadable file, or a `type` that is not `datum_service_account` — so you can correct the file and retry. To inspect the token request in detail while troubleshooting, add `--debug`, which prints the login JWT's claims to standard error. ## Credential storage on headless runners On a runner with no system keyring, `datumctl` falls back to an unencrypted file at `~/.datumctl/credentials.json`. For what that means and how to keep it clean, see [Credential storage on headless runners](/docs/datumctl/cicd/automating#credential-storage-on-headless-runners). ## Related * [Create a service account in the portal](/docs/projects/service-accounts) — generate the credentials file this page consumes. * [Automating datumctl in CI/CD](/docs/datumctl/cicd/automating) — wire service-account login into any pipeline, with credential helpers and structured errors. * [GitHub Actions](/docs/datumctl/cicd/github-actions) — install and authenticate a service account in a GitHub workflow in one step. * [Logging in](/docs/datumctl/auth/logging-in) — the interactive browser and device-code flows for humans at a terminal. * [Managing accounts & tokens](/docs/datumctl/auth/managing-accounts) — list sessions, switch the active identity, and print access tokens. * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — set the organization or project your automated commands target. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — machine-readable output and structured errors for automation. # Automating datumctl in CI/CD Source: https://datum.net/docs/datumctl/cicd/automating Run datumctl non-interactively in any pipeline — headless service-account login, credential helpers, structured errors, and exit-code handling on headless runners. `datumctl` is built to run unattended. Any CI system — GitLab CI, CircleCI, Jenkins, Buildkite, Argo, a cron job, or a container entrypoint — can authenticate as a service account, run commands, and branch on machine-readable results without a human at the keyboard. > [!NOTE] > > On [GitHub Actions](https://docs.github.com/actions), you rarely need to wire this up by hand. The [setup-datumctl-action](/docs/datumctl/cicd/github-actions) installs the CLI and authenticates a service account in one step. This page is for every other pipeline, and for understanding what that action does under the hood. ## Authenticate as a service account Interactive browser login is the default, and it is the wrong choice for a pipeline — there is no browser to open and no prompt a build agent can answer, and the `--no-browser` device flow needs a person to approve a code. Instead, authenticate with a **service account** credentials file using `datumctl login --credentials`. For the credentials file, custom environments, and how to confirm the active identity, see [Service accounts](/docs/datumctl/auth/service-accounts#log-in-with-a-credentials-file). To create a service account and download its credentials file, see [Service accounts in the portal](/docs/projects/service-accounts). In a pipeline, the steps are: **1. Store the credentials JSON as a secret** Save the full contents of the service account credentials file as a CI secret (for example `DATUM_SA_CREDENTIALS`). Do not inline it into a script or commit it to the repository. **2. Materialize it to a file at runtime** `datumctl login --credentials` takes a **path**, so write the secret to a file in the job's workspace first. Prefer a path the runner cleans up automatically. ```bash theme={null} printf '%s' "$DATUM_SA_CREDENTIALS" > "$RUNNER_TEMP/datum-credentials.json" datumctl login --credentials "$RUNNER_TEMP/datum-credentials.json" ``` **3. Run your datumctl commands** Once logged in, the session is cached and later steps in the same job are authenticated. ```bash theme={null} datumctl get projects --organization "$DATUM_ORG" -o json ``` ### A generic pipeline step The same three moves work in any CI system. A GitLab CI job, for example: ```yaml theme={null} deploy: image: alpine:latest script: - printf '%s' "$DATUM_SA_CREDENTIALS" > /tmp/datum-credentials.json - datumctl login --credentials /tmp/datum-credentials.json - datumctl apply -f ./project.yaml --organization "$DATUM_ORG" ``` > [!WARNING] > > Base64-encode the secret if your CI strips newlines or mangles multi-line values — the credentials JSON contains a PEM private key with embedded newlines. Decode it back to a file before calling `datumctl login`. (The GitHub Action accepts raw JSON or base64 for exactly this reason.) ## Use auth get-token as a credential helper When another tool in your pipeline needs a raw bearer token to call the Datum API directly, `datumctl auth get-token` prints one for the active session on stdout: ```bash theme={null} TOKEN=$(datumctl auth get-token) curl -H "Authorization: Bearer $TOKEN" https://api.datum.net/... ``` `datumctl` handles refresh for you: if the stored token has expired, `get-token` uses the stored refresh material to obtain a new one before printing, so the value you capture is always fresh. This makes it a natural credential helper for scripts and HTTP clients that live alongside `datumctl` in a job. > [!NOTE] > > Most commands never need this — `datumctl` authenticates its own requests automatically. Reach for `auth get-token` only when a *separate* process needs the token. For its Kubernetes `ExecCredential` output mode, see [using datumctl with kubectl](/docs/datumctl/kubectl-interop). ## Get machine-readable failures with --error-format=json By default `datumctl` prints errors as human-readable text on stderr. In a pipeline you want to branch on failures programmatically, so pass the global `--error-format=json` flag to get a single-line JSON envelope instead: ```bash theme={null} datumctl get projects --organization does-not-exist --error-format json ``` ```json theme={null} { "error": { "message": "the organization \"does-not-exist\" was not found", "hint": "Run 'datumctl get organizations' to see organizations you can access.", "retryable": false } } ``` `--error-format` is a persistent flag available on every command, and it only shapes errors on stderr — successful output on stdout is still controlled by `-o`. Read the `retryable` field to decide whether to back off and retry, and `code`/`hint` to decide what to do next. For the full envelope schema and the allowed values (`human`, `json`, `yaml`), see [Output formats & scripting](/docs/datumctl/output-and-scripting). ## Branch on exit codes `datumctl` follows conventional exit-code semantics, so a pipeline can gate on `$?`: | Exit code | Meaning | | - | - | | `0` | Success. | | `1` | A general error — unknown command, invalid flag value, authentication failure, or a failed API request. | | `>1` | A lower-level fatal error from the underlying request machinery. | `datumctl diff` is the exception: it exits `0` when there are no differences, `1` when there are, and `>1` on error. Exit code `1` is not a failure, so don't chain `diff && apply`: the apply would run only when there is nothing to change. Handle each code explicitly: ```bash theme={null} datumctl diff -f ./project.yaml --organization "$DATUM_ORG" case $? in 0) echo "No changes." ;; 1) datumctl apply -f ./project.yaml --organization "$DATUM_ORG" ;; *) echo "diff failed" >&2; exit 1 ;; esac ``` > [!WARNING] > > `datumctl delete` runs without a confirmation prompt. In automation, gate destructive operations behind an explicit review step or a `--dry-run=client` preview. See [Output formats & scripting](/docs/datumctl/output-and-scripting) for the full scripting checklist, including pinning `--organization`/`--project` on every call. ## Credential storage on headless runners On a workstation, `datumctl` stores your session in the operating system keyring (Keychain, Secret Service, and so on). CI runners and containers usually have **no system keyring available**, so `datumctl` transparently falls back to an on-disk file at: ```text theme={null} ~/.datumctl/credentials.json ``` The file is written with `0600` permissions but is **not encrypted**, so on a shared or long-lived runner you should treat it as sensitive. When the fallback engages, `datumctl login` prints a one-time warning to stderr, for example: ```text theme={null} warning: system keyring unavailable (...); falling back to insecure file storage at /home/runner/.datumctl/credentials.json ``` This is expected and harmless on an ephemeral runner. To keep it clean: * Prefer **ephemeral runners** that are destroyed after the job, so the credentials file never outlives the build. * On a **persistent or shared runner**, log out at the end of the job so the file does not linger: ```bash theme={null} datumctl logout --all ``` * Point `$HOME` at a job-scoped, cleaned-up directory if your runner reuses home directories across builds, so each job gets its own credential file. ## Related * [GitHub Actions](/docs/datumctl/cicd/github-actions) — install and authenticate `datumctl` in a GitHub workflow in a single step. * [Service accounts](/docs/datumctl/auth/service-accounts) — the credentials-file format and how to use it with `datumctl login --credentials`. * [Create a service account in the portal](/docs/projects/service-accounts) — generate and download the credentials this page consumes. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — the `-o json` output modes, error envelope, and exit-code semantics. * [Using datumctl with kubectl](/docs/datumctl/kubectl-interop) — the `datumctl auth get-token` credential-helper mode for kubectl in CI. # GitHub Actions Source: https://datum.net/docs/datumctl/cicd/github-actions Install and authenticate datumctl in GitHub Actions with the setup-datumctl-action, then run Datum Cloud commands in your workflows. The [`setup-datumctl-action`](https://github.com/datum-cloud/setup-datumctl-action) is the officially supported way to run `datumctl` in a [GitHub Actions](https://docs.github.com/actions) workflow. In a single step it does two things: 1. **Installs `datumctl`** — downloads the released binary for your runner's OS and architecture, verifies its published SHA-256 checksum, and adds it to `PATH` (no `sudo` required). 2. **Authenticates a service account** — writes your service account credentials into the `datumctl` keyring so every later `datumctl` step in the job is already signed in. After the setup step runs, you can call any `datumctl` command directly. > [!NOTE] > > This page is the hub for running `datumctl` in CI/CD. If you use a CI system other than GitHub Actions, see [Automating datumctl in CI/CD](/docs/datumctl/cicd/automating) for the general headless-authentication and scripting patterns. ## Minimal workflow The workflow below checks out your repository, installs and authenticates `datumctl`, then runs Datum Cloud commands. The service account credentials come from a repository secret (see [Set up the credentials secret](#set-up-the-credentials-secret)). ```yaml theme={null} name: datum on: push: branches: [main] jobs: datum: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up datumctl uses: datum-cloud/setup-datumctl-action@v1 with: version: v0.15.0 credentials: ${{ secrets.DATUM_SA_CREDENTIALS }} - name: List projects run: datumctl get projects --organization your-org-id ``` Because the setup step authenticates for you, later steps run `datumctl` without a login command. Pass `--organization` (and `--project` where relevant) explicitly on every command so the workflow does not depend on ambient context state. > [!TIP] > > Prefer `datumctl apply -f` for declarative create-or-update, and gate destructive changes with `datumctl diff -f` — `datumctl delete` proceeds without a confirmation prompt. See [Output formats & scripting](/docs/datumctl/output-and-scripting) for exit codes and the `--error-format json` error envelope you can branch on in a pipeline. ## Inputs | Input | Required | Default | Description | | - | - | - | - | | `credentials` | Yes | — | Service account credentials JSON used to authenticate `datumctl`. Accepts either raw JSON or a base64-encoded blob. Always supply this from a secret — never inline it. | | `version` | No | `v0.15.0` | The `datumctl` release tag to install. Pin this to a known release for reproducible builds. | | `auth-hostname` | No | `auth.datum.net` | Datum Cloud authentication hostname. Override it (for example `auth.example.com`) to target a self-hosted or otherwise custom environment. | ## Outputs | Output | Description | | - | - | | `version` | The `datumctl` version that was installed. | | `bin-path` | The install directory that was added to `PATH`. | You can read these in later steps — for example, to log the installed version: ```yaml theme={null} - name: Set up datumctl id: setup uses: datum-cloud/setup-datumctl-action@v1 with: credentials: ${{ secrets.DATUM_SA_CREDENTIALS }} - name: Show installed version run: echo "Installed datumctl ${{ steps.setup.outputs.version }}" ``` ## Version pinning Pin both the action and the CLI so a workflow always behaves the same way: * **Pin the action tag** — `uses: datum-cloud/setup-datumctl-action@v1` tracks the latest `v1` release. For fully reproducible runs, pin to a specific tag (or commit SHA) instead. * **Pin the `version` input** — set `version:` to an explicit `datumctl` release tag rather than relying on the default, which changes as new releases ship. ```yaml theme={null} - uses: datum-cloud/setup-datumctl-action@v1 with: version: v0.15.0 credentials: ${{ secrets.DATUM_SA_CREDENTIALS }} ``` ## How install and authentication work Understanding what the setup step does under the hood helps you reason about security and troubleshooting. **1. Verified install** The action downloads the released `datumctl` tarball for the runner's OS and architecture — Linux and macOS on both `x86_64` and `arm64` — and verifies it against the checksum published with the release. It computes the SHA-256 of the downloaded artifact and compares it to the expected value, failing the step on a mismatch. The verified binary is placed on `PATH` without `sudo`. **2. Service account keyring auth** The action loads the credentials you passed and authenticates the service account into the `datumctl` keyring. On headless runners there is no system keyring, so `datumctl` uses its file-backed keyring. This is the same non-interactive path as `datumctl login --credentials ` — every subsequent `datumctl` step in the job inherits the stored session and is authenticated automatically. > [!NOTE] > > Service account sessions carry only the permissions granted to that account. Follow least-privilege and grant the service account exactly the access your workflow needs — nothing more. See [Service accounts](/docs/datumctl/auth/service-accounts) for the credential model. ## Set up the credentials secret The `credentials` input expects a service account credentials JSON document. Create one in the Datum portal and store it as an encrypted secret in GitHub. **1. Create a service account** In the [Datum portal](https://cloud.datum.net), open your organization's IAM settings and create a service account. Grant it only the permissions the workflow requires. **2. Create a Datum-managed key** For the service account, choose a Datum-managed key and download the credentials JSON. The full contents are shown only once, so save it immediately. **3. Store it as a secret** In your GitHub repository (or environment), add a secret — for example `DATUM_SA_CREDENTIALS` — and paste in the credentials JSON. The action accepts either the raw JSON or a base64-encoded version. **4. Reference it in the workflow** Pass the secret to the action's `credentials` input with `${{ secrets.DATUM_SA_CREDENTIALS }}`. Never commit credentials to the repository or inline them in workflow YAML. > [!WARNING] > > Treat the downloaded credentials file as sensitive. Store it only in GitHub's encrypted secrets (or your CI's secret store), rotate it periodically, and remove the local copy after uploading it. ## Targeting a custom environment To run against a self-hosted or otherwise custom Datum Cloud environment, set the `auth-hostname` input: ```yaml theme={null} - uses: datum-cloud/setup-datumctl-action@v1 with: credentials: ${{ secrets.DATUM_SA_CREDENTIALS }} auth-hostname: auth.example.com ``` > [!NOTE] > > If your GitHub Actions runners generate a kubeconfig for existing kubectl users, see the [kubectl interop guide](/docs/datumctl/kubectl-interop). In this section we use `datumctl` commands directly. ## Related * [Automating datumctl in CI/CD](/docs/datumctl/cicd/automating) — headless login, `datumctl auth get-token` as a credential helper, and cleanup patterns for any CI system. * [Service accounts](/docs/datumctl/auth/service-accounts) — the credential model behind CI authentication and the credentials-file format the action consumes. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — machine-readable output, exit codes, and the `--error-format json` error envelope you can branch on. * [Using datumctl with kubectl](/docs/datumctl/kubectl-interop) — generate a kubeconfig from a workflow for existing kubectl users. # Building images Source: https://datum.net/docs/datumctl/compute/building-images Turn a Dockerfile into an image Datum Compute can run, check it for compatibility, and publish it with datumctl compute build. `datumctl compute build` packages a Dockerfile and build context into an image, and optionally publishes it. Point it at the same Dockerfile you already use for a normal container — no Compute-specific syntax required. ```bash theme={null} datumctl compute build . ``` `build` is for workloads on the `unikernel` runtime class, the default. That class doesn't run the image as a normal container. It boots the image as a [unikernel](/docs/compute/runtimes/unikernels), which gives each Instance its own virtual machine with container-like density and startup speed. `build` packages your Dockerfile into the artifact that runtime boots from, and validates it against unikernel requirements as part of the same step — so a missing shared library or the wrong architecture shows up as a build-time error instead of a workload that never comes up. Workloads on the `general-purpose` runtime class run standard container images, so they don't need `build`. Push the image with the tools that you already use. For more information, see [Choose a runtime class](/docs/compute/runtimes/overview). > [!NOTE] > > `build` needs a BuildKit backend to run: Docker Desktop or Docker Engine with BuildKit enabled (the default on modern installs), a standalone `buildkitd`, or `BUILDKIT_HOST` pointing at one. With no `--output`, this is a **local debug build**: it packages the image so you can confirm it builds successfully, without writing or publishing anything. Once you're ready to deploy it, add `--output` or move on to [deploying workloads](/docs/datumctl/compute/deploying-workloads). ## Choosing a Dockerfile By default, `build` looks for `Dockerfile.datum` in the build context and falls back to `Dockerfile` if that file does not exist — a convenient way to keep a Compute-specific variant alongside your normal one without extra flags. Use `-f` to point at a Dockerfile anywhere else: ```bash theme={null} # Use a Dockerfile in a non-default location datumctl compute build -f ./deploy/Dockerfile . # Pass build-time variables used by ARG instructions datumctl compute build --build-arg VERSION=1.2.3 . # Package a specific stage of a multi-stage Dockerfile (default: the final stage) datumctl compute build --target production . ``` ## Writing the output somewhere `--output` (`-o`) accepts three kinds of destination: | Destination | Example | | - | - | | Registry reference | `--output ghcr.io/acme/api:latest` | | OCI archive (`.tar`) | `--output ./compute-image.tar` | | OCI layout directory | `--output ./compute-image` | ```bash theme={null} # Publish straight to a registry datumctl compute build --push --output ghcr.io/acme/api:latest . # Write a portable OCI archive instead datumctl compute build --output ./compute-image.tar . ``` > [!WARNING] > > When `--output` is a registry reference, `build` asks for confirmation before pushing. Pass `--push` to skip that prompt — required in CI, where there is no terminal to confirm on. ## Checking compatibility before you package it `--analyze` inspects the built entrypoint for the kind of problem that only shows up once Compute's unikernel runtime tries to boot the image — a binary built for the wrong architecture, a startup script whose interpreter is missing, or a shared library the final stage never copied in: ```bash theme={null} datumctl compute build --analyze . ``` Where a fix is a mechanical Dockerfile edit, the finding shows exactly what to change: ```text theme={null} error[missing-libs]: /app/server requires a runtime file that is not present in the image --> Dockerfile:8 | copy missing runtime files into the final image | - COPY --from=build /app /app | + COPY --from=build /app /app | + COPY --from=build /lib/x86_64-linux-gnu/libc.so.6 /lib/x86_64-linux-gnu/libc.so.6 ``` Reach for `--fix` once you're ready to apply changes like that automatically — it edits the selected Dockerfile in place and rebuilds. `--fix` implies `--analyze`, and only ever applies safe, exact-line edits — it will never restructure your Dockerfile. ```bash theme={null} datumctl compute build --fix . ``` ## Inspecting a published image `build inspect ` is a diagnostic for an image that already exists in a registry — useful when a deployment isn't starting and you want to confirm the image itself is the problem before looking at the workload. It reports the same kind of compatibility findings as `--analyze`, plus the packaged filesystem and startup configuration Compute will boot from. Add `--extended` for additional OCI metadata — the image index, manifest digest, and layer count. Inspection never downloads the packaged filesystem layer itself, so it stays fast even against a large image. ```bash theme={null} datumctl compute build inspect ghcr.io/acme/api:1.4.2 ``` ## Next steps * `datumctl compute build --help` and `datumctl compute build inspect --help` for the full flag reference. * [Deploying workloads](/docs/datumctl/compute/deploying-workloads) — take a published image and run it as a workload. # Deploying workloads Source: https://datum.net/docs/datumctl/compute/deploying-workloads Deploy a container image as a workload with datumctl compute deploy, attach Datum's ALB to it, and check its health with datumctl compute workloads. `datumctl compute deploy` takes a container image and runs it as a workload across one or more locations. It supports two ways of describing what to deploy — flags, or a manifest file for the declarative form — and both converge on the same underlying workload. ```bash theme={null} datumctl compute deploy api \ --image=ghcr.io/acme/api:1.4.2 \ --location=us-central-1,us-east-1 \ --min=2 \ --http-port=8080 ``` > [!WARNING] > > A new workload without `--runtime-class` runs on the `unikernel` runtime class. For that class, `--image` must point to an image produced by `datumctl compute build`, not a standard OCI image from `docker build` — see [Building images](/docs/datumctl/compute/building-images). To run a standard container image unchanged, add `--runtime-class=general-purpose` — see [Choose a runtime class](/docs/compute/runtimes/overview). If a workload named `api` doesn't exist, this command creates it with one [placement](/docs/compute/placement-and-scaling) named `default` that spans both locations, runs at least two Instances in each location, and attaches Datum's ALB to it. If you run the same command again with a new `--image`, it updates the existing workload. Like `datumctl apply`, `deploy` creates or updates. ## Flags at a glance | Flag | Meaning | | - | - | | `--image` | Container image to deploy, including the registry host, required with the flags path. With `--build`, also where the built image is pushed. | | `--location` | One or more locations to deploy to, comma-separated (`us-central-1,us-east-1`). | | `--city` | Every location in one or more cities, comma-separated (`DFW,IAD`). | | `--location-selector` | Every location whose topology labels match a selector (`topology.datum.net/region=us-east-1`). | | `--min` | Minimum Instances per location (default `1`). | | `--instance-type` | Instance type (default `datumcloud/d1-standard-2`). | | `--runtime-class` | `general-purpose` or `unikernel`. Defaults to `unikernel`, and can't be changed after the workload is created. | | `--network` | Network a new workload attaches to (default `default`). Can't be changed after the workload is created. | | `--http-port` | Port the container serves plain HTTP on. Attaches Datum's ALB to the workload. | | `--no-http` | Remove the workload's ALB. | | `--env` | Set an environment variable as `KEY=VALUE`, or remove one as `KEY-`. Repeat the flag for each variable. Values can contain commas. | | `--env-from-configmap` | Import every key of a ConfigMap as environment variables, as `NAME` or `NAME:PREFIX_`. `NAME-` removes the import. Repeatable. | | `--env-from-secret` | Import every key of a Secret as environment variables, as `NAME` or `NAME:PREFIX_`. `NAME-` removes the import. Repeatable. | | `--configmap` | Mount a ConfigMap as files, as `NAME:/mount/path`. `NAME-` removes the mount. Repeatable. | | `--secret` | Mount a Secret as files, as `NAME:/mount/path`. `NAME-` removes the mount. Repeatable. | | `--label` | Set a label on the workload and its Instances as `key=value`, or remove one as `key-`. Repeatable. Changing a label replaces the workload's Instances. | | `--build` | Build and push the image from a directory (default `.`) before deploying — see [Building images](/docs/datumctl/compute/building-images). | | `-f`/`--file` | Deploy from a manifest instead of flags. | | `-y`/`--yes` | Skip the confirmation prompts — needed in scripts and CI. | Pass exactly one of `--location`, `--city`, or `--location-selector`. The `--port` flag isn't supported. Use `--http-port` instead. Environment variables, ConfigMap and Secret imports, mounts, and labels carry forward. A redeploy that doesn't name one keeps the value that the workload already has, so an image update doesn't drop your configuration. To remove one, add a trailing `-`, for example `--env LOG_LEVEL-`. To set a single variable from one key of a Secret, use a manifest. These flags require compute plugin v0.10.7 or later, and you can't combine them with `-f`. > [!NOTE] > > `deploy` needs a network to attach the workload to. If none exists yet in the project, it offers to create a minimal one, with IP addresses assigned automatically, on your behalf — pass `-y` to accept that automatically in a non-interactive run. ## Attaching an ALB `--http-port` declares the port your container serves plain HTTP on, and attaches Datum's ALB to the workload. Datum terminates TLS, so don't serve HTTPS on that port. After the rollout, `deploy` prints the progress of publishing and then the ALB's hostname, on its own line, as the last line of output: ```text theme={null} Publishing... Backends 2 healthy across us-central-1, us-east-1 Edge programmed Certificate issued https://stark-raven-b7k2m.datumproxy.net ``` Redeploying without `--http-port` keeps the workload's current port and its ALB. To remove the ALB, pass `--no-http`. Without an HTTP port, the workload isn't reachable from the internet. For custom domains and publishing workloads you manage with manifests, see [Publish a workload](/docs/compute/compute-and-alb). ## Watching the rollout `deploy` waits and prints progress as Instances come up, the same view `datumctl compute rollout` shows on demand. For more information, see [Operations](/docs/datumctl/compute/scaling-and-operations). Press Control+C to detach from the watch. The rollout continues in the background. To reattach, run `datumctl compute rollout api`. ## Going back to a previous image There's no dedicated rollback command. To go back, run `datumctl compute deploy` again for the same workload with the previous image, and keep the other flags you used for the earlier deploy, since a flag-based `deploy` describes the whole workload. It updates the workload and rolls Instances over like any other change. Keep the tag or digest of each image you ship so you have something to go back to. ```bash theme={null} datumctl compute deploy api --image=registry.example.com/acme/api:1.4.1 --location=us-central-1 --min=2 ``` ## Deploying from a manifest A flag-based deploy writes a `workload.yaml` in the current directory after it succeeds — the same `Workload` resource the flags produced, expressed as YAML. Point `-f` at a manifest to deploy from it instead of flags: ```bash theme={null} datumctl compute deploy -f workload.yaml ``` This path shows a human-readable diff of what would change before touching anything, then asks for confirmation (skip it with `-y`): ```bash theme={null} datumctl compute deploy -f workload.yaml -y ``` > [!TIP] > > A manifest is the only way to reach configuration the flags don't expose — a second placement, additional ports, a custom command, or a variable set from one key of a Secret. Edit the generated `workload.yaml` (or write one from scratch) and apply it with `-f`. A manifest deploy doesn't create or remove an ALB. You can't combine `-f` with `--http-port`, `--no-http`, `--runtime-class`, `--network`, or `--build`. Set the runtime class and network in the manifest instead. > [!WARNING] > > A flag-based `deploy` replaces the workload's whole template and placements with what the flags describe. It keeps environment variables, ConfigMap and Secret imports and mounts, and labels, but it removes registry credentials, custom commands, and extra placements that you set in a manifest, and it sets the Instance count to the `--min` value, which defaults to `1`. Once you manage a workload with a manifest, update it with `datumctl compute deploy -f workload.yaml`. Each flag-based deploy also overwrites `workload.yaml` in the current directory, so keep edited manifests under another name. ## Checking workload health `datumctl compute workloads` lists every workload in the project, with per-location ready counts rolled into a single health summary: ```bash theme={null} # List everything datumctl compute workloads # Only workloads that are degraded right now datumctl compute workloads --health=degraded # Only workloads with a placement in one location datumctl compute workloads --location=us-central-1 ``` The `HEALTH` column shows `Available`, `Degraded`, `Unavailable`, or `Unknown`. The `--health` flag takes the lowercase form, such as `--health=degraded`. The `URL` column shows each workload's ALB hostname. For the full picture on a single workload — its container spec, scale settings, ALB hostname, and per-location ready/desired counts together — use `describe`: ```bash theme={null} datumctl compute workloads describe api ``` ## Next steps * `datumctl compute deploy --help` and `datumctl compute workloads --help` for the full flag reference. * [Operations](/docs/datumctl/compute/scaling-and-operations) — restart, watch a rollout, and inspect individual Instances. * [Destroying workloads](/docs/datumctl/compute/destroying-workloads) — tear a workload down when you're done with it. * [Building images](/docs/datumctl/compute/building-images) — build and publish the image you're deploying here. # Destroying workloads Source: https://datum.net/docs/datumctl/compute/destroying-workloads Delete a workload, every Instance it created, and its ALB with datumctl compute destroy. `datumctl compute destroy` removes a workload, its ALB, and every Instance that it created in every location: ```bash theme={null} datumctl compute destroy api ``` Before it deletes anything, the command prints a summary of the number of [placements](/docs/compute/placement-and-scaling), the locations, the total minimum replicas, and the ALB's hostnames, and then asks for confirmation. Custom domains attached to the ALB stop serving the workload, but Datum keeps the domain resources. To skip the prompt in scripts and CI, pass `-y` or `--yes`: ```bash theme={null} datumctl compute destroy api -y ``` > [!WARNING] > > Unlike `datumctl delete` elsewhere in the CLI, `destroy` has no `--dry-run` and there's no `diff` equivalent to preview it. The confirmation prompt is your only checkpoint — read the summary it prints, or check the workload yourself first: > > ```bash theme={null} > datumctl compute workloads describe api > ``` ## Next steps * `datumctl compute destroy --help` for the full flag reference. * [Deploying workloads](/docs/datumctl/compute/deploying-workloads) — redeploy the workload later from the same manifest or image. * [Operations](/docs/datumctl/compute/scaling-and-operations) — restart or check Instances before you decide to remove a workload altogether. # Compute commands Source: https://datum.net/docs/datumctl/compute/overview Deploy and manage containerized workloads on Datum Cloud with the datumctl compute plugin. > [!NOTE] > > Compute is in preview, and each project needs approved access before it can deploy. For product guides, see [Compute](/docs/compute/overview). To deploy your first app, see [Deploy an app](/docs/getting-started/first-workload). `datumctl compute` deploys a container image as a *workload* that runs in one or more Datum Cloud locations, and provides the day-to-day commands to scale, restart, and delete it. It's a first-party plugin instead of a built-in command. After you install it, its commands behave like the rest of the CLI. ```bash theme={null} datumctl plugin install compute datumctl compute --help ``` > [!NOTE] > > New to plugins? [Using plugins](/docs/datumctl/plugins/using-plugins) covers installation, upgrades, and how a plugin inherits your active context and credentials. Not installed `datumctl` itself yet? Start with [Install](/docs/datumctl/install). ## Core concepts A handful of terms recur across every `compute` command: | Term | Meaning | | - | - | | **Workload** | The thing you deploy — a container image plus its runtime configuration (ports, env, instance type) and one or more [placements](/docs/compute/placement-and-scaling). | | **Placement** | A named group of locations and a scale policy (minimum replica count) in a workload. A flag-based `deploy` creates a single placement named `default`. | | **Location** | A Datum Cloud region a workload can run in, for example `us-central-1`. The `--city` flag selects every location in a city by its code, such as `DFW`. | | **Instance** | One running copy of a workload's container in one location. A placement with `--min=2` across two locations produces four Instances. | `compute` is project-scoped: every command reads `--project` (or your active context's project, injected automatically — see [Contexts & scoping](/docs/datumctl/contexts-and-scoping)). There is no organization-level view and no `--namespace` flag; all compute resources live in the project's `default` namespace. ## Requesting access Compute is a gated service. The first time you run a command that needs it, `datumctl` checks whether your project is entitled to use Compute: > [card] Not requested > > No request has been made yet. On an interactive terminal, the gate offers to submit one for you. > [card] Pending approval > > A request is awaiting a manual decision by the service provider. Latency is unbounded. > [card] Active > > The project is entitled — every `compute` command runs normally. > [card] Denied / Revoked > > The provider rejected the request (or later revoked it). Recovery is submitting a new request. ### The automatic gate Run any gated command — `deploy`, `workloads`, `instances`, and so on — without access, and on a TTY `datumctl` prompts you inline: ```bash theme={null} datumctl compute deploy api --image=ghcr.io/acme/api:1.4.2 --location=us-central-1 --min=1 ``` `datumctl` checks if Compute is enabled for the project, and — if not — asks whether to request access; on confirmation, it submits the request and waits briefly to see whether the platform approves it immediately or the request needs manual review. Decline, and nothing is submitted — run `datumctl compute access request` yourself whenever you're ready. In a non-interactive shell (CI, a script, or a piped command), the gate never prompts; it fails immediately and tells you which command to run. ### Checking and requesting access explicitly Use `datumctl compute access` to check the current state directly, and `access request` for more control over submitting one than the automatic prompt gives you: ```bash theme={null} # Show the current access state datumctl compute access # Submit a request with a justification, useful for the provider's review datumctl compute access request --message="Onboarding the api service" # Submit and block until the platform's first decision (or --timeout elapses) datumctl compute access request --wait # A denied or revoked request is terminal — --renew deletes it and submits fresh datumctl compute access request --renew ``` `datumctl compute access` prints the state, the platform's explanation, and — when there's a next step — the exact command to run: ```text theme={null} Service: Compute (compute.datumapis.com) Project: acme-prod Status: Not requested This service is not enabled for this project. Request access with: datumctl compute access request ``` Add `-o json` or `-o yaml` to script against the state instead of parsing prose. ## Checking quota Once active, `datumctl compute quota` shows how much of your project's compute allotment is used: ```bash theme={null} datumctl compute quota ``` ```text theme={null} Quota for project acme-prod RESOURCE UNIT LIMIT USED AVAILABLE USAGE Workloads workloads 10 3 7 [######--------------] 30% Instances instances 50 12 38 [#####---------------] 24% vCPUs vCPUs 32 8 24 [#####---------------] 25% Memory MiB 65536 16384 49152 [#####---------------] 25% ``` Pass `--constrained` to show only the resource types that are currently at their limit — the fastest way to check whether a stalled rollout is a quota problem: ```bash theme={null} datumctl compute quota --constrained ``` ## Find your way around > [card] [Building images](/docs/datumctl/compute/building-images) > > Turn a Dockerfile into an image Compute can run, and check compatibility before you deploy. > [card] [Deploying workloads](/docs/datumctl/compute/deploying-workloads) > > Deploy a workload from flags or a manifest, and read back its health across locations. > [card] [Operations](/docs/datumctl/compute/scaling-and-operations) > > Roll restarts, watch a rollout, and debug individual Instances. > [card] [Viewing logs](/docs/datumctl/compute/viewing-logs) > > Read Instance output or ALB access logs from the terminal. > [card] [Destroying workloads](/docs/datumctl/compute/destroying-workloads) > > Tear down a workload and every Instance it created. ## Related * `datumctl compute --help` and `datumctl compute --help` for the full flag reference. * [Compute](/docs/compute/overview) — product guides for runtime classes, configuration, placement, and publishing workloads. * [Open a shell with datumctl](/docs/compute/instance-shell#datumctl) — `datumctl compute exec` opens an interactive shell or runs a single command in a running Instance. * [Using plugins](/docs/datumctl/plugins/using-plugins) — how the `compute` plugin is installed, upgraded, and trusted. * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — how the project a `compute` command runs against is resolved. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — `-o json`/`-o yaml` patterns that apply across `compute` subcommands. # Scale and operate workloads with datumctl Source: https://datum.net/docs/datumctl/compute/scaling-and-operations Scale, restart, roll out, and inspect individual Instances of a running workload with datumctl compute. After you deploy a workload, use these commands for day-to-day operation: scaling, restarting Instances, watching a rollout, and inspecting a single Instance when something isn't right. ## Scaling `datumctl compute scale` sets the minimum number of Instances per location for every placement of a workload: ```bash theme={null} # Run three Instances in each location of the workload datumctl compute scale api --min=3 ``` The count is per location, from 1 to 1,000. Workloads don't scale automatically. For different counts per placement, edit `scaleSettings.minReplicas` in the manifest and apply it with `datumctl compute deploy -f`. For more information, see [Placement and scaling](/docs/compute/placement-and-scaling). ## Restarting `datumctl compute restart` starts a rolling restart. Datum replaces Instances without changing the image or configuration. A restart is useful to pick up an external change, such as a rotated Secret: ```bash theme={null} datumctl compute restart api ``` `restart` replaces every Instance of the workload in every location. ## Watching a rollout Any change that replaces Instances, such as a `deploy` with a new image or a `restart`, starts a rollout. `datumctl compute rollout` attaches to the rollout and prints progress for each placement and location: ```bash theme={null} datumctl compute rollout api ``` Each row moves through phases — `Pending`, `Updating`, `Done` — until every placement finishes. Datum replaces Instances one at a time in each location, and waits for each replacement to be ready before it starts the next one. If a placement stalls for more than 30 seconds without progress, its phase becomes `Blocked` and the command prints the reason (a quota limit, a scheduling issue) reported by the platform. > [!NOTE] > > Press Control+C to detach from the watch. The rollout continues in the background. To reattach and see current progress, run `datumctl compute rollout api`. ## Inspecting Instances `datumctl compute instances` lists every Instance in the project. Each row is one copy of a workload's container in one location, whether it's running or trying to start: ```bash theme={null} # List every Instance in the project datumctl compute instances # List the Instances of one workload datumctl compute instances --workload=api # List the Instances in one location datumctl compute instances --location=us-central-1 ``` The status reflects availability, not only whether a process is running. For example, an Instance blocked on quota shows `Pending (quota exceeded)` instead of a generic "not ready" status. Add `-o wide` for the underlying instance type alongside the default columns, or `-o json`/`-o yaml` for the full resource. To see the runtime configuration, environment, network addresses, and a plain-language explanation of any failure for one Instance, use `describe`: ```bash theme={null} datumctl compute instances describe api-default-us-central-1-0 ``` When an Instance can't start because of quota, `describe` also prints the next commands to run, such as `datumctl compute quota`. ## Next steps * `datumctl compute scale --help`, `restart --help`, `rollout --help`, and `instances --help` for the full flag reference. * [Deploying workloads](/docs/datumctl/compute/deploying-workloads) — the `deploy` and `workloads` commands that create what you're operating on here. * [Viewing logs](/docs/datumctl/compute/viewing-logs) — read or follow Instance output and ALB access logs. * [Destroying workloads](/docs/datumctl/compute/destroying-workloads) — remove a workload and every Instance it created. # View Compute logs Source: https://datum.net/docs/datumctl/compute/viewing-logs Read and follow workload Instance output and ALB access logs with datumctl compute logs. `datumctl compute logs` reads the output of a workload's Instances. Pass a **workload name**, not an Instance name. By default, it merges the newest 200 lines from all of the workload's Instances over the last 24 hours and prefixes each line with its Instance. > [!NOTE] > > Install the Compute plugin with `datumctl plugin install compute`. If `datumctl compute logs` is an unknown command, run `datumctl plugin upgrade compute` and check `datumctl compute logs --help`. ## Read and follow Instance output Replace `WORKLOAD_NAME` with the name of your workload: ```bash theme={null} datumctl compute logs WORKLOAD_NAME datumctl compute logs WORKLOAD_NAME --follow ``` Use `--since` to read a longer or shorter window. A time window shows all matching lines unless you also set `--tail`: ```bash theme={null} datumctl compute logs WORKLOAD_NAME --since=1h datumctl compute logs WORKLOAD_NAME --since=1h --tail=100 ``` To narrow the output, use the location, Instance, or container name. `--instance` accepts a full name or the short name shown in line prefixes. `--container` applies to standard container Instances; [unikernel](/docs/compute/runtimes/unikernels) logs aren't split by container. ```bash theme={null} datumctl compute logs WORKLOAD_NAME --location=us-central-1 --since=15m datumctl compute logs WORKLOAD_NAME --instance=INSTANCE_NAME --follow datumctl compute logs WORKLOAD_NAME --container=app --search=error ``` `--search` matches text case-sensitively. For machine-readable output, use `-o json`, which prints one JSON object per line: ```bash theme={null} datumctl compute logs WORKLOAD_NAME -o json | jq .line ``` The command can also read logs left by replaced Instances or a deleted workload. To inspect an Instance's preceding generation, use `--instance=INSTANCE_NAME --previous` and widen the time window with `--since` if needed. To find the Instance name, run `datumctl compute instances --workload=WORKLOAD_NAME` (see [Scaling and operations](/docs/datumctl/compute/scaling-and-operations)), or read it from the prefix on each line of `datumctl compute logs WORKLOAD_NAME`. ## Read ALB access logs For a workload published through an ALB, use `--alb` to read requests to its public URL: ```bash theme={null} datumctl compute logs WORKLOAD_NAME --alb --since=15m datumctl compute logs WORKLOAD_NAME --alb --follow ``` ALB access logs belong to the published URL, so `--alb` can't be combined with `--instance`, `--location`, `--container`, or `--search`. If the workload has no published URL, there are no ALB access logs to read. ## Open logs in the Cloud Portal ```bash theme={null} datumctl compute logs WORKLOAD_NAME --browser ``` This opens the workload's Logs tab. Add one `--instance=INSTANCE_NAME` to open that Instance's Logs tab. For an overview of Compute metrics and logs, see [Observability](/docs/compute/observability). # Interactive console Source: https://datum.net/docs/datumctl/console Browse, inspect, and monitor Datum Cloud resources in a full-screen, keyboard-driven terminal interface. The interactive console is a full-screen terminal UI for browsing and inspecting your Datum Cloud resources. Instead of running one command at a time, you navigate resource types in a sidebar, scroll through resources in a table, and drill into details, quota, activity, and change history — all without leaving the keyboard. For why Datum built it, see [A terminal for everyone](https://www.datum.net/blog/a-terminal-for-everyone/) on the Datum blog. > [!NOTE] > > Reach for the console when you want to *explore* — to see what exists, check quota headroom, or review recent activity at a glance. Reach for plain commands (`datumctl get`, `datumctl apply`, and friends) when you want to *script*, pipe output to JSON or YAML, or make precise, repeatable changes. ## Launching the console ```bash theme={null} datumctl console ``` The console opens against your active context (the organization and project resolved from your saved configuration). Use `datumctl ctx` beforehand if you want to start somewhere specific — you can also switch context from inside the console at any time. See [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how that scope is chosen. If you are not logged in, the console opens on a welcome screen and walks you through authentication in place (see [Logging in from the console](#logging-in-from-the-console)). ### Read-only mode ```bash theme={null} datumctl console --read-only ``` The `--read-only` flag disables mutation operations, which makes it safe for demos, screen-sharing, or automated testing where you want to browse without any risk of changing resources. ## The layout The console is organized into a set of panes. You move between them with a few keys and drill deeper as you go. > [card] Navigation sidebar > > The left column lists the resource types available in your context. Move the cursor with `j`/`k` (or the arrow keys) and press `Enter` to load a type. > [card] Resource list > > The main table shows the resources of the selected type, with a live count in the sidebar. It refreshes automatically in the background. > [card] Detail view > > A full description of a single resource, with toggles for raw YAML, status conditions, and events. > [card] Dashboards > > Dedicated quota and activity dashboards summarize allowance usage and recent changes across the project. ### The welcome panel When a project context is active, the console opens on a welcome panel that greets you with quick-jump shortcuts, a teaser of recent project activity, and a summary of quota health. From here you can jump straight to a common resource type by pressing a single letter — for example `p` for projects, `w` for workloads, `g` for gateways, `b` for backends, `z` for DNS zones and record sets, or `n` for namespaces. A quick-jump key only fires when a matching resource type exists in your current context. Press `Esc` from a quick-jumped list to return to the welcome panel in a single step. ## Navigating Most navigation is the same everywhere: move with `j`/`k` or the arrow keys, go deeper with `Enter`, and step back with `Esc`. | Key | Action | | - | - | | `j` / `k` (or `↓` / `↑`) | Move the cursor down / up | | `Tab` / `Shift+Tab` | Switch focus between the sidebar and the resource list | | `Enter` | Select — load a resource type, or open the detail view for the highlighted resource | | `Esc` | Step back one level (detail → list → welcome panel) | | `/` | Filter the current resource list | | `r` | Refresh the current view | | `c` | Switch organization / project context | | `?` | Toggle the full keybind reference | | `q` | Quit (`Ctrl+C` force-quits) | > [!NOTE] > > Press `?` at any time for the built-in keybind reference. The overlay adapts to where you are, so it shows the actions available in the current pane. ## Inspecting a resource Select a resource type in the sidebar, press `Enter` to load its list, highlight a row, and press `Enter` (or `d`) to open the detail view. The detail view describes the resource and offers several ways to look at it: | Key | View | | - | - | | `y` | Toggle raw YAML | | `C` | Toggle the status conditions table | | `E` | Toggle the events table | | `A` | Open the per-resource activity log | | `H` | Open the resource's change history | These views are mutually exclusive toggles — pressing the same key again returns you to the standard description. `Esc` closes the detail view and returns you to the list. ### Change history and diffs From the detail view, press `H` to open the change history for a resource — a list of past revisions. Highlight a revision and press `Enter` to open a diff. In the diff view, use `[` and `]` to step to the previous or next revision. `Esc` walks back out through the history to the detail view. ## Quota dashboard The quota dashboard turns allowance buckets into a readable summary of how much of each quota you are consuming. | Key | Action | | - | - | | `3` | Open the quota dashboard from anywhere (press again to return) | | `s` | Toggle grouping of buckets | | `t` | Toggle between the dashboard and the raw allowance-bucket table | | `r` | Refresh quota data | The console also surfaces a compact quota banner above the resource list when the type you are viewing has associated allowances, so you see pressure without opening the dashboard. If quota is still loading when you press `3`, the console queues the request and opens the dashboard as soon as the data is ready. ## Activity dashboard Press `4` to open the activity dashboard — a rollup of recent human activity across the current project. Press `4` again (or `Esc`) to return to where you were, and `r` to refresh the rollup. For querying the same activity data from the command line — with filters, exports, and machine-readable output — see [Querying activity](/docs/datumctl/activity/overview). > [!NOTE] > > The activity dashboard is project-scoped. When your context is an organization without a selected project, the dashboard tells you to select a project first. ## AI assistant The console no longer has a built-in chat pane. To ask questions about your resources in natural language, run [`datumctl assistant`](/docs/datumctl/ai/assistant) in another terminal. ## Logging in from the console If you launch the console without an active session, it opens on a welcome screen. Press `l` to start authentication without leaving the terminal: **1. Start the device flow** Press `l`. The console contacts your auth endpoint and displays a verification URL and a one-time code. **2. Authorize in your browser** Open the URL and enter the code, or press `b` to open the URL in your default browser automatically. **3. Pick your context** Once authentication succeeds, the console opens the context picker so you can choose the organization and project to work in, then loads your resources. The console logs in against the same auth endpoint your active session last used, so if you previously authenticated against a custom environment (via `--hostname`), the in-console flow uses that endpoint too. If a login fails, press `l` to try again or `Esc` to dismiss. ## Console vs. plain commands > [tab] Use the console when > > * You want to browse what exists and how resources relate. > * You are checking quota headroom or reviewing recent activity. > * You want to inspect a resource's description, conditions, events, or change history interactively. > * You are exploring a new project and prefer a navigable view over remembering exact resource names. > [tab] Use plain commands when > > * You are scripting or running in CI, where a non-interactive command is required. > * You need machine-readable output to pipe elsewhere (`-o json`, `-o yaml`). > * You are making precise, repeatable changes with `datumctl apply`, `create`, or `delete`. > * You want to diff a local manifest against the server before writing. ## Next steps * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — set your working organization and project before launching the console. * [Logging in](/docs/datumctl/auth/logging-in) — learn how authentication and stored credentials work. * [Reading resources](/docs/datumctl/resources/reading) — the `get` and `describe` commands behind the console's list and detail views. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — machine-readable output and automation for everything you browse here. * [AI assistant](/docs/datumctl/ai/assistant) — chat with Patch, the Datum Cloud assistant, from the terminal. # Contexts & scoping Source: https://datum.net/docs/datumctl/contexts-and-scoping How datumctl decides which organization or project a command acts on, and how to switch between them. Almost every `datumctl` command runs against a single **scope** — either one organization or one project inside that organization. Understanding how `datumctl` chooses that scope is the key to getting the results you expect. If a command returns nothing or the wrong resources, the cause is almost always the active scope. > [!NOTE] > > In short: `datumctl` remembers an **active context** (an org or a project). Any command uses that context unless you override it for a single command with `--organization` or `--project`. ## The scope model Datum Cloud is organized as a two-level hierarchy: * **Organizations** are the top level. You belong to one or more organizations. * **Projects** live inside an organization. Most day-to-day resources (DNS zones, networking, and so on) belong to a project. A **context** captures one point in that hierarchy: * An **org context** points at an organization (for example, resources that live at the organization level). * A **project context** points at a specific project inside an organization. `datumctl` discovers the organizations and projects you can access from the API and stores them locally as a list of contexts. One of them is the **active context** — the scope commands use by default. ## Listing your contexts Run `datumctl ctx` (alias: `datumctl context`) to see every context available to your active user: ```bash theme={null} datumctl ctx ``` This prints a table with the display name, the reference you use to select the context, the type (`org` or `project`), and a `*` marking the current active context. Projects are nested under their organization: ```text theme={null} Display Name Name Type Current Datum Technology, Inc datum org Datum Cloud datum/datum-cloud project * Staging datum/staging project ``` The **Name** column is the reference you pass to `datumctl ctx use`: a bare organization ID for an org context (`datum`), or `org/project` for a project context (`datum/datum-cloud`). > [!NOTE] > > `datumctl ctx list` (alias `datumctl ctx ls`) prints the same table. Running `datumctl ctx` with no subcommand is the shortcut most people use. ### Keeping the list current `datumctl ctx` always fetches your contexts live from the API, so an organization or project you were just given access to appears right away. There's nothing to refresh. To list the contexts of every account you're signed in with, grouped by account, add `--all`: ```bash theme={null} datumctl ctx --all ``` ## Switching the active context Use `datumctl ctx use` to change which context is active. You can name the context directly, or run it with no argument to pick from an interactive list: > [tab] Interactive picker > > ```bash theme={null} > datumctl ctx use > ``` > > Choose an organization or project from the list. This is the easiest way when you do not remember the exact name. > [tab] By name > > ```bash theme={null} > # Switch to a project context > datumctl ctx use datum/datum-cloud > > # Switch to an org context > datumctl ctx use datum > ``` You can identify a context by its resource ID (`datum/datum-cloud`) or by its display name (for example, the human-friendly project name shown in the table). Resource IDs always take precedence, and a display name is only accepted when it matches exactly one context. If a name is ambiguous or not found, `datumctl` tells you and suggests running `datumctl ctx` to see the available choices. On success you'll see a confirmation of the new scope: ```text theme={null} ✓ Switched to project Datum Cloud in Datum Technology, Inc (datum/datum-cloud) ``` The active context persists across commands and terminal sessions until you switch again. ## How scope is resolved for each command When you run a command, `datumctl` determines its scope by checking these sources **in order** and using the first one that applies: **1. Per-command flags** `--organization ` or `--project ` on the command line. These override everything else, but only for that single command. **2. Environment variables** `DATUM_PROJECT` or `DATUM_ORGANIZATION`, if set. Useful in CI or scripts. You cannot set both at once. **3. The active context** Whatever `datumctl ctx use` last selected. If the active context is a project, commands run at project scope; if it is an org context, they run at organization scope. If none of these provide a scope, `datumctl` falls back to your personal user scope. > [!WARNING] > > `--project`, `--organization`, and `--platform-wide` are mutually exclusive — you can supply at most one. Likewise, `DATUM_PROJECT` and `DATUM_ORGANIZATION` cannot both be set. `datumctl` returns an error rather than guessing when the inputs conflict. ### Overriding scope for a single command The flags let you target a different org or project without changing your active context: ```bash theme={null} # Run this one command against a different project datumctl get dnszones --project other-project # Run against an organization datumctl get projects --organization datum ``` The next command with no flag goes back to using your active context. > [!NOTE] > > For advanced access to the platform root rather than a specific project or organization, an opt-in `--platform-wide` flag is also available. Most users never need it. ## When a command returns nothing An empty result is almost always a scope mismatch, not an error. The resource you're looking for probably lives in a different project or organization than the one your command targeted. Work through this checklist: **1. Confirm the active context** Run `datumctl ctx` and look for the `*`. Make sure it points at the org or project that contains your resource. **2. Check for an override you forgot about** An exported `DATUM_PROJECT` or `DATUM_ORGANIZATION` environment variable silently redirects every command. Unset it if it isn't what you want: ```bash theme={null} unset DATUM_PROJECT DATUM_ORGANIZATION ``` **3. Target the right scope explicitly** Re-run the command with an explicit flag to rule out the active context: ```bash theme={null} datumctl get dnszones --project your-project-id ``` **4. Check your account if the context is missing** `datumctl ctx` lists contexts live, so if a project or organization you expect isn't listed, your active account doesn't have access to it. Check which account is active with `datumctl whoami`, and list the contexts of every signed-in account with `datumctl ctx --all`. > [!NOTE] > > Remember that a project's resources aren't visible from its parent org context (and vice versa). Switching to the correct project scope with `datumctl ctx use org/project` resolves most "nothing found" surprises. ## Contexts and users Contexts are tied to the user you're authenticated as. Each stored user session has its own set of discovered contexts. When you switch users, your active context follows that user's most recently used context. To log in, see [Logging in](/docs/datumctl/auth/logging-in); to switch between stored accounts, see [Managing accounts & tokens](/docs/datumctl/auth/managing-accounts). ## Related * [Logging in](/docs/datumctl/auth/logging-in) — sign in, verify your identity with `whoami`, and sign out. * [Managing accounts & tokens](/docs/datumctl/auth/managing-accounts) — switch the active user and how each session maps to a context. * [Reading resources](/docs/datumctl/resources/reading) — the `get` and `describe` commands that run against the scope you set here. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — pinning `--organization` and `--project` explicitly and reading `DATUM_PROJECT`/`DATUM_ORGANIZATION` in CI. * [Overview](/docs/datumctl/overview) — how `datumctl` is structured and talks to the Datum API. * `datumctl ctx --help` for the context commands and the `--organization`/`--project` flags. # Discovering resources & schemas Source: https://datum.net/docs/datumctl/discovering-resources Find which Datum Cloud resource types exist and inspect their fields before you author manifests. Before you can create or update a resource, you need to know two things: which resource types Datum Cloud offers, and what fields each type accepts. `datumctl` answers both questions directly from the API server, so what you see always reflects the exact version of the platform you are connected to. Three commands cover discovery: | Command | Answers | | - | - | | `datumctl api-resources` | Which resource types can I manage? | | `datumctl api-versions` | Which API group/version pairs are available? | | `datumctl explain ` | What fields does this type have, and what do they mean? | > [!NOTE] > > These commands read from the control plane for your **currently configured > context**. Make sure you are logged in (see [Logging in](/docs/datumctl/auth/logging-in)) > and have an organization or project selected first — pass `--organization your-org-id` > or `--project your-project-id`, or set a default context. See > [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how that scope is chosen. ## List available resource types `datumctl api-resources` prints a table of every resource type the current control plane supports. It is the starting point for discovering what you can manage with `datumctl get`, `datumctl apply`, and `datumctl explain`. The table includes each type's short names, API group, whether it is namespaced, and its kind. ```bash theme={null} # List all available resource types datumctl api-resources ``` The list is fetched fresh from the server on each run. To reuse a locally cached copy, pass `--cached`. ### Narrow and sort the list For a busy control plane, filter and reshape the output to find what you need: ```bash theme={null} # Show extra detail, including verbs and short names datumctl api-resources -o wide # Sort by name datumctl api-resources --sort-by=name # Only namespaced resource types datumctl api-resources --namespaced=true # Only resource types in a specific API group datumctl api-resources --api-group=networking.datumapis.com ``` > [!NOTE] > > The short names shown here are the same aliases you can use anywhere a > resource type is expected — for example in `datumctl get`, `datumctl explain`, > or `datumctl describe`. ## List API versions `datumctl api-versions` prints every API group/version combination the current control plane exposes, one per line in the form `group/version` (for example, `networking.datumapis.com/v1alpha`). ```bash theme={null} # List all API group/version pairs datumctl api-versions ``` This is the value you place in the `apiVersion` field of a manifest. Use `api-versions` when you need the group/version string, and `api-resources` when you also want to see the individual resource types within each group. ## Inspect a type's schema Once you know a type exists, `datumctl explain` shows its schema and field-level documentation. Information is retrieved from the API server in OpenAPI format, so it always matches the platform version you are connected to. ```bash theme={null} # Show the schema for the Project resource type datumctl explain projects ``` Fields are referenced with dot notation — `TYPE.fieldName.subFieldName` — so you can drill into any part of the tree: ```bash theme={null} # Show documentation for just the spec field datumctl explain projects.spec ``` To see the whole schema tree at once instead of one level at a time, use `--recursive`: ```bash theme={null} # Expand every field recursively datumctl explain projects --recursive ``` If you prefer the OpenAPI v2 rendering, request it explicitly: ```bash theme={null} # Use the OpenAPI v2 output format datumctl explain projects --output=plaintext-openapiv2 ``` ## From discovery to a manifest Discovery feeds directly into authoring manifests for `datumctl apply` and `datumctl create`. A typical flow: **1. Find the type** Run `datumctl api-resources` to confirm the resource type exists and note its kind and API group. ```bash theme={null} datumctl api-resources --api-group=networking.datumapis.com ``` **2. Get the apiVersion** Run `datumctl api-versions` to find the exact `group/version` string for that group. ```bash theme={null} datumctl api-versions ``` **3. Learn the fields** Run `datumctl explain ` (add `--recursive` for the full tree) to see which fields are required and what they do. ```bash theme={null} datumctl explain projects --recursive ``` **4. Write and validate the manifest** Author your YAML with the `apiVersion`, `kind`, and `spec` fields you discovered, then validate it against the server before persisting. ```bash theme={null} datumctl apply -f ./project.yaml --organization your-org-id --dry-run=server ``` > [!TIP] > > `datumctl explain .spec` is the fastest way to see exactly which fields > belong under `spec` when you are filling out a new manifest. ## Search across your resources When you know roughly what you're looking for but not its kind or exact name, use the `search` plugin. It searches your resources by name and kind, and returns the best matches first. To install it, run `datumctl plugin install search`. ```text theme={null} $ datumctl search api KIND NAME NAMESPACE TENANT SCORE AGE Workload api default your-project-id 0.99 3d HTTPProxy api default your-project-id 0.97 3d Workload api-postgres default your-project-id 0.82 3d ``` To narrow the results to one or more kinds, add `--kind`, such as `--kind Workload --kind HTTPProxy`. To see which kinds are searchable, run `datumctl search kinds`. For scripts, add `-o json`. ## Check which services your project can use Some Datum services, such as Compute, must be enabled for a project before you can create their resources. To see every service and whether it's enabled for your current project, run the following command: ```bash theme={null} datumctl services list ``` The state of each service is `Active`, pending approval, `NotRequested`, or denied. To request a service, run `datumctl services enable SERVICE_NAME`. Some services need approval from Datum, so add `--wait` to wait for the answer, or check back later with `datumctl services status SERVICE_NAME`. To explain why you need a service, add `--message`. ## Related * [Changing resources](/docs/datumctl/resources/changing) — author, apply, and edit resources once you know which types exist and what fields they take. * [Reading resources](/docs/datumctl/resources/reading) — list, retrieve, and inspect resources of the types you discover here. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — render this discovery output as JSON or YAML and pipe it into other tooling. * [Logging in](/docs/datumctl/auth/logging-in) — sign in before querying the control plane. * `datumctl api-resources --help`, `datumctl api-versions --help`, and `datumctl explain --help` for the full flag reference. # GitOps with Flux & Argo CD Source: https://datum.net/docs/datumctl/gitops Run a git-driven deployment flow onto Datum Cloud by pointing your GitOps tooling at a control plane through datumctl's exec-credential kubeconfig. > [!WARNING] > > This guide is for teams **bringing existing Kubernetes tooling** — Flux, Argo > CD, and their CLIs. Datum Cloud itself requires no Kubernetes knowledge: you > manage organizations, projects, DNS zones, and other resources directly with > `datumctl get`, `datumctl apply`, `datumctl delete`, and friends. If you are > not already running a GitOps controller, see the simpler > [datumctl-native loop](#a-simpler-loop-datumctl-in-ci) below. [Flux](https://fluxcd.io/) and [Argo CD](https://argo-cd.readthedocs.io/) let you keep your desired state in git and continuously reconcile it against a control plane. Because `datumctl auth update-kubeconfig` produces a standard [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) whose credential comes from `datumctl` itself, these tools can reconcile git-defined resources onto a Datum Cloud control plane without you managing any static keys. ## The supported topology The key detail is *how* authentication works. `datumctl auth update-kubeconfig` writes a kubeconfig whose user is a [client-go exec credential plugin](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins): on each request the tooling invokes the `datumctl` binary to fetch a fresh, short-lived token. Any tool that reads a kubeconfig and uses client-go authentication can therefore talk to the Datum Cloud control plane — **as long as the environment running that tool has the `datumctl` binary available and an active session.** See [Using kubectl](/docs/datumctl/kubectl-interop) for the full mechanics; this page does not repeat them. That constraint defines the supported shape for GitOps: * Run your GitOps tooling (Flux, Argo CD, and their CLIs) from a **CI runner or machine that has `datumctl` installed** and is authenticated. * In CI, authenticate with a [service account](/docs/datumctl/auth/service-accounts) so there is no interactive login step and no long-lived keys checked into your repo. * Point the tooling at the control plane through the exec-credential kubeconfig that `datumctl` generates, then let it reconcile the resources defined in your git repository. > [!WARNING] > > Authentication runs the `datumctl` binary as an exec plugin, so the > environment executing the GitOps tool must have `datumctl` installed **and** an > active session (a service-account session in CI). A controller running > somewhere else — for example, inside an unrelated Kubernetes cluster with no > `datumctl` binary and no Datum Cloud session — is **not** covered by this > flow. Frame Flux and Argo CD usage as "run from CI (or a machine) that has > `datumctl` plus a service-account session." ## The shape of the flow 1. **Set up `datumctl`** on the runner or machine and authenticate. In CI, this is a service account; see [Automating in CI/CD](/docs/datumctl/cicd/automating). 2. **Generate a kubeconfig** for the target control plane with `datumctl auth update-kubeconfig` — see [Using kubectl](/docs/datumctl/kubectl-interop). Use `--exec-interactive-mode Never` so the exec plugin fails fast instead of waiting on a prompt no one can answer. 3. **Run the GitOps step.** Flux, Argo CD, and their CLIs read that kubeconfig and authenticate to the Datum Cloud control plane through the exec plugin, reconciling the resources your git repository declares. Which control plane you target is set by the `--project` or `--organization` flag on `update-kubeconfig`; see [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how scope resolution works. ## A concrete CI example The `datum-cloud/setup-datumctl-action` installs `datumctl` and authenticates a service account on a GitHub Actions runner. From there you generate a kubeconfig and hand off to your GitOps tool. See [GitHub Actions](/docs/datumctl/cicd/github-actions) for the full action reference. ```yaml theme={null} jobs: reconcile: runs-on: ubuntu-latest steps: # Installs datumctl and signs in a service account - uses: datum-cloud/setup-datumctl-action@v1 with: credentials: ${{ secrets.DATUM_SA_CREDENTIALS }} # Write a kubeconfig for the target project's control plane. # --exec-interactive-mode Never keeps the exec plugin non-interactive. - name: Configure control plane access run: | datumctl auth update-kubeconfig \ --project your-project-id \ --exec-interactive-mode Never # Run your GitOps step. The tool reads the kubeconfig above and # authenticates through the datumctl exec plugin. - name: Reconcile from git run: | # e.g. invoke your Flux or Argo CD CLI here — see their docs # for the exact commands for your setup. ... ``` > [!NOTE] > > The exact `flux` / `argocd` CLI commands depend on how you have structured > your repository and your reconciliation model. Consult the > [Flux](https://fluxcd.io/) and [Argo CD](https://argo-cd.readthedocs.io/) > documentation for those specifics. What matters here is the connection > pattern: both read a kubeconfig, and the kubeconfig `datumctl` writes carries > the credential automatically. Nothing above pins static keys. ## A simpler loop: datumctl in CI You do not need a GitOps controller to get a git-driven deployment flow. If your resources live in git, a CI job can treat the repository as the source of truth and reconcile on every change using `datumctl` directly: * Preview what would change with `datumctl diff -f `. * Apply the desired state with `datumctl apply -f `. This keeps the whole loop inside `datumctl` — no exec-credential kubeconfig and no second tool to install — and it runs on the same CI runner with the same service-account session. See [Safe changes & declarative config](/docs/datumctl/resources/safe-changes) for the diff-then-apply workflow and [Automating in CI/CD](/docs/datumctl/cicd/automating) for wiring it into a pipeline. ## Related * [Using kubectl](/docs/datumctl/kubectl-interop) — how the exec-credential kubeconfig is generated and how the credential handoff works. * [Kubernetes tools](/docs/datumctl/kubernetes-tools) — point Helm, the Terraform Kubernetes provider, k9s, and other kubeconfig-driven tools at Datum Cloud. * [Service accounts](/docs/datumctl/auth/service-accounts) — non-interactive identities for CI and automation. * [GitHub Actions](/docs/datumctl/cicd/github-actions) — the `setup-datumctl-action` reference. * [Automating in CI/CD](/docs/datumctl/cicd/automating) — general patterns for running `datumctl` in pipelines. * [Safe changes & declarative config](/docs/datumctl/resources/safe-changes) — the `diff`-then-`apply` workflow for the datumctl-native loop. # Install Source: https://datum.net/docs/datumctl/install Install datumctl on macOS, Linux, or Windows, log in to Datum Cloud, choose an organization or project, and run your first command. This guide takes you from nothing installed to running your first command against Datum Cloud. Follow the steps in order; it takes a few minutes. > [!NOTE] > > Prefer a one-click download? Download a pre-built binary from the [GitHub Releases page](https://github.com/datum-cloud/datumctl/releases/latest). **1. Install datumctl** Choose the method that fits your platform. > [tab] Homebrew (macOS) > > If you use [Homebrew](https://brew.sh/), install `datumctl` from the official Datum Cloud tap: > > ```bash theme={null} > # Tap the Datum Cloud formula repository (only needs to be done once) > brew tap datum-cloud/homebrew-tap > > # Trust the datumctl formula > brew trust --formula datum-cloud/homebrew-tap/datumctl > > # Install datumctl > brew install datumctl > ``` > > Later, upgrade with `brew upgrade datumctl`. > [tab] Pre-built binaries > > Download a pre-built binary from the [GitHub Releases page](https://github.com/datum-cloud/datumctl/releases/latest): > > 1. Find the archive for your OS and architecture (for example `datumctl_Darwin_arm64.tar.gz`, `datumctl_Linux_x86_64.tar.gz`, or `datumctl_Windows_amd64.zip`). > 2. Download and extract it. > 3. Move the `datumctl` (or `datumctl.exe`) binary onto your `PATH` — for example `/usr/local/bin` on Linux/macOS. > 4. On Linux/macOS, make sure it is executable: `chmod +x /path/to/datumctl`. > [tab] Nix > > If you use [Nix](https://nixos.org/), install `datumctl` from its official flake: > > ```bash theme={null} > nix profile install github:datum-cloud/datumctl > ``` > > For a one-off invocation without adding `datumctl` to your profile, use `nix run github:datum-cloud/datumctl` instead. > [tab] Windows > > Download the latest Windows release from the [GitHub Releases page](https://github.com/datum-cloud/datumctl/releases/latest), extract it, and move `datumctl.exe` to a directory such as `C:\\Tools\\datumctl\\`. > > Add that directory to your `PATH`, then open a new terminal. > [tab] Build from source > > Install [Go](https://go.dev/dl/) 1.21 or later and Git, then clone and build `datumctl`: > > ```bash theme={null} > git clone https://github.com/datum-cloud/datumctl.git > cd datumctl > go build -o datumctl . > ``` > > Move the resulting `datumctl` binary onto your `PATH`. Verify the install: ```bash theme={null} datumctl version ``` This prints the installed client version. **2. Log in** Authenticate with Datum Cloud. `datumctl` uses [OAuth 2.0](https://oauth.net/2/) with [PKCE](https://oauth.net/2/pkce/) and stores your credentials securely in your system keyring — there are no API keys to manage. ```bash theme={null} datumctl login ``` This opens your browser to complete sign-in, then discovers the organizations and projects you can access. For the full sign-in flow — the browser and device-code flows, `whoami`, and signing out — see [Logging in](/docs/datumctl/auth/logging-in); for juggling several accounts see [Managing accounts & tokens](/docs/datumctl/auth/managing-accounts). > [!NOTE] > > On a headless machine you sit at, such as an SSH session or a container with no browser, add `--no-browser` to authenticate with a device-code flow instead. For unattended jobs such as CI, use a [service account](/docs/datumctl/auth/service-accounts) and `datumctl login --credentials` instead, because the device-code flow needs a person to approve it: > > ```bash theme={null} > datumctl login --no-browser > ``` > > To sign in against a custom environment — such as a self-hosted Datum Cloud deployment — point at its auth server with `--hostname`, for example `datumctl login --hostname auth.example.com`. **3. Select your working context** A context is the organization or project that commands target by default, so you do not have to pass `--organization` or `--project` every time. For the full picture of how scope is resolved, read [Contexts & scoping](/docs/datumctl/contexts-and-scoping). Right after login, `datumctl` shows the organizations and projects you can access and prompts you to pick one interactively. To see your contexts again at any time: ```bash theme={null} datumctl ctx ``` To switch contexts — either interactively or by name using the `org` or `org/project` format: ```bash theme={null} # Interactive picker datumctl ctx use # Switch to a specific project context datumctl ctx use your-org-id/your-project-id ``` **4. Run your first command** List the organizations you belong to. This command works without a context selected: ```bash theme={null} datumctl get organizations ``` Once a context is set, list resources in it — for example the projects in your organization: ```bash theme={null} datumctl get projects --organization your-org-id ``` Results are shown as a table by default. Add `-o json` or `-o yaml` for scripting or inspection (see [Output formats & scripting](/docs/datumctl/output-and-scripting)), and use `datumctl api-resources` to [discover every resource type you can manage](/docs/datumctl/discovering-resources). ## Keep datumctl up to date To upgrade to the latest release, run the following command: ```bash theme={null} datumctl auto-update ``` To upgrade automatically, run `datumctl auto-update enable`. When a newer release is available, `datumctl` then upgrades itself the next time you run any command, and re-runs that command on the new version. To go back to a warning only, run `datumctl auto-update disable`, and to see the current setting, run `datumctl auto-update status`. If you installed `datumctl` with a package manager such as Homebrew, you can upgrade it there instead. Plugins update separately. To upgrade every plugin you have installed, run `datumctl plugin upgrade --all`. ## Get help from the CLI `datumctl` documents itself. These docs explain how to use it; the built-in help is the full, current list of commands and flags for the version you have installed. To list every command, grouped by area (Authentication, Context, Resource Management, and more), run the following command: ```bash theme={null} datumctl --help ``` To see a command's flags and examples, add `--help` after it. For example: ```bash theme={null} datumctl get --help datumctl login --help ``` Commands that come from plugins work the same way. After you [install a plugin](/docs/datumctl/plugins/using-plugins), run its help to see what it adds: ```bash theme={null} datumctl compute --help ``` To check which version you are running, run `datumctl version`. ## Next steps You are now installed, authenticated, and pointed at a context. From here you can explore what else the CLI can do. > [card] [Contexts & scoping](/docs/datumctl/contexts-and-scoping) > > Understand how commands choose an organization or project, and how to switch between them. > [card] [Reading resources](/docs/datumctl/resources/reading) > > List, retrieve, and inspect Datum Cloud resources with get and describe. > [card] [Discovering resources & schemas](/docs/datumctl/discovering-resources) > > Find which resource types exist and inspect their fields before you author manifests. > [card] [Output formats & scripting](/docs/datumctl/output-and-scripting) > > Machine-readable output, structured errors, and wrapping datumctl in CI and agents. > [card] [Interactive console](/docs/datumctl/console) > > Browse and inspect resources in a full-screen, keyboard-driven UI. > [card] [Using plugins](/docs/datumctl/plugins/using-plugins) > > Extend datumctl with additional commands from a plugin catalog. > [card] [Managing accounts](/docs/datumctl/auth/managing-accounts) > > Manage multiple accounts, switch users, sign out, and print access tokens. > [card] [Overview](/docs/datumctl/overview) > > Understand how datumctl is structured and how it talks to Datum Cloud. # Using datumctl with kubectl Source: https://datum.net/docs/datumctl/kubectl-interop Point kubectl at a Datum Cloud control plane using your active datumctl session — an opt-in escape hatch for existing kubectl users. > [!WARNING] > > This guide is for **kubectl users only**. Datum Cloud requires no Kubernetes > knowledge — you manage organizations, projects, DNS zones, and other > resources directly with `datumctl get`, `datumctl apply`, `datumctl delete`, > and friends. If you have never used kubectl, you can skip this page entirely. Some teams already run [kubectl](https://kubernetes.io/docs/reference/kubectl/) and want to reuse it against Datum Cloud. The commands on this page are an **opt-in escape hatch**, not the primary path. Everything you can do here you can also do with native `datumctl` commands, and we recommend the native commands for day-to-day work. If you do use kubectl, `datumctl` acts as the credential source so you never manage static keys. > [!NOTE] > > Before you start, log in with the primary command so `datumctl` has an active > session to hand to kubectl: > > ```text theme={null} > datumctl login > ``` > > For CI or headless environments, add `--no-browser`. For a self-hosted or > otherwise custom environment, add `--hostname auth.example.com`. ## Generate a kubeconfig entry `datumctl auth update-kubeconfig` adds or updates a cluster, user, and context entry in your [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) so kubectl can authenticate to a Datum Cloud control plane using your active `datumctl` session. After it runs, kubectl automatically calls `datumctl auth get-token` to obtain a fresh credential on each request — there is nothing to rotate by hand. You must specify exactly **one** of `--organization` or `--project`. The two flags are mutually exclusive. > [tab] Organization control plane > > ```bash theme={null} > datumctl auth update-kubeconfig --organization your-org-id > ``` > [tab] Project control plane > > ```bash theme={null} > datumctl auth update-kubeconfig --project your-project-id > ``` On success, `datumctl` prints the kubeconfig path it wrote to, the user it configured, and the API server it targeted. It also sets the new entry as your current context. ### Flags | Flag | Description | | - | - | | `--organization ` | Configure access to an organization's control plane. Mutually exclusive with `--project`. | | `--project ` | Configure access to a specific project's control plane. Mutually exclusive with `--organization`. | | `--kubeconfig ` | Path to the kubeconfig file to update. | | `--hostname ` | Override the API server hostname. Useful for self-hosted environments where the hostname cannot be derived from stored credentials. | | `--exec-interactive-mode ` | Interactive mode for the kubeconfig exec credential plugin. One of `Never`, `IfAvailable` (default), or `Always`. | ### Choosing where the kubeconfig is written By default, `datumctl` updates `$HOME/.kube/config`. If the `KUBECONFIG` environment variable is set, that path is used instead. Pass `--kubeconfig` to write to an explicit file — handy for keeping Datum Cloud entries separate from your other clusters. ```bash theme={null} datumctl auth update-kubeconfig --organization your-org-id --kubeconfig ~/.kube/datum-config ``` ### Non-interactive use in CI and scripts The generated kubeconfig entry runs `datumctl` as a [client-go exec credential plugin](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins). In an interactive shell it can prompt you (for example, to complete a login) when a credential is needed. In CI or other non-interactive contexts, set the exec plugin's interactive mode to `Never` so kubectl fails fast instead of waiting on a prompt that can never be answered. ```bash theme={null} datumctl auth update-kubeconfig --project your-project-id --exec-interactive-mode Never ``` > [!NOTE] > > The default is `IfAvailable`, which uses interactive prompts only when a > terminal is attached. `Always` requires an interactive terminal. Use `Never` > for automation. ## Confirm your identity Once a control plane context is configured, `datumctl auth whoami` queries the Datum Cloud API server and shows the user identity and group memberships it resolved from your current credentials. It is useful for confirming which account kubectl is using and for troubleshooting access-denied errors. ```bash theme={null} datumctl auth whoami # Show the resolved identity as JSON datumctl auth whoami -o json ``` > [!NOTE] > > `datumctl auth whoami` requires a control plane context configured with > `datumctl auth update-kubeconfig`. To see which `datumctl` account is > currently active without any control plane context, use `datumctl auth list` > instead. ## Check permissions `datumctl auth can-i` verifies whether the active user is allowed to perform a specific action against a Datum Cloud resource type on the configured control plane. Provide an API verb (`get`, `list`, `watch`, `create`, `update`, `patch`, `delete`, or `*`) and a resource type. ```bash theme={null} # Check if you can list projects on the control plane datumctl auth can-i list projects # Check if you can create DNS zones datumctl auth can-i create dnszones # List all your permitted actions datumctl auth can-i --list # List permitted actions in a specific namespace datumctl auth can-i --list --namespace default ``` > [!TIP] > > Run `datumctl api-resources` to see the resource types you can name in a > `can-i` check — [Discovering resources & schemas](/docs/datumctl/discovering-resources) > covers resource discovery in full. ## How the credential handoff works **1. You configure the entry** `datumctl auth update-kubeconfig` writes a context whose user runs `datumctl auth get-token` as an exec credential plugin. **2. kubectl requests a token** On each request, kubectl invokes `datumctl auth get-token`, which returns a short-lived credential for the API server. **3. datumctl refreshes automatically** If the stored token has expired, `datumctl` uses your refresh token to get a new one before returning it — no static keys, no manual rotation. Because the kubeconfig entry pins to the `datumctl` session that was active when you generated it, switching accounts with `datumctl auth switch` and regenerating the entry keeps kubectl aligned with the account you intend to use. ## Related * [Kubernetes tools](/docs/datumctl/kubernetes-tools) — point Helm, the Terraform Kubernetes provider, k9s, and other kubeconfig-driven tools at Datum Cloud. * [GitOps with Flux & Argo CD](/docs/datumctl/gitops) — run a git-driven deployment loop onto a Datum Cloud control plane through the same exec-credential kubeconfig. * [Managing accounts & tokens](/docs/datumctl/auth/managing-accounts) — the native, no-Kubernetes-required workflow for switching users, managing sessions, and the `auth get-token` credential helper kubectl relies on. * [Logging in](/docs/datumctl/auth/logging-in) — sign in with the browser or device-code flow. * [Reading resources](/docs/datumctl/resources/reading) — the native `get` and `describe` commands that replace kubectl for day-to-day work. * [Output formats & scripting](/docs/datumctl/output-and-scripting) — retrieving tokens and machine-readable output for scripting and credential-helper use. # Kubernetes tools Source: https://datum.net/docs/datumctl/kubernetes-tools Point Helm, the Terraform Kubernetes provider, k9s, and other kubeconfig-driven tools at a Datum Cloud control plane using your active datumctl session. > [!WARNING] > > This guide is for teams that **already run Kubernetes tooling** and want to > reuse it against Datum Cloud. Datum Cloud itself requires no Kubernetes > knowledge — you manage organizations, projects, DNS zones, and other > resources directly with `datumctl get`, `datumctl apply`, `datumctl delete`, > and friends. If you don't already use these tools, you can skip this page. Once you generate a kubeconfig entry with `datumctl` — see [Using kubectl](/docs/datumctl/kubectl-interop) for the full walkthrough — you get a standard [kubeconfig](https://kubernetes.io/docs/concepts/configuration/organize-cluster-access-kubeconfig/) whose user is a [client-go exec credential plugin](https://kubernetes.io/docs/reference/access-authn-authz/authentication/#client-go-credential-plugins). That means **any** client-go-based tool that reads a kubeconfig authenticates to the Datum Cloud control plane the same way kubectl does: it runs `datumctl` to fetch a fresh, short-lived credential on each request. There are no static keys to embed and nothing to rotate by hand. ## The one requirement > [!NOTE] > > The machine running the tool must have the `datumctl` binary on its `PATH` > **and** an active session (an interactive login, or a > [service account](/docs/datumctl/auth/service-accounts) for automation). Because authentication works by executing `datumctl` as an exec credential plugin, the credential handoff only happens where `datumctl` is installed and signed in. A tool can read the kubeconfig from anywhere, but the actual token exchange runs locally through `datumctl`. If `datumctl` isn't on `PATH` or has no session, the tool will fail to authenticate. The generated kubeconfig entry is portable across tools, but it references `datumctl` by name — so keep the binary available in the same environment where the tool runs. For non-interactive environments, generate the entry with `--exec-interactive-mode Never` so the exec plugin fails fast instead of waiting on a prompt that can never be answered (see [Using kubectl](/docs/datumctl/kubectl-interop)). ## Generate the kubeconfig first Every tool below assumes you have already written a kubeconfig entry and selected it as your current context: ```bash theme={null} # Log in so datumctl has an active session datumctl login # Write a context for the project (or organization) control plane datumctl auth update-kubeconfig --project your-project-id ``` The full flag reference — `--organization` vs `--project`, `--kubeconfig`, `--hostname`, and `--exec-interactive-mode` — lives on [Using kubectl](/docs/datumctl/kubectl-interop). To keep Datum Cloud entries separate from your other clusters, write to a dedicated file with `--kubeconfig` and point each tool at it. See [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how organization and project scope map to control planes. ## Helm [Helm](https://helm.sh/) uses the current kubeconfig context by default, so once your Datum Cloud context is selected, Helm authenticates through `datumctl` with no extra configuration. As with kubectl, you can point Helm at a specific file or context: ```bash theme={null} # Use a dedicated Datum Cloud kubeconfig helm --kubeconfig ~/.kube/datum-config list # Or select the context explicitly helm --kube-context my-datum-context list ``` Refer to the [Helm](https://helm.sh/) documentation for chart and release management specifics. ## Terraform Kubernetes provider Configure the [Terraform Kubernetes provider](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs) against your kubeconfig. The simplest form points the provider at the file and context: ```hcl theme={null} provider "kubernetes" { config_path = "~/.kube/datum-config" config_context = "my-datum-context" } ``` The provider also supports an `exec` block, which mirrors what `datumctl auth update-kubeconfig` already writes into the kubeconfig — the provider invokes `datumctl` to obtain a credential on each apply. Either way, `datumctl` must be on `PATH` with an active session on the machine (or CI runner) that runs `terraform`. Consult the [Terraform Kubernetes provider](https://registry.terraform.io/providers/hashicorp/kubernetes/latest/docs) docs for the full schema. ## k9s and other kubeconfig-driven tools k9s and similar kubeconfig-driven tools read the same kubeconfig and honor the `KUBECONFIG` environment variable and current context. Point them at your Datum Cloud context — for example by exporting `KUBECONFIG` — and they authenticate through `datumctl` automatically: ```bash theme={null} export KUBECONFIG=~/.kube/datum-config k9s ``` The same pattern applies to any tool built on client-go: it reads the kubeconfig, runs the exec plugin, and talks to the Datum Cloud control plane. Check each tool's own documentation for how it selects a kubeconfig or context. ## Automation and GitOps For CI/CD and GitOps, run the tool from a **CI runner or workstation that has `datumctl` installed and is authenticated with a [service account](/docs/datumctl/auth/service-accounts)**. The [GitHub Actions](/docs/datumctl/cicd/github-actions) setup action installs `datumctl` and authenticates a service account for you; see [Automating in CI/CD](/docs/datumctl/cicd/automating) for the broader pattern. > [!WARNING] > > Because authentication runs the `datumctl` binary as an exec credential > plugin, an environment **without** `datumctl` cannot use this flow. A > controller running inside an unrelated cluster with no `datumctl` binary is > not covered. Frame [Flux](https://fluxcd.io/) or > [Argo CD](https://argo-cd.readthedocs.io/) usage as running from CI (or a > machine) that has `datumctl` plus a service account session — not as an > in-cluster controller reconciling the Datum Cloud control plane directly. ## Related * [Using kubectl](/docs/datumctl/kubectl-interop) — how `datumctl auth update-kubeconfig` writes the kubeconfig entry and how the credential handoff works. * [GitOps with Flux & Argo CD](/docs/datumctl/gitops) — run a git-driven deployment loop onto a Datum Cloud control plane using the same exec-credential kubeconfig. * [Service accounts](/docs/datumctl/auth/service-accounts) — non-interactive credentials for automated environments. * [GitHub Actions](/docs/datumctl/cicd/github-actions) — install and authenticate `datumctl` on a CI runner. * [Automating in CI/CD](/docs/datumctl/cicd/automating) — the supported topology for running these tools from automation. * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — how organization and project scope map to control planes. # Output formats & scripting Source: https://datum.net/docs/datumctl/output-and-scripting Machine-readable output, structured errors, exit codes, and patterns for wrapping datumctl in CI pipelines and AI agents. > [!NOTE] > > **Working with this CLI via an AI agent?** This page is the contract an agent should code against: request `-o json`, read data from standard output and diagnostics from standard error, and branch on exit codes. See also [Agents Overview](/docs/agents/overview). `datumctl` prints human-friendly tables by default, but it is built to be scripted. Every read command can emit JSON or YAML, errors can be serialized into a structured envelope, and exit codes are predictable. This page shows you how to drive `datumctl` from shell scripts, CI pipelines, and AI agents. > [!NOTE] > > `datumctl` writes command results to **stdout** and errors to **stderr**. These are two independent channels controlled by two independent flags: `-o`/`--output` shapes the data on stdout, and `--error-format` shapes errors on stderr. You can set them independently. ## Choosing an output format Read and inspection commands accept the `-o` (or `--output`) flag. When you omit it, `datumctl` prints a formatted table designed for humans. Pass a machine format to get output you can parse. The `-o` flag is **per-command** — it is not a global flag. It is available on commands that render resources, including `datumctl get`, `datumctl edit`, `datumctl create`, `datumctl apply`, `datumctl version`, and `datumctl api-resources`. For `datumctl get`, the supported values are: | Value | Description | | - | - | | *(none)* | Formatted table (the default), for reading in a terminal. | | `wide` | The default table plus extra columns. | | `json` | A single JSON document. Best for piping into `jq`. | | `yaml` | A YAML document. | | `name` | Just the resource identifiers, one per line — ideal for shell loops. | | `jsonpath`, `jsonpath-as-json`, `jsonpath-file` | Extract specific fields with a JSONPath expression. | | `custom-columns`, `custom-columns-file` | Build your own column layout. | | `go-template`, `go-template-file`, `template`, `templatefile` | Render output through a Go template. | > [!NOTE] > > A few commands take `-o` but with their own value set. `datumctl explain` accepts `-o plaintext` or `-o plaintext-openapiv2` to choose a schema-rendering style, and `datumctl diff` always emits YAML (it has no `-o` flag). When in doubt, run `datumctl --help`. ```bash theme={null} # Default table — human reading datumctl get projects --organization your-org-id # Full resource as JSON datumctl get project your-project-id --organization your-org-id -o json # Full resource as YAML datumctl get project your-project-id --organization your-org-id -o yaml # Just the names, one per line datumctl get projects --organization your-org-id -o name ``` ## Piping and jq patterns Because `-o json` produces standard JSON, you can pipe it straight into [`jq`](https://jqlang.org/) or any JSON tooling. ```bash theme={null} # Pull a single field out of one resource datumctl get project your-project-id --organization your-org-id -o json \ | jq -r '.metadata.name' # List returns a JSON object with an "items" array — iterate it datumctl get projects --organization your-org-id -o json \ | jq -r '.items[].metadata.name' # Filter by a status condition datumctl get projects --organization your-org-id -o json \ | jq -r '.items[] | select(.status.conditions[]?.type == "Ready") | .metadata.name' ``` The `name` format is the simplest way to feed identifiers into a loop: ```bash theme={null} # Describe every project in an organization datumctl get projects --organization your-org-id -o name \ | while read -r ref; do datumctl describe "$ref" --organization your-org-id done ``` > [!TIP] > > For extracting one or two fields you often do not need `jq` at all — a `jsonpath` expression keeps the whole thing in one process: > > ```bash theme={null} > datumctl get projects --organization your-org-id \ > -o jsonpath='{.items[*].metadata.name}' > ``` ## Structured errors By default `datumctl` prints errors as plain text on stderr, prefixed with `error:`. For automation you can switch to a structured envelope with the global `--error-format` flag. `--error-format` is a **persistent (global) flag** — it works on every command. It accepts one of three values: | Value | Behavior | | - | - | | `human` | The default. Human-readable `error: ` text on stderr. | | `json` | A single-line JSON error envelope. | | `yaml` | The same envelope rendered as YAML. | Passing any other value fails fast before the command runs: ```text theme={null} $ datumctl get organizations --error-format foo error: invalid value "foo" for --error-format Allowed values: human, json, yaml. ``` In `json` or `yaml` mode, a failed command emits an envelope under a top-level `error` key: ```bash theme={null} datumctl get projects --organization does-not-exist --error-format json ``` ```json theme={null} { "error": { "message": "the organization \"does-not-exist\" was not found", "hint": "Run 'datumctl get organizations' to see organizations you can access.", "retryable": false } } ``` The `code`, `hint`, and `details` fields are omitted when they are empty, so the exact set of keys varies by error. The envelope fields are: | Field | Meaning | | - | - | | `message` | The user-facing error message. Always present. | | `code` | An optional machine-readable identifier (for example `AUTH_EXPIRED`) that agents can branch on. Omitted when empty. | | `hint` | Actionable guidance for resolving the error, when available. Omitted when empty. | | `retryable` | `true` when the operation can be safely retried without further action. Always present. | | `details` | The underlying technical error, when one is attached. Omitted when empty. | > [!NOTE] > > `--error-format` only changes how **errors** are serialized on stderr. It does not affect successful command output on stdout — that is controlled entirely by `-o`. A script can safely combine, for example, `-o json` for results and `--error-format json` for failures. ## Treating warnings as errors `--warnings-as-errors` is a global boolean flag (hidden from `--help` output) that causes server-side warnings returned by resource commands to be treated as failures, producing a non-zero exit. Use it in pipelines when you want deprecation notices or other API warnings to break the build rather than pass silently. ```bash theme={null} datumctl apply -f ./project.yaml --organization your-org-id --warnings-as-errors ``` ## Exit codes `datumctl` follows conventional exit-code semantics, so scripts can branch on `$?`: | Exit code | Meaning | | - | - | | `0` | Success. | | `1` | A general error — for example an unknown command, an invalid flag value, an authentication failure, or an API request that failed. | | `>1` | A lower-level fatal error surfaced by the underlying request machinery. | `datumctl diff` is a special case, mirroring the `diff` convention: | Exit code | Meaning | | - | - | | `0` | No differences were found. | | `1` | Differences were found. | | `>1` | An error occurred. | This makes `diff` useful as a gate before writing. Exit code `1` is not a failure, so don't chain `diff && apply`: the apply would run only when there is nothing to change. Handle each code explicitly: ```bash theme={null} datumctl diff -f ./project.yaml --organization your-org-id case $? in 0) echo "No changes." ;; 1) datumctl apply -f ./project.yaml --organization your-org-id ;; *) echo "diff failed" >&2; exit 1 ;; esac ``` ## Wrapping datumctl in scripts and agents **1. Authenticate non-interactively** Interactive browser login is the default. For CI and headless environments, pass `--no-browser` to `datumctl login`, or use a service account. Never rely on a prompt that a pipeline cannot answer. See [Service accounts](/docs/datumctl/auth/service-accounts) for non-interactive credentials, or [Automating datumctl in CI/CD](/docs/datumctl/cicd/automating) for the full pipeline setup. **2. Always pass a machine format** Add `-o json` (or `-o yaml`) to read commands and `--error-format json` globally. Do not parse the human table — its columns are meant to change over time. **3. Pin the context explicitly** Pass `--organization` and `--project` on every invocation instead of relying on the active context. Scripts and agents should be self-describing and not depend on ambient state that another process might change. See [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how scope is resolved and which flags and environment variables win. **4. Branch on exit codes and the error envelope** Check `$?` first. When it is non-zero and you asked for `--error-format json`, parse the `error` envelope: read `retryable` to decide whether to back off and retry, and `code`/`hint` to decide what to do next. **5. Prefer safe, idempotent operations** Use `datumctl apply` for declarative create-or-update, validate with `--dry-run=server`, and preview writes with `datumctl diff -f`. Remember that `datumctl delete` proceeds without a confirmation prompt. > [!WARNING] > > `datumctl delete` has no confirmation prompt. In automation, gate destructive operations behind an explicit review step or a `--dry-run=client` preview before running the real delete. A minimal, robust wrapper looks like this: ```bash theme={null} #!/usr/bin/env bash set -euo pipefail ORG="your-org-id" if ! output=$(datumctl get projects --organization "$ORG" -o json --error-format json 2>err.json); then code=$(jq -r '.error.code // empty' err.json) retryable=$(jq -r '.error.retryable' err.json) echo "datumctl failed (code=${code:-none}, retryable=$retryable)" >&2 exit 1 fi echo "$output" | jq -r '.items[].metadata.name' ``` ## Related * [Logging in](/docs/datumctl/auth/logging-in) — how to log in interactively, including `--no-browser` for headless flows. * [Automating datumctl in CI/CD](/docs/datumctl/cicd/automating) — headless service-account login, credential helpers, and cleanup for any pipeline. * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — how the active organization and project are resolved, and how to set them explicitly. * [Reading resources](/docs/datumctl/resources/reading) — the `get` and `describe` commands these output controls apply to. * [Changing resources](/docs/datumctl/resources/changing) — the `apply`, `diff`, and `delete` commands you gate on exit codes in scripts. * [Discovering resources & schemas](/docs/datumctl/discovering-resources) — the `api-resources`, `api-versions`, and `explain` commands that also honor `-o` for machine-readable output. * [Audit logs & events](/docs/datumctl/activity/audit-and-events) — apply these JSON/JSONPath and pagination patterns to export audit records and stream them into a SIEM. # datumctl Source: https://datum.net/docs/datumctl/overview datumctl is the command-line tool for Datum Cloud: one set of commands for every resource, for people, scripts, CI, and AI agents. > [!NOTE] > > **Working with this CLI via an AI agent?** Agents can run `datumctl` directly. Use `-o json` and the structured errors described in [Output and scripting](/docs/datumctl/output-and-scripting), and install Datum's [Skills](/docs/agents/skills/index) so the agent knows the canonical commands for each resource. For an MCP client instead of a shell, see [Datum MCP](/docs/agents/datum-mcp). `datumctl` is the command-line tool for working with Datum Cloud. It exposes a consistent resource model — the same `get`, `apply`, `describe`, and `delete` verbs across every resource type — so both developers and AI agents get a fast, predictable way to interact with Datum. > [!NOTE] > > Ready to play? [Install](/docs/datumctl/install) `datumctl` and get started. ## Find your way around > [card] [Authentication](/docs/datumctl/auth/logging-in) > > Sign in, use service accounts for automation, and manage sessions and tokens. > [card] [Working with resources](/docs/datumctl/resources/reading) > > Read, change, and safely preview changes to Datum Cloud resources. > [card] [Contexts & scoping](/docs/datumctl/contexts-and-scoping) > > Control which organization and project your commands act on. > [card] [Compute](/docs/datumctl/compute/overview) > > Deploy and manage containerized workloads with the compute plugin. > [card] [Activity & audit](/docs/datumctl/activity/overview) > > Investigate who changed what, review events, and follow the activity feed. > [card] [Interactive console](/docs/datumctl/console) > > Browse and monitor resources in a full-screen terminal UI. > [card] [AI & MCP](/docs/datumctl/ai/assistant) > > Ask about your resources in natural language, or connect AI clients over MCP. > [card] [CI/CD](/docs/datumctl/cicd/github-actions) > > Install and authenticate `datumctl` in GitHub Actions and other pipelines. > [card] [Plugins](/docs/datumctl/plugins/using-plugins) > > Extend the CLI with additional commands from plugin catalogs. Also useful: [Output formats & scripting](/docs/datumctl/output-and-scripting), [Discovering resources & schemas](/docs/datumctl/discovering-resources), and — for existing Kubernetes users — the Kubernetes interop section, which covers [using datumctl with kubectl](/docs/datumctl/kubectl-interop), [Kubernetes tools](/docs/datumctl/kubernetes-tools) like Helm and k9s, and [GitOps with Flux & Argo CD](/docs/datumctl/gitops). ## Product plugins Each Datum product adds its own commands to `datumctl` as an official [plugin](/docs/datumctl/plugins/using-plugins). Install the ones you use, and run `datumctl plugin search` to see them all: | Plugin | What it does | Documentation | | - | - | - | | `alb` | Create and manage Application Load Balancers, their routes, hostnames, firewall, and basic authentication, and read their access logs. | [ALB operations](/docs/alb/operations/overview) | | `assistant` | Chat with Patch, the Datum Cloud assistant. | [AI assistant](/docs/datumctl/ai/assistant) | | `compute` | Build, deploy, scale, and debug Compute workloads. | [Compute commands](/docs/datumctl/compute/overview) | | `dns` | Manage DNS zones and records, check delegation, and import and export zone files. | [DNS operations](/docs/domain-dns/operations) | | `network` | Create and manage gVPC networks and network services. | [gVPC](/docs/galactic-vpc/overview) | | `search` | Search your resources by name and kind. | [Search across your resources](/docs/datumctl/discovering-resources#search-across-your-resources) | | `telemetry` | Query the logs in your project, including load balancer access logs. | [Query access logs](/docs/alb/observability#query-access-logs-across-load-balancers) | `datumctl --help` lists the plugins you have installed, and the plugins in the catalog that you haven't installed yet. ## How it works At a high level, `datumctl` does three things: 1. **Authenticates you** - Opens a browser login, stores credentials securely, and refreshes tokens automatically. 2. **Talks to the Datum API** - Sends requests to create, read, update, and delete resources with authentication handled behind the scenes. 3. **Formats output** - Displays results as tables by default (supports JSON and YAML). # Adding Catalogs Source: https://datum.net/docs/datumctl/plugins/adding-catalogs Register additional and internal plugin catalogs, and understand the trust decision and enterprise guardrails involved. A **catalog** is a published list of datumctl plugins. It's the marketplace that `plugin search`, `plugin browse`, and `plugin install` read from. In command syntax a catalog is called an **index**, so the commands live under `datumctl plugin index`. The official **datum** catalog is always present and trusted with no setup. You can register additional catalogs — a company's internal catalog or a community one — and their plugins then appear alongside Datum's. > [!NOTE] > > Registering a catalog only makes its plugins discoverable. Nothing is downloaded or run until you install a specific plugin from it. ## Registering a catalog ```bash theme={null} datumctl plugin index add ``` The `` is a short local alias you'll use to refer to the catalog (for example `acme/deploy` when installing). The `` can be: > [tab] HTTPS URL > > ```bash theme={null} > datumctl plugin index add acme https://plugins.acme.example/index.yaml > ``` > [tab] GitHub repo > > ```bash theme={null} > datumctl plugin index add community datum-community/datumctl-plugins > ``` > [tab] Local path > > ```bash theme={null} > datumctl plugin index add local ./my-catalog > ``` Remote sources must be HTTPS. Local paths are supported for development and air-gapped use. ### The one-time trust decision Adding a third-party catalog is a deliberate, one-time trust decision. datumctl prompts you to confirm and explains what you're agreeing to: > [!WARNING] > > Datum does not review, verify, or endorse plugins from a third-party catalog. Its plugins are programs that run on your machine **with your Datum credentials**. Only add catalogs you trust. To confirm without a prompt in scripts or CI, pass `--yes`: ```bash theme={null} datumctl plugin index add acme https://plugins.acme.example/index.yaml --yes ``` > [!NOTE] > > Regardless of which catalog a plugin comes from, downloads remain HTTPS-only and every plugin binary is checksum-verified on install and on every run. The trust decision is about *who authored the plugin*, not about transport security. ## Listing catalogs ```bash theme={null} datumctl plugin index list ``` This shows every registered catalog with its type, plugin count, trust badge (`official` or `third-party`), and description. The official **datum** catalog is always listed first. ## Refreshing catalog metadata Catalog listings are cached. Refresh them to pick up newly published plugins or versions: ```bash theme={null} # Refresh every catalog datumctl plugin index update # Refresh a single catalog datumctl plugin index update community ``` ## Removing a catalog ```bash theme={null} datumctl plugin index remove community ``` The official **datum** catalog cannot be removed, and catalogs pre-seeded by your organization cannot be removed here. Removing a catalog does **not** uninstall plugins you already installed from it. ## Enterprise guardrails Platform teams can pre-seed approved catalogs onto a workstation and constrain which catalogs users may add. This is configured through environment variables read by datumctl: * `DATUMCTL_PLUGIN_MANAGED_CONFIG` — path to a YAML file that pre-registers approved catalogs and, optionally, an allow-list. * `DATUMCTL_PLUGIN_ALLOWED_INDEXES` — a comma-separated allow-list that supplements the file. A managed configuration file looks like this: ```yaml theme={null} # Catalogs pre-registered for the user (no manual action, cannot be removed) indexes: - name: acme source: github.com/acme-corp/datumctl-plugins description: ACME internal plugins owner: ACME Platform Team # Constrains which catalogs a user may add themselves. # When empty, any catalog may be added. allowedIndexes: - github.com/acme-corp/* - plugins.acme.example ``` ### How the allow-list scopes publishers When an allow-list is in effect, only catalogs matching an entry may be registered. Entries are matched by form: | Entry form | Example | Authorizes | | - | - | - | | GitHub owner | `github.com/acme-corp` | any repo under that owner | | GitHub owner (wildcard) | `github.com/acme-corp/*` | any repo under that owner | | GitHub repo | `github.com/acme-corp/datumctl-plugins` | exactly that repo | | Any GitHub repo | `github.com/*` | every GitHub repo | | Host pattern | `plugins.acme.example` or `*.acme.example` | a non-GitHub remote host | | Bare name | `local` | a local-path catalog by name | GitHub sources are authorized **only** by a `github.com/` scope entry — a plain host pattern such as `raw.githubusercontent.com` will not green-light every GitHub repository. This lets platform teams pin permitted publishers down to a specific GitHub owner or repository. > [!NOTE] > > The allow-list is enforced continuously, not just when a catalog is added. If a catalog's source stops being permitted, datumctl marks it disabled: it still appears in `plugin index list` (flagged as disabled) but is excluded from search, browse, and install. ## Next steps * [Using plugins](/docs/datumctl/plugins/using-plugins) — install and run plugins from your catalogs * [Publishing catalogs](/docs/datumctl/plugins/publishing-catalogs) — author and host a catalog of your own # Building Plugins Source: https://datum.net/docs/datumctl/plugins/building-plugins Build a datumctl plugin in Go using the official SDK — manifest, context injection, credentials, and command forwarding. A datumctl plugin is a standalone binary that datumctl runs on your users' behalf. When a user runs `datumctl dns zones list`, datumctl finds your `datumctl-dns` binary, injects the current context and a credentials helper, and hands off to it. This guide walks through building a plugin in Go with the official SDK, `go.datum.net/datumctl/plugin`. > [!NOTE] > > The SDK reads only environment variables and execs subprocesses, so your plugin depends on it without pulling in any datumctl internals. A worked reference plugin lives at [`examples/plugin-dns`](https://github.com/datum-cloud/datumctl/tree/main/examples/plugin-dns) in the datumctl repository. ## The model * A plugin is a binary named `datumctl-` (for datumctl-native plugins) or `milo-` (for portable milo-os platform plugins). * Users install it with `datumctl plugin install`, which places it in the managed plugins directory (`~/.datumctl/plugins/` by default), or it can live on their `PATH`. * Users then run it as `datumctl ` — datumctl forwards the arguments, injects context and credentials, and forwards shell completion. * Before running a managed plugin, datumctl verifies its SHA256 fingerprint every time. ## Building your plugin **1. Add the SDK dependency** ```bash theme={null} go get go.datum.net/datumctl/plugin ``` **2. Declare a manifest and serve it** Call `plugin.ServeManifest` at the very top of `main()`. datumctl invokes your binary with `--plugin-manifest` to read its metadata; `ServeManifest` handles that protocol and exits before your command logic runs. ```go theme={null} var manifest = plugin.Manifest{ Name: "dns", Version: "v0.1.0", Description: "Manage Datum Cloud DNS zones", APIVersion: 1, MinAPIVersion: 1, } ``` **3. Build a root command with pre-wired flags** `plugin.NewRootCmd` returns a Cobra command with `--org`, `--project`, and `--output` flags already wired to the values datumctl injects. ```go theme={null} root := plugin.NewRootCmd("dns", "Manage Datum Cloud DNS resources") ``` **4. Read context and fetch a token** `plugin.Context()` reads the injected `DATUM_*` environment, and `plugin.Token()` calls the datumctl credentials helper for a fresh, short-lived access token. Call `Token()` immediately before each API request. ```go theme={null} ctx := plugin.Context() token, err := plugin.Token() ``` ## A minimal plugin The following is a complete plugin grounded in the reference example. It serves its manifest, wires the root command, and adds a `zones list` subcommand that acquires a token and calls the Datum Cloud API: ```go theme={null} package main import ( "fmt" "net/http" "os" "github.com/spf13/cobra" "go.datum.net/datumctl/plugin" ) var manifest = plugin.Manifest{ Name: "dns", Version: "v0.1.0", Description: "Manage Datum Cloud DNS zones", APIVersion: 1, MinAPIVersion: 1, } func main() { // Handle --plugin-manifest and exit before cobra runs. plugin.ServeManifest(manifest) root := plugin.NewRootCmd("dns", "Manage Datum Cloud DNS resources") zones := &cobra.Command{Use: "zones", Short: "Manage DNS zones"} zones.AddCommand(&cobra.Command{ Use: "list", Short: "List DNS zones", RunE: func(cmd *cobra.Command, args []string) error { ctx := plugin.Context() // Fetch a fresh token right before the API call. token, err := plugin.Token() if err != nil { return fmt.Errorf("failed to get credentials: %w", err) } url := fmt.Sprintf("https://%s/v1/organizations/%s/projects/%s/dnszones", ctx.APIHost, ctx.Org, ctx.Project) req, err := http.NewRequestWithContext(cmd.Context(), http.MethodGet, url, nil) if err != nil { return err } req.Header.Set("Authorization", "Bearer "+token) resp, err := http.DefaultClient.Do(req) if err != nil { return fmt.Errorf("API request failed: %w", err) } defer resp.Body.Close() fmt.Fprintf(cmd.OutOrStdout(), "DNS zones for org=%s project=%s (status %d)\n", ctx.Org, ctx.Project, resp.StatusCode) return nil }, }) root.AddCommand(zones) if err := root.Execute(); err != nil { os.Exit(1) } } ``` ### The injected context `plugin.Context()` returns the values datumctl sets before running your plugin — these come from the user's [active context](/docs/datumctl/contexts-and-scoping): | Field | Environment variable | Description | | - | - | - | | `Org` | `DATUM_ORG` | Current organization slug. | | `Project` | `DATUM_PROJECT` | Current project slug (may be empty). | | `APIHost` | `DATUM_API_HOST` | API base URL, e.g. `api.datum.net`. | | `PluginAPIVersion` | `DATUM_PLUGIN_API_VERSION` | Plugin API version the host declares. | | `CredentialsHelper` | `DATUM_CREDENTIALS_HELPER` | Absolute path to the datumctl binary. | | `Session` | `DATUM_SESSION` | Active session name (may be empty). | `plugin.Token()` runs the credentials helper (`$DATUM_CREDENTIALS_HELPER auth get-token`, adding `--session` when a session is active) and returns the token. Because tokens are short-lived, fetch one immediately before each request rather than caching it. ## Building and testing locally ```bash theme={null} # Build with the datumctl- prefix so datumctl recognizes it. go build -o datumctl-dns . # Drop it into the managed plugins directory. cp datumctl-dns ~/.datumctl/plugins/ # Run it through datumctl — context and credentials are injected automatically. datumctl dns zones list # Verify the manifest protocol directly. ./datumctl-dns --plugin-manifest ``` Shell completion is automatic: datumctl forwards completion requests to your binary, so your Cobra command's built-in completion works when users run `datumctl dns `. For flag-first commands, wrap your completion function with `plugin.WithFlagCompletion` to surface flags on a bare ``. ## Trust and verification for authors datumctl treats your plugin as untrusted code, which shapes how you distribute it: * **Ship checksums.** Whether users install from a catalog or directly from a GitHub release, datumctl verifies a SHA256 checksum before running your binary. Publish accurate checksums for every release archive. * **Managed installs are fingerprinted.** After install, datumctl records the binary's hash and re-verifies it on every run. Rebuilding or replacing the binary in place will cause datumctl to refuse it until it's reinstalled. * **A binary a user drops on their PATH is blocked by default.** datumctl will not run an unmanaged `datumctl-` binary until the user explicitly trusts it with `datumctl plugin trust `. ## Getting your plugin into a catalog Once your plugin is released with archives and checksums, list it in a catalog so users can install it by name. See [Publishing catalogs](/docs/datumctl/plugins/publishing-catalogs) for the manifest format, validation, and hosting. Users can always install directly from a GitHub release without a catalog: ```bash theme={null} datumctl plugin install your-org/datumctl-dns ``` This path requires a `checksums.txt` alongside your release archives, in goreleaser's default two-column format. ## Next steps * [Publishing catalogs](/docs/datumctl/plugins/publishing-catalogs) — list your plugin so users install it by name * [Using plugins](/docs/datumctl/plugins/using-plugins) — how users install and run what you build # Publishing Catalogs Source: https://datum.net/docs/datumctl/plugins/publishing-catalogs Author, validate, and host a datumctl plugin catalog so users and enterprises can discover and install your plugins. A catalog is a single manifest file — an `index.yaml` — that lists the plugins you publish, their versions, and per-platform download URLs and checksums. Users register your catalog with [`datumctl plugin index add`](/docs/datumctl/plugins/adding-catalogs), then search, browse, and install from it like any other. This guide covers the manifest format, validation, hosting, and versioning guidance. > [!NOTE] > > If you're writing the plugin itself (not just publishing a catalog), start with [Building plugins](/docs/datumctl/plugins/building-plugins). This guide assumes you already have released plugin binaries to point at. ## The catalog manifest A catalog manifest is a `PluginList` document. It has an optional catalog-level identity header followed by an `items` list of plugin entries. Here is a complete, annotated example: ```yaml theme={null} apiVersion: datumctl.datum.net/v1alpha1 kind: PluginList # Optional catalog identity, shown in `plugin index list` and `plugin browse`. name: acme description: ACME internal datumctl plugins owner: ACME Platform Team homepage: https://plugins.acme.example items: - apiVersion: datumctl.datum.net/v1alpha1 kind: Plugin metadata: name: deploy # the name users install and run spec: shortDescription: Deploy ACME workloads to Datum Cloud description: | Longer description shown when inspecting the plugin in the browser. homepage: https://github.com/acme-corp/datumctl-deploy version: v1.2.0 # the recommended version to install platforms: - selector: matchLabels: os: linux arch: amd64 uri: https://github.com/acme-corp/datumctl-deploy/releases/download/v1.2.0/datumctl-deploy_Linux_x86_64.tar.gz sha256: - selector: matchLabels: os: darwin arch: arm64 uri: https://github.com/acme-corp/datumctl-deploy/releases/download/v1.2.0/datumctl-deploy_Darwin_arm64.tar.gz sha256: ``` ### Field reference **Catalog header (all optional):** | Field | Purpose | | - | - | | `name` | Friendly catalog name shown in listings. | | `description` | One-line description of the catalog. | | `owner` | Who maintains the catalog. | | `homepage` | Catalog homepage URL. | A manifest with no header still parses — the only required content is the plugin list. **Each entry under `items` (`spec`):** | Field | Required | Purpose | | - | - | - | | `metadata.name` | Yes | The name users install and run (lowercase letters, digits, hyphens). | | `spec.version` | Yes | The recommended version to install. | | `spec.shortDescription` | Recommended | Shown in `search` and `browse`. | | `spec.description` | Optional | Longer text shown when inspecting the plugin. | | `spec.homepage` | Optional | The plugin's homepage. | | `spec.platforms` | Yes | One entry per supported OS/arch. | **Each entry under `spec.platforms`:** | Field | Required | Purpose | | - | - | - | | `selector.matchLabels` | Yes | Matched against `{os: , arch: }` to pick the right archive. | | `uri` | Yes | HTTPS URL of the release archive (`.tar.gz`, `.tgz`, or `.zip`). | | `sha256` | Yes | Lowercase hex SHA256 of the archive, verified on download. | | `files` | Optional | `from`/`to` directives naming the binary to extract from the archive. | > [!WARNING] > > Every `uri` must be HTTPS. datumctl rejects an entire catalog manifest if any download URI uses a non-HTTPS scheme. The binary inside each archive should be named `datumctl-` (or `datumctl-.exe` on Windows). datumctl runs it as `datumctl `. ## Validating before you publish Lint a manifest before you ship it. The validator reports missing plugin names or versions, missing or non-HTTPS download URIs, missing checksums, and invalid platform selectors: ```bash theme={null} # Validate a local manifest datumctl plugin index validate ./index.yaml # Validate a published manifest datumctl plugin index validate https://plugins.acme.example/index.yaml ``` The source may be a local path, an HTTPS URL, or a GitHub `owner/repo`. A clean manifest reports `OK: manifest is valid`. ## Hosting your catalog datumctl fetches catalog manifests over HTTPS only. You can host `index.yaml` in either of these ways: * **A GitHub repository.** Place `index.yaml` at the repository root. Users then add it with the `owner/repo` shorthand: ```bash theme={null} datumctl plugin index add acme acme-corp/datumctl-plugins ``` datumctl fetches from the default branch (`main`, falling back to `master`). * **Any HTTPS host.** Serve `index.yaml` from your own site and have users add the full URL: ```bash theme={null} datumctl plugin index add acme https://plugins.acme.example/index.yaml ``` ## Publisher scoping for enterprises Enterprises can constrain which catalogs their users may add through an allow-list, and GitHub sources are scoped by **owner** (and optionally repository). To make your catalog easy for enterprises to approve: * Host it under a stable GitHub owner or repository so it can be allow-listed as `github.com/` or `github.com//`, or under a stable dedicated hostname for a host-pattern allow-list entry. * Avoid moving the catalog between owners or hosts, which would require every enterprise to update their allow-list. See [Adding catalogs](/docs/datumctl/plugins/adding-catalogs#enterprise-guardrails) for how allow-list entries are matched. ## Versioning guidance `spec.version` is the version datumctl installs when a user runs `datumctl plugin install ` without pinning. Users can pin a specific version with `datumctl plugin install @`. To make pinning reliable for your users: * Use consistent, sortable version strings (semantic versioning, e.g. `v1.2.0`) so newer releases are recognized as updates. * Keep `spec.version`, the archive `uri`, and its `sha256` in lockstep — bump all three together for each release. * Publish immutable release archives at stable URLs so a pinned version keeps resolving to the same bytes. ## Next steps * [Building plugins](/docs/datumctl/plugins/building-plugins) — write the plugin binaries your catalog points at * [Adding catalogs](/docs/datumctl/plugins/adding-catalogs) — how users register and trust your catalog # Using Plugins Source: https://datum.net/docs/datumctl/plugins/using-plugins Install, run, and manage datumctl plugins that extend the CLI with additional commands. Plugins extend `datumctl` with additional commands without changing the core CLI. A plugin is an independent binary that datumctl runs on your behalf — once installed, you invoke it exactly like a built-in command. For the thinking behind plugins, see [Extending our CLI with plugins](https://www.datum.net/blog/extending-our-cli-with-plugins/) on the Datum blog. > [!NOTE] > > Plugins are optional. You never need one to use `datumctl`, and the core CLI works the same whether or not you have any installed. ## What a plugin is When you install the `compute` plugin from the official **datum** catalog, its commands become available under `datumctl compute`: ```bash theme={null} datumctl compute --help ``` (See [Compute](/docs/datumctl/compute/overview) for what that plugin does.) datumctl injects your current organization, project, and a fresh short-lived access token into the plugin automatically, so plugins work with your [active context](/docs/datumctl/contexts-and-scoping) and [credentials](/docs/datumctl/auth/managing-accounts) without a separate login. Plugins come from **catalogs** (also called indexes). The official **datum** catalog is curated by Datum and always available with no setup — its plugins carry an `official` trust badge. You can also add third-party catalogs, whose plugins carry a `third-party` badge. See [Adding catalogs](/docs/datumctl/plugins/adding-catalogs) to register more. ## Finding plugins Search every registered catalog for available plugins: ```bash theme={null} # List everything available across all catalogs datumctl plugin search # Filter by name or description datumctl plugin search compute # Scope the search to a single catalog datumctl plugin search deploy --index acme ``` Results show the plugin name, the catalog it came from, its version, its trust badge, and a short description. Prefer to explore interactively? `plugin browse` opens a filterable picker where you can inspect a plugin's details and install it in place: ```bash theme={null} datumctl plugin browse ``` > [!NOTE] > > `plugin browse` requires an interactive terminal. In scripts or CI, use `plugin search` instead. ## Installing a plugin Datum's own plugins are documented with the products they manage. For example, see [DNS operations](/docs/domain-dns/operations) for the `dns` plugin and [Compute commands](/docs/datumctl/compute/overview) for the `compute` plugin. Install by name from a catalog: ```bash theme={null} # Install from the official datum catalog (searched first) datumctl plugin install compute # Install from a specific catalog datumctl plugin install acme/deploy ``` You can also install directly from a GitHub release, without a catalog: ```bash theme={null} # Latest release datumctl plugin install your-org/datumctl-example # A specific release datumctl plugin install your-org/datumctl-example@v1.2.0 ``` > [!NOTE] > > If a bare name exists in more than one catalog, datumctl lists the matches instead of guessing. Qualify the name with its catalog (for example `acme/deploy`) to choose. ### Pinning a version Pin a specific version at install time by appending `@version`: ```bash theme={null} # Pin a catalog plugin datumctl plugin install compute@v0.8.0-dev.7 datumctl plugin install acme/deploy@v2.0.0 # Pin a GitHub release datumctl plugin install your-org/datumctl-example@v1.2.0 ``` Without `@version`, datumctl installs the version the catalog recommends. ### Restoring your plugin set Run `install` with no arguments to restore every plugin you previously installed, at its recorded version. This reproduces your plugin set on a new machine: ```bash theme={null} datumctl plugin install ``` ## Listing what you have ```bash theme={null} datumctl plugin list ``` This shows each installed plugin with its catalog, version, trust badge, description, and a status indicator: | Status | Meaning | | - | - | | `ok` | Installed and compatible with this datumctl. | | `update` | A newer version is available in its catalog. | | `!` | Built for a different datumctl version. | | `?` | Version information is unavailable. | ## Running an installed plugin An installed plugin behaves like a native command: ```bash theme={null} datumctl compute --help ``` Tab completion works transparently — datumctl forwards completion requests to the plugin, so completing subcommands and flags just works: ```bash theme={null} datumctl compute ``` ## Upgrading and removing ```bash theme={null} # Upgrade a plugin to the latest version datumctl plugin upgrade compute # Remove a plugin datumctl plugin remove compute ``` To upgrade every plugin you have installed at once, run `datumctl plugin upgrade --all`. `datumctl plugin list` shows `update` in the `STATUS` column for any plugin with a newer version. `plugin upgrade` runs the same verified install flow as a fresh install. ## How plugins stay trustworthy datumctl treats a plugin as untrusted code and protects you in a few ways: * **HTTPS-only downloads.** Plugin archives and catalog manifests are fetched over HTTPS only. Downloads to private, loopback, or link-local addresses are refused, and redirects are re-checked on every hop. * **Checksum verification on every run.** datumctl records a SHA256 fingerprint of each plugin binary at install time and verifies it *every time the plugin runs*. If a managed plugin binary is modified after installation, datumctl refuses to run it. * **Trust badges.** Every plugin and catalog shows whether it is `official` (Datum's curated **datum** catalog) or `third-party`, so you always know the source. > [!WARNING] > > Third-party plugins are programs that run on your machine with your Datum credentials, and Datum does not review them. Only install plugins from catalogs and sources you trust. See [Adding catalogs](/docs/datumctl/plugins/adding-catalogs) for the trust decision involved in registering a catalog. ### Trusting a plugin already on your PATH If you have a plugin binary on your `PATH` that datumctl did not install (named `datumctl-` or `milo-`), datumctl blocks it from running until you explicitly trust it: ```bash theme={null} # Allow an unmanaged PATH plugin to run datumctl plugin trust dns # Revoke that trust later datumctl plugin untrust dns ``` Trusting records the binary's path and fingerprint. If the binary changes afterward, datumctl blocks it again until you re-run `plugin trust`. ## Next steps * [Adding catalogs](/docs/datumctl/plugins/adding-catalogs) — register additional or internal marketplaces * [Building plugins](/docs/datumctl/plugins/building-plugins) — write your own plugin with the Go SDK * [Install](/docs/datumctl/install) — new to `datumctl`? Install and log in first * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — the organization and project a plugin inherits when it runs # Changing resources Source: https://datum.net/docs/datumctl/resources/changing Create, update, and delete Datum Cloud resources — with declarative apply as the primary workflow. `datumctl` treats your Datum Cloud infrastructure as declarative configuration: you describe the state you want in a manifest, store it in source control, and let the CLI reconcile the platform to match. The same handful of write verbs — `apply`, `create`, `edit`, and `delete` — work across every resource type, so once you learn them you can change anything the platform exposes. > [!NOTE] > > Every write runs against your active scope. Use `--organization` or `--project` to target a context, and see [Contexts & scoping](/docs/datumctl/contexts-and-scoping) for how a command decides where it acts. Run `datumctl ctx` to check your active context before making changes. ## Apply is the primary workflow `datumctl apply` is the recommended way to manage resources. You give it a manifest describing the state you want; if the resource does not exist it is created, and if it already exists it is updated to match. Because apply is idempotent, you can run the same command as many times as you like and converge on the same result. ```bash theme={null} # Apply a single manifest datumctl apply -f ./proxy.yaml --project your-project-id --namespace default # Apply every manifest in a directory datumctl apply -f ./infra/ --organization your-org-id # Apply from stdin cat dnszone.yaml | datumctl apply -f - --project your-project-id ``` Manifests are YAML or JSON. Multiple resources can live in a single file separated by YAML document markers (`---`), and each manifest must set the correct `apiVersion` and `kind` for its resource type. > [!TIP] > > Keep your manifests in version control and apply them from CI/CD. Treating network and platform configuration as code gives you review, history, and repeatable deployments — and lets you rebuild state from the same source at any time. A typical declarative loop looks like this: **1. Author or edit a manifest** Describe the resource you want in a `.yaml` file. Not sure which fields a type takes? Inspect the schema first with `datumctl explain `. **2. Preview the change** Run `datumctl diff -f ./proxy.yaml` to see exactly what the platform would change before you write anything. See [Safe changes](/docs/datumctl/resources/safe-changes). **3. Apply it** `datumctl apply -f ./proxy.yaml` creates or updates the resource to match your manifest. **4. Verify the result** `datumctl get -o yaml` confirms the live state matches what you intended. > [!WARNING] > > Preview before you write. Run `datumctl diff -f` to see the server-computed changes, and add `--dry-run=server` to `apply` to validate a manifest against the API without persisting anything. [Safe changes](/docs/datumctl/resources/safe-changes) covers both. ## Create from a manifest `datumctl create` provisions a new resource from a manifest. Unlike `apply`, it does not update an existing resource — if a resource of the same name already exists, the create fails. Reach for `create` when you specifically want to fail loudly on a name collision; reach for `apply` for idempotent create-or-update. ```bash theme={null} # Create a project from a manifest datumctl create -f ./project.yaml --organization your-org-id # Create from stdin cat dnszone.yaml | datumctl create -f - --project your-project-id # Validate without creating datumctl create -f ./project.yaml --organization your-org-id --dry-run=server ``` ## Edit a resource in place `datumctl edit` fetches a resource, opens it in your local text editor, and applies whatever you save back to the platform. It is convenient for quick, one-off changes when you do not have a manifest on disk. ```bash theme={null} # Edit a project datumctl edit project your-project-id --organization your-org-id # Edit a DNS zone in a chosen editor EDITOR="code --wait" datumctl edit dnszone my-zone --project your-project-id --namespace default # Edit in JSON instead of the default YAML datumctl edit project your-project-id --organization your-org-id -o json ``` The editor is chosen from the `EDITOR` environment variable (the CLI also honors `KUBE_EDITOR`), falling back to `vi` on Linux/macOS or `notepad` on Windows. Changes are applied when you save and close the file. If the resource was modified server-side while your editor was open, `datumctl` detects the conflict and saves your edits to a temporary file so you can reconcile them. > [!NOTE] > > For changes you want to keep and repeat, prefer editing a manifest and running `datumctl apply` — that keeps your source of truth in version control. Use `edit` for exploratory or one-time tweaks. ## Patch individual fields `datumctl patch` changes one or more fields of a live resource in a single command, without rendering and applying a full manifest. Pass the patch inline with `-p`, or from a file with `--patch-file`. You can write patches in JSON or YAML. ```bash theme={null} # Change a single field with a merge patch datumctl patch workload my-app -p '{"spec":{"replicas":2}}' --project your-project-id # Write the patch as YAML datumctl patch workload my-app -p $'spec:\n replicas: 2' --project your-project-id # Patch a resource identified by a manifest file datumctl patch -f ./workload.yaml -p '{"spec":{"replicas":2}}' --project your-project-id # Use a JSON patch for precise, positional edits datumctl patch workload my-app --type=json -p '[{"op":"replace","path":"/spec/replicas","value":2}]' --project your-project-id ``` Choose the patch format with `--type`: | Type | Format | When to use it | | - | - | - | | `merge` (default) | A JSON merge patch: a partial document whose fields replace the matching fields of the live resource. | Nearly every change. | | `json` | A JSON patch: an ordered list of operations (`add`, `replace`, `remove`, and others) with explicit paths. | Precise edits to items inside lists. | ## Add, change, or remove labels and annotations `datumctl label` and `datumctl annotate` update the metadata on one or more live resources. Labels organize and select resources, for example with the `-l` flag of `datumctl get` and `datumctl delete`. Annotations are free-form key/value metadata that is not used for selection. Use annotations to record notes or to nudge a resource, for example setting a fresh annotation on a workload to trigger a rollout. Both commands take the same arguments: * Pass `KEY=VALUE` to add a label or annotation. * Pass `KEY-` (a trailing dash) to remove one. * Add `--overwrite` to change a key that already exists. Without it, `datumctl` leaves an existing key untouched. * Add `--list` to show the current labels or annotations on a resource. ```bash theme={null} # Add a label to a workload datumctl label workload my-app tier=frontend --project your-project-id # Change a label that already exists datumctl label workload my-app tier=backend --project your-project-id --overwrite # Remove a label datumctl label workload my-app tier- --project your-project-id # Label every DNS zone matching a label selector datumctl label dnszones -l team=platform env=prod --project your-project-id # Add an annotation to a workload datumctl annotate workload my-app restart=now --project your-project-id # Show the annotations on a project datumctl annotate project your-project-id --list --organization your-org-id ``` > [!TIP] > > `patch`, `label`, and `annotate` all accept `--dry-run=server`. Combine it with `-o yaml` to validate the change against the API server and preview the result without persisting anything: > > ```bash theme={null} > datumctl patch workload my-app -p '{"spec":{"replicas":2}}' --project your-project-id --dry-run=server -o yaml > ``` ## Delete a resource `datumctl delete` removes one or more resources, selected by type and name, by label selector, or from a manifest file. ```bash theme={null} # Delete by name datumctl delete project your-project-id --organization your-org-id # Delete from a manifest datumctl delete -f ./my-resource.yaml --organization your-org-id # Delete everything matching a label selector datumctl delete dnszones -l app=my-app --project your-project-id ``` > [!WARNING] > > `datumctl delete` has **no confirmation prompt** — it deletes immediately. It also does not perform a version check first, so a delete proceeds even if the resource changed after you last fetched it. Preview what would be removed with `--dry-run=client` before you commit: > > ```bash theme={null} > datumctl delete project your-project-id --organization your-org-id --dry-run=client > ``` ## Namespaces Many project-scoped resources live in a namespace. The examples above target the `default` namespace with `--namespace default`; if a command targets `default`, you can omit the flag. Currently only `default` is supported. Which organization or project a write acts on is covered in [Contexts & scoping](/docs/datumctl/contexts-and-scoping). ## Next steps * [Safe changes & declarative config](/docs/datumctl/resources/safe-changes) — preview every change with `datumctl diff -f` and `--dry-run=server` before it touches live infrastructure. * [Discovering resources & schemas](/docs/datumctl/discovering-resources) — find resource types and inspect their fields with `explain` and `api-resources` before authoring a manifest. * [Reading resources](/docs/datumctl/resources/reading) — verify the result with `datumctl get` and `datumctl describe` after a write. * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — control which organization or project a change targets. # Reading resources Source: https://datum.net/docs/datumctl/resources/reading List, retrieve, and inspect Datum Cloud resources with datumctl get and datumctl describe. Reading is the safe half of the resource model: `datumctl get` and `datumctl describe` never change anything on the platform, so they are the commands you reach for constantly — to see what exists, confirm a change landed, or troubleshoot a resource that isn't behaving. Both use the same verbs across every Datum Cloud resource type, so once you learn them you can read anything the platform exposes. > [!NOTE] > > New to the CLI? Start with [Install](/docs/datumctl/install) to install, log in, and run your first command. Every read runs against your active scope — see [Contexts & scoping](/docs/datumctl/contexts-and-scoping) to control which organization or project that is. *** ## `datumctl get` — list and retrieve `datumctl get` displays one or more resources. By default it prints a compact, human-readable table; add `-o` to switch to machine-readable output for scripting and inspection. ### List a resource type Pass a resource type to list every resource of that type in your current scope: ```bash theme={null} # List all DNS zones in a project datumctl get dnszones --project your-project-id # List all projects in an organization datumctl get projects --organization your-org-id ``` > [!TIP] > > `datumctl get organizations` lists your organization memberships and does not require an `--organization` or `--project` flag. Every other resource type needs one of those flags to identify the target context. ### Retrieve a single resource Add a name to fetch just one resource. Pairing this with `-o yaml` is the usual way to see a resource's full definition: ```bash theme={null} # Table row for one zone datumctl get dnszone my-zone --project your-project-id # The complete resource definition datumctl get dnszone my-zone --project your-project-id -o yaml ``` ### Choose an output format The `-o` (`--output`) flag controls how results are rendered. The formats you'll use most: | Format | What you get | | - | - | | *(default)* | A compact table of the most relevant columns | | `-o wide` | The default table plus extra columns | | `-o json` | Full resource JSON — ideal for `jq` and scripts | | `-o yaml` | Full resource YAML — matches the manifest form you'd `apply` | | `-o name` | Just `type/name`, one per line — handy for piping into other commands | ```bash theme={null} # Extra columns in the table datumctl get dnszones --project your-project-id -o wide # Full JSON for scripting datumctl get dnszones --project your-project-id -o json # Bare identifiers to feed another command datumctl get dnszones --project your-project-id -o name ``` > [!NOTE] > > `-o json` and `-o yaml` return the complete server-side object, including `status`, while `-o wide` only adds columns to the table. For structured-output patterns, `jq`/`yq` pipelines, and error formatting, see [Output formats & scripting](/docs/datumctl/output-and-scripting). ### Narrow, sort, and watch A few flags make lists easier to work with: ```bash theme={null} # Filter by label selector datumctl get dnszones --project your-project-id -l app=my-app # List across every namespace in scope datumctl get dnszones --organization your-org-id --all-namespaces # Watch for changes and stream updates as they happen datumctl get projects --organization your-org-id --watch ``` *** ## `datumctl describe` — full detail and status Where `get` gives you a concise row, `datumctl describe` prints a detailed, human-readable report for one or more resources, including status conditions and related events where available. Reach for it when a resource isn't reaching a ready state and you need to understand why. ```bash theme={null} # Describe one resource datumctl describe dnszone my-zone --project your-project-id # Describe every resource of a type in a namespace datumctl describe dnszones --project your-project-id # Describe by label selector datumctl describe dnszones -l app=my-app --project your-project-id # Describe resources named in a manifest file datumctl describe -f ./zone.yaml --project your-project-id ``` You can select resources by name, by label selector (`-l`), or from a manifest file (`-f`). Passing a name prefix shows details for every resource whose name starts with that prefix. > [!TIP] > > Use `datumctl get` for a quick inventory and `datumctl describe` when you need the full status picture — conditions, messages, and events that explain what the platform is doing with a resource. *** ## `get` vs. `describe` at a glance | Question | Reach for | | - | - | | What resources exist right now? | `datumctl get ` | | I want the raw definition to inspect or script against | `datumctl get -o yaml` / `-o json` | | Why isn't this resource ready? | `datumctl describe ` | | I want to feed identifiers into another command | `datumctl get -o name` | *** ## Related * [Changing resources](/docs/datumctl/resources/changing) — create, update, and delete the resources you have been reading, with declarative `apply` as the primary workflow * [Discovering resources & schemas](/docs/datumctl/discovering-resources) — find which resource types exist and inspect their fields before you read or write them * [Output formats & scripting](/docs/datumctl/output-and-scripting) — get the most out of `-o json`/`-o yaml`, structured errors, and automation * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — control which organization or project your reads target * [Interactive console](/docs/datumctl/console) — browse and inspect these same resources in a full-screen, keyboard-driven UI # Safe changes & declarative config Source: https://datum.net/docs/datumctl/resources/safe-changes Preview changes before you write them, manage Datum Cloud resources declaratively, and keep your infrastructure in sync from source control. Datum Cloud resources are defined declaratively: you describe the state you want in a manifest, and the platform reconciles reality to match. That model makes changes reviewable and repeatable — but it also means a single `apply` can update live infrastructure. This guide shows how to preview every change before it lands and how to run a safe, GitOps-style loop. > [!WARNING] > > `datumctl delete` has **no confirmation prompt**, and neither `apply` nor `edit` asks you to confirm before writing. Preview first — the habits below exist to catch mistakes before they reach live resources. *** ## Preview before you write Two mechanisms let you see exactly what a change will do before committing it. ### Diff a manifest against live state `datumctl diff -f ` shows the difference between what is currently deployed and what your manifest would apply — the same server-side comparison the platform uses, rendered as a unified diff. ```bash theme={null} # Preview changes from a manifest before applying datumctl diff -f ./proxy.yaml --project your-project-id # Diff from stdin cat dnszone.yaml | datumctl diff -f - --project your-project-id ``` Review the diff, then apply the manifest as a separate step: ```bash theme={null} datumctl apply -f ./proxy.yaml --project your-project-id ``` The diff exits `0` when there are no differences, `1` when there are, and greater than `1` on error. Exit code `1` doesn't mean failure, so don't chain `diff && apply` — the apply would run only when there is nothing to change. In a script, handle each exit code explicitly: ```bash theme={null} datumctl diff -f ./proxy.yaml --project your-project-id case $? in 0) echo "No changes." ;; 1) echo "Changes found. Review the diff above, then run apply." ;; *) echo "diff failed." >&2; exit 1 ;; esac ``` > [!TIP] > > Set `DATUMCTL_EXTERNAL_DIFF` to use a different diff tool, for example `DATUMCTL_EXTERNAL_DIFF="colordiff -N -u"`. ### Validate a write with `--dry-run=server` Every write command accepts `--dry-run=server`, which sends the request to the API server for full validation — including admission checks — without persisting anything. ```bash theme={null} # Validate a manifest against the server without saving datumctl apply -f ./proxy.yaml --project your-project-id --dry-run=server # Same for create datumctl create -f ./project.yaml --organization your-org-id --dry-run=server ``` > [!NOTE] > > `diff` answers *"what will change?"* and `--dry-run=server` answers *"is this change valid?"*. Use both: diff to review the delta, dry-run to confirm the server accepts it. *** ## The declarative loop A repeatable workflow for changing any Datum Cloud resource follows four steps: inspect the schema, preview the change, apply it, then verify the result. **1. Inspect the schema** Confirm the fields a resource type accepts before you author or edit a manifest. ```bash theme={null} datumctl explain httpproxy --recursive ``` See [Discovering resources & schemas](/docs/datumctl/discovering-resources) for finding resource types and reading their fields in depth. **2. Preview the change** Review the delta and validate it against the server. ```bash theme={null} datumctl diff -f ./proxy.yaml --project your-project-id datumctl apply -f ./proxy.yaml --project your-project-id --dry-run=server ``` **3. Apply** Once the preview looks right, apply the manifest. `apply` creates the resource if it does not exist and updates it to match your manifest if it does. ```bash theme={null} datumctl apply -f ./proxy.yaml --project your-project-id ``` **4. Verify** Read the result back to confirm the applied state. ```bash theme={null} datumctl get httpproxy my-proxy --project your-project-id -o yaml ``` Because the desired state lives entirely in your manifest, you can commit it to version control and treat network configuration as code: * Store resource definitions alongside the rest of your infrastructure. * Review changes as diffs in pull requests. * Apply manifests from CI/CD so deployments are automated and auditable. > [!TIP] > > Multiple resources can live in a single file separated by YAML document markers (`---`), and `datumctl apply -f ./dir/` applies every manifest in a directory — so an entire environment can be applied in one command. *** ## Namespaces Some project-scoped resource types live in a namespace. Where a namespace applies, target it with `--namespace`: ```bash theme={null} datumctl apply -f ./proxy.yaml --project your-project-id --namespace default ``` Currently only the `default` namespace is supported, and it is assumed when you omit the flag — so most of the time you can leave `--namespace` off entirely. Which organization or project a command runs against is a separate concern, covered under contexts and scoping; run `datumctl ctx` to see your active context. *** ## Related * [Changing resources](/docs/datumctl/resources/changing) — the `apply`, `create`, `edit`, and `delete` write verbs these previews wrap * [Reading resources](/docs/datumctl/resources/reading) — verify applied state with `datumctl get` and `datumctl describe` * [Discovering resources & schemas](/docs/datumctl/discovering-resources) — find resource types and inspect their fields before authoring manifests * [Contexts & scoping](/docs/datumctl/contexts-and-scoping) — control which organization or project a change targets * [Output formats & scripting](/docs/datumctl/output-and-scripting) — machine-readable output, exit codes, and CI patterns # Desktop Apps Source: https://datum.net/docs/desktop-apps Datum offers native developer apps for macOS, Linux, and Windows that help you safely and quickly expose local environments to the internet using a secure Rust-based tunnel connected to an Application Load Balancer on Datum's edge cloud. The apps use a secure, Rust-based tunnel (built on the [Iroh protocol](https://www.iroh.computer/)) connected to an [Application Load Balancer](/docs/alb/overview) on Datum's edge cloud. * After creating a Tunnel via the desktop app, you'll get a simple HTTPS hostname that you can share with anyone. You can also protect it with a simple username and password combo. * You can create Tunnels in any project, including ones that you share with colleagues (which helps them find your publicly shareable hostnames). ## Installation The desktop apps are available for macOS, Windows, and Linux. Choose your platform below. > [tab] macOS > > Install with [Homebrew](https://brew.sh/): > > ```bash theme={null} > brew install --cask datum-cloud/tap/desktop > ``` > > Or download the [latest macOS installer](https://github.com/datum-cloud/datum-connect/releases/latest/download/Datum.dmg). > [tab] Windows > > Download and run the [latest Windows installer](https://github.com/datum-cloud/app/releases/latest/download/Datum-setup.exe). > [tab] Linux > > Download the [latest Linux AppImage](https://github.com/datum-cloud/app/releases/latest/download/Datum.AppImage), then make it executable and launch it: > > ```bash theme={null} > chmod +x Datum.AppImage > ./Datum.AppImage > ``` After installing, open the app and sign in. Create a Tunnel in any project to expose a local environment with a secure public HTTPS hostname. ## Demo video [YouTube video player](https://www.youtube.com/embed/Flv8rUqR9uU) # Authoritative DNS Source: https://datum.net/docs/domain-dns/dns Datum's authoritative DNS: zones, the record types it supports, including ALIAS, and DNSSEC, managed in the Cloud Portal or with datumctl. > [!NOTE] > > **Working with this resource via an AI agent?** Datum publishes a [DNS skill](/docs/agents/skills/dns) that teaches agents the canonical patterns for this resource. To manage zones and records from the terminal with the `datumctl dns` plugin, see [Operations](/docs/domain-dns/operations). Looking for names that resolve only inside a VPC? See [Private DNS](/docs/domain-dns/private-dns). ## Current features 1. Zone Management – Add DNS zones for any domain and subdomain, whether they are hosted by Datum or externally, for broad visibility across all of your DNS zones. 2. Global Authoritative DNS – Datum serves authoritative DNS through a globally distributed anycast system for performance and redundancy. To use Datum’s authoritative DNS, set up your domain in Datum Cloud ([cloud.datum.net](http://cloud.datum.net) or via *datumctl*) and point it to Datum’s nameservers: * [ns1.datumdomains.net](http://ns1.datumdomains.net) * [ns2.datumdomains.net](http://ns2.datumdomains.net) * [ns3.datumdomains.net](http://ns3.datumdomains.net) * [ns4.datumdomains.net](http://ns4.datumdomains.net) 3. Bulk Import/Export – Import existing zone files (BIND format, screenshot from DNS provider, sync by querying for DNS records) or export current configurations. 4. Record Operations – Add, edit, and delete modern DNS records: A, AAAA, CAA, NS, SRV, TXT, CNAME, MX, SOA, TLSA, SVCB, HTTPS. 5. View Zone Details – Active nameserver assignments; record count and zone size; last modified timestamp. 6. Audit Log – Track all zone and record changes with user attribution and timestamps. 7. Project Scoped – Manage zones within individual projects with role-based access control. > [!WARNING] > > Datum DNS doesn't currently support DNSSEC. If DNSSEC is turned on for your domain, turn it off and wait for the DS record to expire **before** you point your domain at Datum's nameservers. Otherwise your domain stops resolving for most of the internet. See [DNSSEC](#dnssec). ## DNSSEC Zones hosted on Datum aren't signed, and you can't create `DS`, `DNSKEY`, `RRSIG`, `NSEC`, or `NSEC3` records. This matters most when you move an existing domain to Datum. If your registrar still publishes a `DS` (Delegation Signer) record for the domain, validating resolvers (Google Public DNS, Cloudflare 1.1.1.1, Quad9, and most ISP resolvers) expect signed answers. Datum's answers aren't signed, so those resolvers treat them as tampered with and return `SERVFAIL`. To your users, the domain looks like it has gone offline. ### Check whether DNSSEC is on Look for a `DS` record at the parent zone: ```bash theme={null} dig +short DS example.com ``` If this prints nothing, DNSSEC isn't on and you can move to Datum without extra steps. If it prints one or more lines, DNSSEC is on. If you've added the domain to Datum, its registration details also show this. `datumctl describe domain --project your-project-id` reports `status.registration.dnssec.enabled` and any `DS` records the registry has on file. ### Turn off DNSSEC before you switch nameservers Do these steps in order. Keep your current DNS provider serving the zone until the last step. **1. Remove the DS record at your registrar** Turn off DNSSEC, or delete the `DS` record, wherever your domain is registered. If your current DNS provider manages DNSSEC for you (Cloudflare, for example), turn it off there as well. Leave the zone signed at your current provider for now. See [Cloudflare's guide to disabling DNSSEC](https://developers.cloudflare.com/dns/dnssec/) for an example of this flow. **2. Wait for the DS record to disappear** Run `dig +short DS example.com` until it returns nothing. Then wait one more full `DS` TTL (time to live) so cached copies expire. You can see the TTL with `dig DS example.com`. For `.com` and `.net` it's 24 hours, and other TLDs vary. **3. Create the zone on Datum** Add the zone and import your records. If your zone file includes `DS`, `DNSKEY`, `RRSIG`, `NSEC`, or `NSEC3` records, the importer skips them and lists them in a warning. You can ignore that warning. **4. Point your domain at Datum** At your registrar, change the nameservers to `ns1`–`ns4.datumdomains.net`. ### If your domain stopped resolving after the move If the domain resolves when you skip validation but fails normally, a leftover `DS` record is the likely cause: ```bash theme={null} dig example.com # status: SERVFAIL dig +cd example.com # status: NOERROR, with answers ``` Remove the `DS` record at your registrar. Validating resolvers recover as their cached copy of the `DS` record expires, which can take up to the `DS` TTL. ### Subdomains delegated to Datum You can delegate a subdomain (for example `app.example.com`) to Datum even when the parent zone is signed somewhere else. This works as long as the parent zone has no `DS` record for that subdomain. Without one, resolvers treat the subdomain as unsigned and accept Datum's answers. ## DNS record types and behaviors Most record types are standardized (A, AAAA, CNAME, etc.), but some DNS “record-like” features are provider-specific behaviors (for example, “CNAME flattening”). See [ALIAS (“CNAME flattening”)](#alias-%2F-cname-flattening) for details. ## Concepts and definitions 1. **DNS Zone -** A DNS zone is a segment of the Domain Name System that contains the DNS records for one domain or subdomain, managed as a single administrative unit. 2. **DNS Host -** A DNS host is the provider or server that stores and serves the DNS records for your domain, responding to queries from the internet. Some examples of DNS hosts are Datum, Cloudflare, Amazon Route 53, and GoDaddy. 3. **Nameserver -** A nameserver is a specialized DNS server that tells the internet where to find a domain’s DNS Zone and routes queries to the correct host. 4. **DNS Record -** A DNS record is an individual entry within a DNS zone that maps a domain name to a specific resource, such as an IP address, mail server, or another domain. 5. **BIND Format -** BIND format is a standardized text format used to represent DNS zone files, listing all the records and settings that define a domain’s configuration. 6. **Domain Connect -** Domain Connect is an open standard that allows web services and domain registrars to automatically configure DNS settings for users with simple authorization. ## ALIAS / CNAME flattening ### What is an ALIAS record? An **ALIAS** record is a provider-side feature that lets you point a hostname (including the **zone apex**, like `example.com`) at another hostname the way a CNAME does, while still returning **A/AAAA answers** to clients. Different providers use different names for the same idea: * **Cloudflare** commonly describes this as **“CNAME flattening”** * Other DNS providers may call it **ALIAS**, **ANAME**, or **flattened CNAME** ### When to use ALIAS Use ALIAS when you want “point this name at that hostname” behavior but **you can’t (or shouldn’t) use CNAME**, most commonly: * At the **zone apex** (`example.com`) where CNAME is not allowed by standard DNS rules. * When you want the convenience of targeting a hostname that may change IPs (CDNs, hosted services), but still need clients to receive A/AAAA records. ### Expected name and value * **Name**: the hostname inside the zone you’re creating the record for. * For the zone apex, many DNS tools use `@` (BIND/zone-file notation) to mean “the zone root”. * For a subdomain, use the label (for example `www` for `www.example.com`). * **Value**: a **target hostname (FQDN)**, like `myapp.hosting-provider.com`. * ALIAS values should be **hostnames**, not IP addresses. * The target hostname should ultimately resolve to **A and/or AAAA** records (directly or through other DNS indirection). ### How it works ALIAS does **not** exist as a standardized DNS RRType that recursive resolvers understand everywhere. Instead, the DNS provider’s authoritative system does the work: 1. A client’s recursive resolver asks for `A` and/or `AAAA` for the ALIAS name (for example `example.com`). 2. Datum resolves the ALIAS target hostname (for example `myapp.hosting-provider.com`) to its current `A`/`AAAA` records. 3. Datum returns the resulting `A`/`AAAA` answers **as if they were directly configured on the ALIAS name**. Important consequence: clients typically **do not** see a CNAME in the response. They see `A`/`AAAA` records for the name they queried. ### What to expect compared to CNAME * **CNAME**: returns a CNAME response and relies on the resolver to chase it. * **ALIAS / “CNAME flattening”**: returns `A`/`AAAA` directly (synthesized by the DNS provider). ### Notes and limitations * **Portability**: ALIAS/flattening is not uniform across providers. If you move DNS providers, you may need to translate this into whatever equivalent that provider supports. * **Answer types**: ALIAS is primarily about synthesizing `A`/`AAAA` answers. It is not a general replacement for other record types. # Domains Source: https://datum.net/docs/domain-dns/domains Track and verify the domains you own, wherever they are registered. > [!NOTE] > > **Working with this resource via an AI agent?** Datum publishes a [Domains skill](/docs/agents/skills/domains) that teaches agents the canonical patterns for this resource. At Datum, a domain is a resource like any other. By tracking your domains in one place (regardless of where they are registered or where DNS is hosted), you and your colleagues gain programmatic access to a foundational component of your infrastructure. No more spreadsheets! ![Domains Listing](https://mintcdn.com/datum-4926dda5/9pdr37W0ZlTfIFim/images/domains-listing.png?fit=max\&auto=format\&n=9pdr37W0ZlTfIFim\&q=85\&s=280f279b48200127500e4163d824d4df) A list of domains we wished we owned! ## Add a domain You can add and manage domains through the Datum Cloud portal as well as our programmatic interfaces (MCP and datumctl). * Add a single domain - Type in the domain name and a resource is created. * Bulk upload domains - Copy and paste multiple domains or import via a .csv or .txt file. * [Application Load Balancer](/docs/alb/overview) - Adding a custom hostname to an Application Load Balancer will create a domain resource. ![Domain Details](https://mintcdn.com/datum-4926dda5/9pdr37W0ZlTfIFim/images/domain-details.png?fit=max\&auto=format\&n=9pdr37W0ZlTfIFim\&q=85\&s=8e0fbfdafdd1f6f60cb0ce9da33089f1) Details on each domain are provided via public sources ## Verify a domain Once you add a domain as a resource, you can verify ownership. Verified domains become available for use in other Datum services such as Proxy and DNS Zone Hosting. ![Domain Verification](https://mintcdn.com/datum-4926dda5/9pdr37W0ZlTfIFim/images/domain-verification.png?fit=max\&auto=format\&n=9pdr37W0ZlTfIFim\&q=85\&s=27af933fe39ddfc21f9ba2da3696dfdc) Verify ownership by adding a DNS record Datum gives you two ways to prove ownership, and checks both. Each domain's status holds the exact values to use: ```bash theme={null} datumctl get domain DOMAIN_NAME --project your-project-id -o yaml ``` | Method | Status field | What to do | | - | - | - | | DNS record | `status.verification.dnsRecord` (`name`, `type`, `content`) | Create the record exactly as given at the domain's DNS provider. | | HTTP token | `status.verification.httpToken` (`url`, `body`) | Serve `body` at `url`, over plain HTTP, from the domain itself. | Use the DNS record unless you can't change the domain's DNS. The HTTP token only works when the domain already serves your own content: a domain that is parked at its registrar returns the registrar's page, so the token check fails. Datum retries verification on its own, and `status.verification.nextVerificationAttempt` shows when. For records that don't seem to take effect, see [Prove you own the domain](/docs/alb/operations/custom-domains#prove-you-own-the-domain). ## Moving DNS hosting to Datum A domain's registration details include its DNSSEC status as reported by the registry. Datum DNS doesn't currently support DNSSEC. If DNSSEC is on for a domain, turn it off before you point the domain at Datum's nameservers. See [DNSSEC](/docs/domain-dns/dns#dnssec) for the steps. # DNS limitations and roadmap Source: https://datum.net/docs/domain-dns/limitations-and-roadmap Review what Datum Domains and DNS don't support yet and where to follow what comes next. Domains and DNS are still being built. This page lists known limitations and where Datum is headed. ## Limitations * **DNSSEC isn't supported.** Zones hosted on Datum aren't signed. Turn DNSSEC off at your registrar before you move a domain to Datum. See [DNSSEC](/docs/domain-dns/dns#dnssec). * **No DNS query metrics yet.** Datum doesn't publish metrics about the queries it answers, such as query volume or response codes. To check that a zone is live, see [Observability](/docs/domain-dns/observability). * **A zone's domain name can't change.** To use a different domain, create a new zone. * **One zone for each domain.** A domain can belong to only one zone on Datum. * **`ALIAS` isn't portable.** `ALIAS` isn't a standard record type, so a zone file exported from Datum can't carry it to another provider. See [ALIAS](/docs/domain-dns/dns#alias-/-cname-flattening). * **The DNS API is `v1alpha1`.** Field names in `DNSZone` and `DNSRecordSet` can change between releases. The `datumctl dns` plugin keeps up with those changes for you. ## Roadmap Datum acquired CloudValid and is open sourcing its domain validation technology. Datum is building it into Domains and DNS, including: * **Simpler domain verification**, built into every step from adding a domain to serving traffic on it. * **Domain registration**, from search to a working, verified domain. * **Custom domains for SaaS platforms**, combining domains, certificates, and the [Application Load Balancer](/docs/alb/overview) for platforms that serve their customers' domains. * **Email authentication**, making it easier to publish the SPF, DKIM, and DMARC records that inbox providers now require. To learn more, see [Why we're open sourcing domain validation](https://www.datum.net/blog/open-sourcing-domain-validation/). For what Datum is building across the platform, see the [Platform roadmap](/docs/platform/roadmap). > [!NOTE] > > Plans can change. This list isn't a commitment to deliver any feature or to meet any timeline. # DNS limits and quotas Source: https://datum.net/docs/domain-dns/limits-and-quotas The limits and quotas that apply to domains, DNS zones, and records. Documentation is coming soon. > [!NOTE] > > **Coming soon.** Datum is still building Domains and DNS, and documentation for this topic is on the way. ## Follow the roadmap > [card] [Domains and DNS limitations and roadmap](/docs/domain-dns/limitations-and-roadmap) > > See what Domains and DNS doesn't support yet and what Datum is building next. # DNS observability Source: https://datum.net/docs/domain-dns/observability Check whether your DNS zones and records are live, whether your domain is delegated to Datum, and who changed what. This page covers how to see the state of your zones and records: whether Datum is serving them, whether the internet is asking Datum for them, and who changed them. The examples use the `datumctl dns` plugin. To install it, see [Operations](/docs/domain-dns/operations#before-you-begin). ## Is a zone or record live? Every zone and record reports two conditions: | Condition | What it means | | - | - | | `Accepted` | The zone or record is valid, and everything it depends on exists. | | `Programmed` | Datum's authoritative DNS is serving it. | A record is live once it's `Programmed`. To see the status of every record in a zone, run the following command: ```bash theme={null} datumctl dns record list example.com ``` The `STATUS` column shows each value as `Programmed`, `Pending`, `Conflict`, `Not owner`, `Error`, or `Rejected`, with a count below the table. To list only the records that aren't live, filter by status: ```bash theme={null} datumctl dns record list example.com --status pending datumctl dns record list example.com --status error ``` `Rejected` on the records of a new zone usually isn't a refusal. It means the zone's domain isn't verified yet, so Datum is waiting for you to prove you own it. [Verify the domain](/docs/domain-dns/domains#verify-a-domain) and the records move on without being recreated. `Pending` is normal for a few moments after a change. If it lasts, ask why: ```bash theme={null} datumctl dns record describe example.com www ``` `describe` shows each value, its fields, and the reason it is or isn't live. For what each reason means, see [Troubleshooting](/docs/domain-dns/troubleshooting). For zones, `datumctl dns zone list` shows each zone's status and record count, with a summary such as `3 zones — 2 OK, 1 Pending, 0 Rejected, 0 Error`. ## Is the internet asking Datum? A zone can be fully `Programmed` and still not answer for your domain, because your registrar decides which nameservers the internet asks. To check delegation against public DNS, run the following command: ```bash theme={null} datumctl dns zone nameservers example.com --check ``` `--check` queries Datum's nameservers for your zone and asks the public DNS where your domain is delegated, so it reports whether resolution actually works. A delegation of `Complete` means your registrar publishes all of Datum's nameservers. The `DELEGATED` column of `datumctl dns zone list` shows the same for every zone. To see exactly what Datum answers for a name, ask one of Datum's nameservers directly: ```bash theme={null} dig www.example.com @ns1.datumdomains.net ``` To see what the rest of the internet sees, ask a public resolver: ```bash theme={null} dig www.example.com @1.1.1.1 ``` A resolver can keep an old answer until its TTL expires. If Datum's nameserver has the new answer and a public resolver doesn't, wait for the old record's TTL. ## Check for drift If you keep a zone in a BIND zone file, `record apply --dry-run` shows any difference between the file and the live zone without changing anything: ```bash theme={null} datumctl dns record apply example.com -f example.com.zone --dry-run ``` An unchanged zone reports `No changes.` Run it on a schedule, or in CI, to catch changes made outside your file. To capture the live zone as a file, run `datumctl dns zone export example.com`. For zones that you manage with manifests, `datumctl diff -f dns.yaml` does the same. See [Manage DNS with manifests](/docs/domain-dns/operations#manage-dns-with-manifests). ## Who changed what Every change to a zone or record is recorded with who made it and when. To review changes, see [Activity logs](/docs/operations/activity-logs). ## Metrics Datum publishes the state of your zones, records, and domains to your project's metrics store, so you can alert when something stops serving or a domain loses verification. To query them in your own tools, [export them](/docs/operations/metrics-export). | Metric | What it reports | | - | - | | `dns_zone_status_condition` | `1` when a zone condition is true. The `condition` label is `Accepted` or `Programmed`, and `reason` explains it. | | `dns_zone_info` | One series for each zone, with its `domain_name`. | | `dns_record_set_status_condition` | `1` when a record set condition is true, with `condition` and `reason` labels. | | `dns_record_set_info` | One series for each record set, with its `dns_zone_name` and `record_type`. | | `datum_cloud_networking_domain_status_condition` | `1` when a domain condition is true. `condition="Verified"` reports whether you've proven you own the domain. | | `datum_cloud_networking_domain_info` | One series for each domain, with its `domain_name`. | | `datum_cloud_networking_domain_status_next_verification_attempt` | When Datum will next check a domain's verification, as a Unix timestamp. | On these series, `resource_name` is the zone, record set, or domain name, and `resourcemanager_datumapis_com_project_name` is your project ID. To find record sets that Datum isn't serving, replace `PROJECT_ID`: ```promql theme={null} dns_record_set_status_condition{resourcemanager_datumapis_com_project_name="PROJECT_ID", condition="Programmed"} == 0 ``` To find domains that aren't verified: ```promql theme={null} datum_cloud_networking_domain_status_condition{resourcemanager_datumapis_com_project_name="PROJECT_ID", condition="Verified"} == 0 ``` ## ExternalDNS webhook metrics If you use [ExternalDNS](/docs/domain-dns/operations#automate-records-with-externaldns) to manage records, the Datum webhook exposes Prometheus metrics at `/metrics` on its metrics port (`8080` by default): | Metric | Type | What it counts | | - | - | - | | `datum_dns_zones_discovered` | Gauge | `DNSZone` resources the webhook found | | `datum_dns_recordsets_managed` | Gauge | `DNSRecordSet` resources the webhook manages | | `datum_dns_operations_total` | Counter | Record creates, updates, and deletes | | `datum_dns_translation_errors_total` | Counter | ExternalDNS records the webhook couldn't translate | | `datum_dns_ownership_conflicts_total` | Counter | Records owned by someone else that the webhook left alone | | `datum_dns_http_requests_total` | Counter | Requests from ExternalDNS to the webhook | | `datum_dns_http_request_duration_seconds` | Histogram | How long those requests took | The same port serves `/healthz` for liveness and `/readyz`, which reports ready once the webhook has discovered your zones. ## Query metrics The metrics above report the state of your zones and records. Datum doesn't publish metrics about the DNS queries it answers yet, such as query volume or response codes. See [Limitations and roadmap](/docs/domain-dns/limitations-and-roadmap). # DNS operations Source: https://datum.net/docs/domain-dns/operations Create, update, import, export, and delete DNS zones and records with the datumctl dns plugin, manifests, or ExternalDNS. This page covers day-to-day work with DNS zones and records. Most examples use the `datumctl dns` plugin, which is the quickest way to manage DNS from a terminal. Every zone and record is also a resource in your project, so you can manage them with manifests and GitOps as well. See [Manage DNS with manifests](#manage-dns-with-manifests). ## Before you begin * Install `datumctl` and log in. For more information, see [Install](/docs/datumctl/install). * Select a project with `datumctl ctx use your-org-id/your-project-id`, or add `--project your-project-id` to each command. * Install the `dns` plugin from the official catalog: ```bash theme={null} datumctl plugin install dns datumctl dns version ``` `datumctl dns version` needs no login, project, or network, so it's a good first check when something else fails. * Verify that you own the domain. Datum serves a zone only for a verified domain. For more information, see [Verify a domain](/docs/domain-dns/domains#verify-a-domain). If a command reports that DNS isn't enabled for your project, enable it. To see the services available to your project, run `datumctl services list`. ```bash theme={null} datumctl services enable dns.networking.miloapis.com --wait ``` ## Create a zone A zone holds the records for one domain. To create one, run the following command, replacing `example.com` with a domain that you control: ```bash theme={null} datumctl dns zone create example.com ``` The command waits up to 2 minutes for Datum to assign nameservers, then prints them. If the domain isn't verified yet, Datum still creates the zone, but assigns no nameservers and reports `Waiting for domain ownership verification before provisioning DNS`. Records that you add stay `Rejected` until you [verify the domain](/docs/domain-dns/domains#verify-a-domain). Add `--no-wait` to return as soon as the zone exists, or `--description "production apex"` to label it. > [!WARNING] > > You can't change a zone's domain name after you create it. To use a different domain, create a new zone. ## Point your domain at Datum Your domain doesn't resolve through Datum until your registrar publishes Datum's nameservers. To see the nameservers assigned to your zone, run the following command: ```bash theme={null} datumctl dns zone nameservers example.com ``` Set those nameservers at your registrar, then check delegation against public DNS: ```bash theme={null} datumctl dns zone nameservers example.com --check ``` `--check` asks the public DNS what your domain actually delegates to, so it tells you whether delegation works rather than what Datum expects. | Delegation | What it means | | - | - | | `Complete` | Your registrar publishes every assigned nameserver. | | `Partial` | Your registrar publishes some of them, usually from a half-finished edit. | | `Incomplete` | Your registrar publishes none of them. | | `Unknown` | There's nothing to compare yet. This is normal for the first few minutes after you create a zone. | Registrar changes take time to spread. Allow for the parent zone's TTL before you treat a change as failed. > [!WARNING] > > Datum DNS doesn't support DNSSEC yet. If DNSSEC is on for your domain, turn it off and wait for the `DS` record to expire before you change nameservers. Otherwise your domain stops resolving for most of the internet. See [DNSSEC](/docs/domain-dns/dns#dnssec). ## Add and change records Give the zone, the name, the type, and one or more values. Names are relative to the zone: `www`, `*`, `_dmarc`, or `@` for the domain itself. ```bash theme={null} datumctl dns record create example.com www A 203.0.113.10 ``` `create` adds to the values already at that name. `set` replaces everything at that name: ```bash theme={null} datumctl dns record create example.com www A 203.0.113.11 # www now has two addresses datumctl dns record set example.com www A 203.0.113.20 # www now has only this one ``` Add `--wait` to wait until the record is live, or `--dry-run` to check a command without changing anything. Simple types take their value as an argument. Types with several parts take named flags: ```bash theme={null} datumctl dns record set example.com @ TXT "v=spf1 include:_spf.example.com ~all" datumctl dns record create example.com cdn CNAME lb.example.net. datumctl dns record create example.com @ MX --preference 10 --exchange mail.example.com. datumctl dns record create example.com @ CAA --flag 0 --tag issue --value letsencrypt.org datumctl dns record create example.com _sip._tcp SRV --priority 10 --weight 5 --port 5060 --target sip.example.com. datumctl dns record create example.com api HTTPS --priority 1 --target . --param alpn=h3,h2 ``` You can also paste a record in zone-file form, as it appears in `dig` output or a provider export: ```bash theme={null} datumctl dns record create example.com --line "www 300 IN A 203.0.113.10" ``` For long TXT values such as DKIM keys, `--data` reads a file with `@path`, or standard input with `-`: ```bash theme={null} datumctl dns record create example.com selector1._domainkey TXT --data @dkim.txt ``` Two rules catch most mistakes: * **Names are relative.** Use `www`, not `www.example.com`. Use `@` for the domain itself. * **Targets are absolute.** End every hostname inside a value with a dot: `mail.example.com.`, not `mail`. For the record types Datum supports, see [Authoritative DNS](/docs/domain-dns/dns). To point a name at a load balancer, see [Point a domain at a load balancer](/docs/domain-dns/point-to-alb). ### Set a TTL `--ttl` takes seconds or a duration, such as `--ttl 300`, `--ttl 5m`, or `--ttl 1h30m`. A record without `--ttl` uses `Auto`, which is 5 minutes. ## List and inspect To list your zones, run the following command: ```bash theme={null} datumctl dns zone list ``` Each zone shows its status, record count, nameservers, and whether it's delegated to Datum. Filter with `--status ok|pending|error`. To see a zone's delegation, a count of its records by type, and suggested next steps, run the following command: ```bash theme={null} datumctl dns zone describe example.com ``` To list a zone's records, one row per value, run the following command: ```bash theme={null} datumctl dns record list example.com ``` Filter with `--type A,MX`, `--name www`, or `--status programmed|pending|conflict|not-owner|error|rejected`. Add `--no-managed` to hide the records Datum manages for you and see only your own. To see one name's values, fields, and why each one is or isn't live, run the following command: ```bash theme={null} datumctl dns record describe example.com www ``` ## Delete records and zones To remove one value, pass it. To remove everything at a name and type, leave the value off: ```bash theme={null} datumctl dns record delete example.com www A 203.0.113.11 datumctl dns record delete example.com www A ``` To delete a zone, run the following command: ```bash theme={null} datumctl dns zone delete example.com ``` > [!WARNING] > > Deleting a zone deletes every record in it, and the domain stops resolving through Datum. Destructive commands ask you to confirm by typing the name. In a script, pass `--yes`. Without a terminal and without `--yes`, they refuse to run. ## Move a zone to Datum To bring a zone from another provider, import a zone file you exported there: ```bash theme={null} datumctl dns zone import example.com --file example.com.zone --dry-run datumctl dns zone import example.com --file example.com.zone ``` Add `--replace` to overwrite the existing records of each type in the file instead of adding to them. If you don't have a zone file, take a snapshot of what the domain resolves to today: ```bash theme={null} datumctl dns zone import example.com --discover ``` The importer skips `DS`, `DNSKEY`, `RRSIG`, `NSEC`, and `NSEC3` records and lists them in a warning. That's expected, because Datum DNS doesn't support DNSSEC yet. Before you import, review what doesn't carry over cleanly: * **TTLs.** The importer takes each TTL exactly as the file writes it, including any long TTL you lowered for the migration. Check them after the import. * **Apex CNAME records.** Standard DNS doesn't allow a CNAME at the zone apex. Some providers accept one through a proprietary extension. Recreate it as an [ALIAS record](/docs/domain-dns/dns#what-is-an-alias-record). * **ALIAS, ANAME, and flattened CNAME records.** Providers name these differently, and a BIND zone file can't represent them. Recreate them by hand as ALIAS records. * **Provider-specific record types.** Redirect records and pointers to provider-hosted services aren't DNS records. Rebuild that behavior with a service such as an [Application Load Balancer](/docs/alb/overview). * **The old SOA and apex NS records.** They name your old provider's nameservers. Leave them out, because [Datum manages them](#records-that-datum-manages) for each zone. When the records look right, [point your domain at Datum](#point-your-domain-at-datum). ## Export a zone and keep it in sync To write a zone out in BIND format, run the following command: ```bash theme={null} datumctl dns zone export example.com > example.com.zone ``` `ALIAS` isn't a standard record type, so other providers and BIND tools reject those lines. The command warns about them. To make a zone match a zone file, use `record apply`. Run it with `--dry-run` first to see what would change: ```bash theme={null} datumctl dns record apply example.com -f example.com.zone --dry-run datumctl dns record apply example.com -f example.com.zone ``` By default, `apply` only adds and updates records. Add `--prune` to also delete records that the file doesn't mention. An unchanged file reports `No changes.`, so `apply --dry-run` also works as a drift check. See [Observability](/docs/domain-dns/observability#check-for-drift). ## Records that Datum manages Datum manages some records for you: each zone's `SOA` and apex `NS` records, and the records that an [Application Load Balancer](/docs/alb/overview) creates for its hostnames. `record apply` never changes or prunes them, and if you edit one by hand, Datum changes it back. To see them, run `datumctl dns record list example.com --managed`. ## Script with datumctl dns Every command accepts `-o table|wide|json|yaml|name`. Data goes to standard output and everything else to standard error, so `-o json > file.json` is always clean: ```bash theme={null} datumctl dns zone list -o json | jq -r '.items[].spec.domainName' datumctl dns zone list -o name | xargs -n1 datumctl dns zone describe ``` The table view can change between releases. Script against `-o json` or `-o name`. Scripts can rely on the exit codes: | Code | Name | Meaning | | - | - | - | | 0 | — | Success | | 1 | `DNS_ERROR` | Unexpected failure | | 2 | `DNS_USAGE` | Bad flags, arguments, or record values | | 3 | `DNS_FORBIDDEN` | Not authorized, or DNS isn't enabled for the project | | 4 | `DNS_NOT_FOUND` | Zone or record not found | | 5 | `DNS_CONFLICT` | Something else already owns that name | | 6 | `DNS_INVALID` | Datum rejected the request | | 8 | `DNS_UNAVAILABLE` | Can't reach the DNS API | | 9 | `DNS_ABORTED` | You declined a confirmation | A bulk operation that partly fails never exits `0`. ## Manage DNS with manifests Zones and records are `DNSZone` and `DNSRecordSet` resources in the `dns.networking.miloapis.com/v1alpha1` API, so you can keep them in Git and apply them with `datumctl` or a [GitOps tool](/docs/datumctl/gitops): ```yaml theme={null} apiVersion: dns.networking.miloapis.com/v1alpha1 kind: DNSZone metadata: name: example-com spec: domainName: example.com dnsZoneClassName: datum-external-global-dns --- apiVersion: dns.networking.miloapis.com/v1alpha1 kind: DNSRecordSet metadata: name: www-a spec: dnsZoneRef: name: example-com recordType: A records: - name: www ttl: 300 a: content: "203.0.113.10" ``` To see the zone classes available to your project, run `datumctl get dnszoneclasses`. To preview and apply the manifests, run the following commands: ```bash theme={null} datumctl diff -f dns.yaml datumctl apply -f dns.yaml ``` Each entry in `records` holds one value. To give a name several addresses, add one entry per value. ## Automate records with ExternalDNS If you run Kubernetes clusters of your own, [ExternalDNS](https://github.com/kubernetes-sigs/external-dns) can create Datum DNS records for your services and routes automatically. Datum's [ExternalDNS webhook provider](https://github.com/datum-cloud/external-dns-webhook) runs as a sidecar next to ExternalDNS and turns its records into `DNSRecordSet` resources in your project. The webhook: * Manages only hostnames that fall inside one of your `DNSZone` resources. * Sends each record to the most specific matching zone, so `app.sub.example.com` goes to the `sub.example.com` zone rather than `example.com`. * Labels each record with an owner ID, and changes only the records it owns. Records you create by hand, and records from another ExternalDNS instance, are left alone. To install it, deploy ExternalDNS with the webhook provider set to `ghcr.io/datum-cloud/external-dns-webhook`, and give it a unique `--owner-id`. The [webhook's README](https://github.com/datum-cloud/external-dns-webhook#installation) has a Helm example and the required permissions. To give it credentials for your project, use a [service account](/docs/projects/service-accounts) with a kubeconfig from `datumctl auth update-kubeconfig`. See [Using datumctl with kubectl](/docs/datumctl/kubectl-interop). # Domains and DNS Source: https://datum.net/docs/domain-dns/overview Track and verify domains, host authoritative DNS, and resolve private names inside Datum VPCs. Datum treats domains and DNS zones as resources in your project. You can track and verify domains, host public zones on Datum's global anycast network, and use Private DNS to resolve resources inside your VPCs. > [card] [Domains](/docs/domain-dns/domains) > > Track domains wherever they are registered, and verify ownership to use them with other Datum services. > [card] [Authoritative DNS](/docs/domain-dns/dns) > > Host authoritative DNS for your zones on Datum's global anycast network. > [card] [Private DNS](/docs/domain-dns/private-dns) > > Resolve private resource and service names inside your VPCs. ## How it works ```mermaid theme={null} flowchart LR Y["You
Cloud Portal · datumctl dns
GitOps · ExternalDNS"] subgraph proj["Your project"] Z["DNSZone
example.com"] R["DNSRecordSet
www A 203.0.113.10"] end A["Datum authoritative DNS
ns1–ns4.datumdomains.net"] Q["Resolvers worldwide"] Y --> Z Y --> R Z --> A R --> A Q -->|"DNS queries · anycast"| A classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 class Y,Z,R,A,Q box style proj fill:#EDEDEC,stroke:#8A8E99,color:#B5605E linkStyle 0,1,2,3,4 stroke:#5A5F6B,stroke-width:1.4px ``` * **Zones and records are resources.** A `DNSZone` holds one domain, and each `DNSRecordSet` holds the records at a name in that zone. They live in your project, with the same access control and [activity logs](/docs/operations/activity-logs) as everything else. * **Datum serves them worldwide.** Datum programs your records into its authoritative DNS and publishes them to nameservers on its global anycast network. Each zone and record reports `Programmed` once Datum is serving it. * **Your registrar points the internet at Datum.** Your domain resolves through Datum once your registrar lists Datum's nameservers. ## Ways to manage DNS * **[Cloud Portal](https://cloud.datum.net)** for zones, records, and zone file import and export. * **The [`datumctl dns` plugin](/docs/domain-dns/operations)** for the terminal and scripts, including delegation checks, zone file import, and drift checks. * **Manifests and [GitOps](/docs/datumctl/gitops)**, to keep zones and records in Git. See [Manage DNS with manifests](/docs/domain-dns/operations#manage-dns-with-manifests). * **[ExternalDNS](/docs/domain-dns/operations#automate-records-with-externaldns)**, to create records automatically for services in your own Kubernetes clusters. * **AI agents**, with [Datum MCP](/docs/agents/datum-mcp) or the [DNS skill](/docs/agents/skills/dns). Domains and DNS work together with the [Application Load Balancer](/docs/alb/overview): a verified domain can serve traffic on a load balancer, and a Datum-hosted zone can point at it. See [Point a domain at a load balancer](/docs/domain-dns/point-to-alb). ## Get started > [card] [Move a zone to Datum](/docs/domain-dns/operations#move-a-zone-to-datum) > > Import your records, check DNSSEC, and point your domain at Datum. > [card] [Check that DNS is live](/docs/domain-dns/observability) > > See whether records are serving and your domain is delegated. # Point a domain at a load balancer Source: https://datum.net/docs/domain-dns/point-to-alb Use Datum DNS to point your domain, and its subdomains, at an Application Load Balancer with an apex ALIAS record and a CNAME. When Datum hosts your domain's DNS, you can point the domain at an [Application Load Balancer](/docs/alb/overview) with two records: * An `ALIAS` record at the apex, `example.com`. A `CNAME` isn't allowed at the apex, so Datum answers `ALIAS` queries with the load balancer's current addresses. See [ALIAS](/docs/domain-dns/dns#alias-/-cname-flattening). * A `CNAME` record for each subdomain, such as `www.example.com`. Both point at the load balancer's generated hostname, such as `famous-tree-abc12.datumproxy.net`. To serve traffic for your domain, the load balancer also needs the domain as a [custom hostname](/docs/alb/operations/custom-domains). > [!NOTE] > > If your DNS is hosted somewhere else, you don't need anything on this page. Create a `CNAME` to the load balancer's generated hostname at your own DNS provider. ## Point a domain with datumctl dns The quickest way is the `datumctl dns` plugin. To install it, run `datumctl plugin install dns`. **1. Find the load balancer's hostname** ```bash theme={null} datumctl alb describe ALB_NAME ``` The `Default hostname` line shows the generated hostname. **2. Create the zone, if you haven't already** ```bash theme={null} datumctl dns zone create example.com ``` Then point your domain at Datum's nameservers. See [Point your domain at Datum](/docs/domain-dns/operations#point-your-domain-at-datum). **3. Add the records** ```bash theme={null} datumctl dns record set example.com @ ALIAS GENERATED_HOSTNAME. datumctl dns record set example.com www CNAME GENERATED_HOSTNAME. ``` Replace `GENERATED_HOSTNAME` with the load balancer's hostname, and keep the trailing dot. **4. Check that it works** ```bash theme={null} datumctl dns record list example.com --type ALIAS,CNAME dig +short www.example.com @ns1.datumdomains.net curl -sI https://www.example.com/ ``` The records are live when their status is `Programmed`. ## Point a domain with manifests The rest of this page creates the same zone and records with `DNSZone` and `DNSRecordSet` manifests, using `datumctl apply`. ### Prerequisites * `datumctl` installed and authenticated * A valid **Project** * Domain delegated to Datum nameservers *** ### Configuration steps ### Step 1: Set variables The `TARGET` value is your Application Load Balancer endpoint hostname. Find it in the Datum portal under your Application Load Balancer's generated hostname, or via: ```bash theme={null} datumctl get httpproxy --namespace default -o yaml ``` > [tab] macOS / Linux > > ```bash theme={null} > PROJECT="your-project-id" > NAMESPACE="default" > ZONE="your-zone-name" > DOMAIN="your-domain.example.com" > TARGET="your-endpoint.datumproxy.net." > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > $PROJECT = "your-project-id" > $NAMESPACE = "default" > $ZONE = "your-zone-name" > $DOMAIN = "your-domain.example.com" > $TARGET = "your-endpoint.datumproxy.net." > ``` > [!WARNING] > > `TARGET` must include a trailing dot. Without it, the value will be treated as a relative name and DNS resolution will fail. *** ### Step 2: Create the DNS zone > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: dns.networking.miloapis.com/v1alpha1 > kind: DNSZone > metadata: > name: $ZONE > namespace: $NAMESPACE > spec: > dnsZoneClassName: datum-external-global-dns > domainName: $DOMAIN > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: dns.networking.miloapis.com/v1alpha1 > kind: DNSZone > metadata: > name: $ZONE > namespace: $NAMESPACE > spec: > dnsZoneClassName: datum-external-global-dns > domainName: $DOMAIN > "@ | datumctl apply -f - --project $PROJECT --validate=false > ``` > [!NOTE] > > `--validate=false` disables client-side schema validation. It is required here because `datumctl` does not bundle schemas for DNS API types locally. Verify the zone is accepted and programmed: ```bash theme={null} datumctl get dnszones --project $PROJECT --namespace $NAMESPACE ``` Wait until both `ACCEPTED` and `PROGRAMMED` show `True` before proceeding. *** ### Step 3: Create the apex ALIAS record Points the root domain (`@`) at the Application Load Balancer endpoint. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: dns.networking.miloapis.com/v1alpha1 > kind: DNSRecordSet > metadata: > name: ${ZONE}-apex > namespace: $NAMESPACE > spec: > dnsZoneRef: > name: $ZONE > recordType: ALIAS > records: > - name: "@" > ttl: 300 > alias: > content: $TARGET > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: dns.networking.miloapis.com/v1alpha1 > kind: DNSRecordSet > metadata: > name: ${ZONE}-apex > namespace: $NAMESPACE > spec: > dnsZoneRef: > name: $ZONE > recordType: ALIAS > records: > - name: "@" > ttl: 300 > alias: > content: $TARGET > "@ | datumctl apply -f - --project $PROJECT --validate=false > ``` *** ### Step 4: Create a subdomain CNAME Points a subdomain (e.g., `app.your-domain.example.com`) at the same endpoint. > [tab] macOS / Linux > > ```bash theme={null} > cat < apiVersion: dns.networking.miloapis.com/v1alpha1 > kind: DNSRecordSet > metadata: > name: ${ZONE}-app-cname > namespace: $NAMESPACE > spec: > dnsZoneRef: > name: $ZONE > recordType: CNAME > records: > - name: "app" > ttl: 300 > cname: > content: $TARGET > EOF > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > @" > apiVersion: dns.networking.miloapis.com/v1alpha1 > kind: DNSRecordSet > metadata: > name: ${ZONE}-app-cname > namespace: $NAMESPACE > spec: > dnsZoneRef: > name: $ZONE > recordType: CNAME > records: > - name: "app" > ttl: 300 > cname: > content: $TARGET > "@ | datumctl apply -f - --project $PROJECT --validate=false > ``` *** ## Verification ### Check record status ```bash theme={null} datumctl get dnsrecordsets --project $PROJECT --namespace $NAMESPACE ``` Both records should show `ACCEPTED=True` and `PROGRAMMED=True`. *** ### Check DNS resolution > [tab] macOS / Linux > > ```bash theme={null} > dig $DOMAIN > dig app.$DOMAIN > ``` > > Both names should resolve to the Application Load Balancer endpoint. > [tab] Windows (PowerShell) > > ```powershell theme={null} > Resolve-DnsName $DOMAIN > Resolve-DnsName "app.$DOMAIN" > ``` *** ## Cleanup > [tab] macOS / Linux > > ```bash theme={null} > datumctl delete dnsrecordset ${ZONE}-app-cname \ > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > > datumctl delete dnsrecordset ${ZONE}-apex \ > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > > datumctl delete dnszone $ZONE \ > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` > [tab] Windows (PowerShell) > > ```powershell theme={null} > datumctl delete dnsrecordset ${ZONE}-app-cname ` > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > > datumctl delete dnsrecordset ${ZONE}-apex ` > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > > datumctl delete dnszone $ZONE ` > --project $PROJECT --namespace $NAMESPACE --ignore-not-found > ``` *** ## Troubleshooting | Symptom | Root Cause | Resolution | | - | - | - | | Zone stuck at `PROGRAMMED=False` | Most often the domain isn't verified yet, which also leaves the zone's records `Rejected`. Delegation doesn't cause this: Datum provisions a zone whether or not your registrar points at it | [Verify the domain](/docs/domain-dns/domains#verify-a-domain), then wait for the zone to be programmed | | `nslookup` / `dig` returns NXDOMAIN | Zone not yet propagated | Wait 1–2 minutes and retry | | ALIAS resolves to wrong address | `TARGET` missing trailing dot | Re-apply the record with a trailing dot on the target | | Record not appearing | `--validate=false` omitted | Re-apply with `--validate=false` | *** ## Summary * DNS zones use `kind: DNSZone` at `dns.networking.miloapis.com/v1alpha1` * The DNS zone class is always `datum-external-global-dns` — it is platform-provided and not user-configurable * Use `recordType: ALIAS` for the apex (`@`) and `recordType: CNAME` for subdomains * The `TARGET` value must include a trailing dot * All DNS apply commands require `--validate=false` * Delete record sets before deleting the zone # Private DNS Source: https://datum.net/docs/domain-dns/private-dns Resolve private service and resource names within a Datum VPC. Private DNS gives resources in a Datum VPC names that resolve only from that VPC. It provides a consistent discovery layer for private addresses without requiring every product or workload to run its own DNS server. > [!NOTE] > > Private DNS is under active development. Availability and supported service > integrations may vary while the feature is in preview. ## What you can do With Private DNS, you can: * **Reach resources by name:** Use DNS names instead of private IP addresses that may change over a resource's lifetime. * **Use automatic service names:** Integrated Datum services publish names for resources such as Compute instances. * **Create private namespaces:** Organize internal applications under private zones that are available only to associated VPCs. * **Reuse internal names safely:** Different VPCs can use the same names and overlapping private addresses without exposing answers to each other. * **Keep records aligned with resources:** Service-managed records update or disappear as their resources and addresses change. Applications use ordinary DNS lookups. When they run in an associated VPC, private names resolve alongside public names without application-specific resolver configuration. ## Private and public DNS Datum offers two complementary DNS capabilities: | Capability | Where names resolve | Typical use | | - | - | - | | [Authoritative DNS](/docs/domain-dns/dns) | The public internet | Websites, public APIs, email, and domain records | | Private DNS | Inside an associated VPC | Private instances, internal services, and service discovery | Private DNS records are not published to the public DNS. Creating a private name does not make a resource reachable from the internet or grant network access to it. ## Names and zones Each VPC receives a managed DNS suffix for automatic service names. You can also associate private zones with a VPC for organization-specific names. * **Managed names:** Datum services publish predictable names beneath the VPC's managed suffix. * **Private zones:** Your internal names are visible only from the VPCs with which the zone is associated. The complete service-managed name includes the VPC's allocated suffix. Use the name reported by the service instead of constructing it in application code. ## Service integrations The resource name, published record types, and eligibility rules are defined by the service that owns the resource. > [card] [Compute](/docs/compute/private-dns) > > Learn how Compute publishes stable private names for instance addresses. Additional Datum services will be documented here as Private DNS integrations become available. ## What Private DNS does not do * **Provide public discovery:** Private names are not available through public DNS. * **Grant network access:** Routing and access policy still determine whether a client can reach the returned address. * **Replace application health checks:** A name can resolve while the application itself is unavailable. * **Remove cached answers immediately:** Clients may retain an older answer until its DNS TTL expires. # DNS troubleshooting Source: https://datum.net/docs/domain-dns/troubleshooting Fix common problems with Datum DNS zones, records, delegation, and DNSSEC. This page covers common DNS problems and how to fix them. The examples use the `datumctl dns` plugin. To install it, see [Operations](/docs/domain-dns/operations#before-you-begin). Add `--verbose` to any `datumctl dns` command to see the underlying cause of an error. ## My domain doesn't resolve Work through these checks in order. 1. **Check delegation.** Run `datumctl dns zone nameservers example.com --check`. If delegation is `Incomplete` or `Partial`, set Datum's nameservers at your registrar. If it's `Unknown`, the zone is new; wait a few minutes and check again. 2. **Check for DNSSEC.** If the domain resolves with `dig +cd example.com` but returns `SERVFAIL` with `dig example.com`, a leftover `DS` record at your registrar is the cause. Datum DNS doesn't support DNSSEC yet. Remove the `DS` record at your registrar, and resolvers recover as their cached copy expires. See [DNSSEC](/docs/domain-dns/dns#dnssec). 3. **Check the record.** Run `datumctl dns record list example.com --name www`. If the record isn't there, or isn't `Programmed`, see the sections below. 4. **Check Datum's answer.** Run `dig www.example.com @ns1.datumdomains.net`. If Datum answers correctly but your resolver doesn't, it's holding an old answer. Wait for the old record's TTL. ## A record is stuck at Pending `Pending` is normal for a few moments after a change. If it lasts, ask Datum why: ```bash theme={null} datumctl dns record describe example.com www ``` `describe` shows the reason for each value. Use the reason to find the fix in the table below. ## A record reports Conflict or Not owner | Status | What it means | What to do | | - | - | - | | `Conflict` | Another record already occupies that name. | The name is usually outside the zone. Check for `www.example.com` where you meant `www`: names are relative to the zone. | | `Not owner` | Another record set already owns that name. `describe` names it. | Change the record through the record set that owns it, or delete that record set first. | Records that Datum manages for you, such as a zone's `SOA` and apex `NS` records and the records an [Application Load Balancer](/docs/alb/overview) creates, change back if you edit them by hand. To see which records Datum manages, run `datumctl dns record list example.com --managed`. ## I can't create a zone for my domain A domain can belong to only one zone on Datum. If another zone already claims the domain, the new zone isn't accepted. Check whether the zone already exists in another of your projects. If you can't find it, contact [support@datum.net](mailto:support@datum.net). A zone's domain name can't be changed after you create it. To use a different domain, create a new zone. ## My zone has no nameservers If `datumctl dns zone create` reports `Waiting for domain ownership verification before provisioning DNS`, the zone exists, but Datum won't serve it until you prove you own the domain. Until then, the zone has no nameservers and its records stay `Rejected`. You don't need to recreate the zone. [Verify the domain](/docs/domain-dns/domains#verify-a-domain), and Datum provisions the zone and its records. ## DNS isn't enabled for my project If a command fails with exit code `3` (`DNS_FORBIDDEN`), either your account isn't allowed to manage DNS in the project, or DNS isn't enabled for it. To enable DNS, run the following command: ```bash theme={null} datumctl services enable dns.networking.miloapis.com --wait ``` If DNS is already enabled, ask your project's administrator for access. ## No project is set The `dns` plugin never guesses a project. Run `datumctl ctx use your-org-id/your-project-id`, or add `--project your-project-id` to the command. ## Import skipped some records When you import a zone, the importer skips `DS`, `DNSKEY`, `RRSIG`, `NSEC`, and `NSEC3` records and lists them in a warning. That's expected, because Datum DNS doesn't support DNSSEC yet. Before you move your domain to Datum, turn off DNSSEC at your registrar. See [DNSSEC](/docs/domain-dns/dns#dnssec). ## A record that points at a load balancer doesn't work See [Point a domain at a load balancer](/docs/domain-dns/point-to-alb#troubleshooting). ## ExternalDNS doesn't create records If you use [ExternalDNS](/docs/domain-dns/operations#automate-records-with-externaldns) with the Datum webhook: * The webhook manages only hostnames inside one of your `DNSZone` resources. Create the zone first. * The webhook only changes records labeled with its own `--owner-id`. A record created by hand, or by another ExternalDNS instance, isn't changed. The `datum_dns_ownership_conflicts_total` metric counts these. * The webhook needs permission to read `dnszones` and to manage `dnsrecordsets` in your project. For the webhook's metrics, see [Observability](/docs/domain-dns/observability#externaldns-webhook-metrics). # gVPC configuration Source: https://datum.net/docs/galactic-vpc/configuration Create a gVPC network, attach a Compute workload to it, and keep Instance addresses. To learn how networks relate to projects, locations, and Instances, see [Networks overview](/docs/galactic-vpc/networks/overview). To view, change, or delete a network, see [Operations](/docs/galactic-vpc/operations). > [!NOTE] > > [gVPC](/docs/galactic-vpc/overview) is in preview, and the `networking.datumapis.com/v1alpha` API can change. ## Before you begin * Install `datumctl`, log in, and select a project. For more information, see [Install datumctl](/docs/datumctl/install). * To create, change, or delete networks, you need the **Network Admin** role in the project. The same role lets you attach workloads to a network. For more information, see [Permissions](#permissions). ## Create a network To create a network, follow these steps: 1. Save the following manifest as `network.yaml`: ```yaml theme={null} apiVersion: networking.datumapis.com/v1alpha kind: Network metadata: name: backend spec: ipam: mode: Auto ``` `spec.ipam.mode: Auto` tells Datum to assign the network's address space. `ipam.mode` is the only field that you need to set. 2. To create the network, run the following command: ```bash theme={null} datumctl apply -f network.yaml ``` 3. To check that the network is ready, run the following command: ```bash theme={null} datumctl get networks ``` When the network is ready, the `IPV6PREFIX` column shows the private IPv6 prefix that Datum assigned, and the `READY` column shows `True`. A network has two optional settings: `spec.ipFamilies` defaults to `[IPv6]`, and `spec.mtu` defaults to `1440`. Keep the defaults unless you have a reason to change them. Datum rejects a network that doesn't include IPv6, and a larger MTU can cause connections between locations to stall. To create a network with the `network` plugin instead of a manifest, run the following command: ```bash theme={null} datumctl network create backend --wait ``` The plugin creates the same IPv6-only network. To install the plugin, see [Operations](/docs/galactic-vpc/operations#install-the-plugin). ## Attach a workload to a network A workload chooses its network when you create it. You can't move a workload to a different network later. To attach a workload with `datumctl`, pass `--network` when you create it: ```bash theme={null} datumctl compute deploy WORKLOAD_NAME \ --image=IMAGE \ --location=LOCATION \ --network=NETWORK_NAME ``` Replace the following: * `WORKLOAD_NAME`: a name for your workload. * `IMAGE`: the full image reference, including the registry host. * `LOCATION`: one or more locations, separated by commas. * `NETWORK_NAME`: the name of the network, such as `backend`. If you omit `--network`, the command attaches the workload to the `default` network. For more information, see [The default network](/docs/galactic-vpc/networks/overview#the-default-network). To attach a workload in a manifest, set `network.name` in the workload's only entry under `spec.template.spec.networkInterfaces`. The following workload runs in Dallas and Ashburn on the `backend` network: ```yaml theme={null} apiVersion: compute.datumapis.com/v1alpha kind: Workload metadata: name: api spec: template: spec: runtime: class: general-purpose resources: instanceType: datumcloud/d1-standard-2 sandbox: containers: - name: api image: ghcr.io/example/api:1.4.2 ports: - name: http port: 8080 networkInterfaces: - network: name: backend placements: - name: dallas locations: - name: us-central-1 scaleSettings: minReplicas: 1 - name: ashburn locations: - name: us-east-1 scaleSettings: minReplicas: 1 ``` If the network doesn't exist, the workload reports `NetworkNotFound` and no Instances start. If you don't have permission to use the network, Datum rejects the workload with `permission to use the network was denied`. ## Keep addresses after scale-down By default, an Instance keeps its address when Datum replaces it during a rollout or restart, but Datum releases the address when it removes the Instance, such as when you scale down or delete the workload. To keep addresses reserved even after scale-down, set `reclaimPolicy: Retain` on the network interface when you create the workload: ```yaml theme={null} networkInterfaces: - network: name: backend reclaimPolicy: Retain ``` Retained addresses stay reserved even when you scale down. You can't change `reclaimPolicy` after you create the workload. To see and release the interfaces that hold retained addresses after their Instance is gone, see [Manage network interfaces](/docs/galactic-vpc/operations#manage-network-interfaces). ## Permissions Networks use the following roles and permissions: | Task | What you need | | - | - | | View networks | The **Network Viewer** role or the **Network Admin** role. | | Create, change, or delete a network | The **Network Admin** role. | | Attach a workload to a network | The `networking.datumapis.com/networks.use` permission on the network, in addition to permission to create the workload. The **Network Admin** role includes `networks.use`. Datum checks the permission when you create the workload. | | Attach an ALB to a workload | The **Network Admin** role. For more information, see [Publish a workload](/docs/compute/compute-and-alb). | ## What's next * To view, change, and delete networks, see [Operations](/docs/galactic-vpc/operations). * To give a workload a stable name for a load balancer, see [Network services](/docs/galactic-vpc/networks/network-services). * To learn how Instances get addresses, see [Addressing](/docs/galactic-vpc/networks/addressing). * To review what networks can't do, see [Limitations and roadmap](/docs/galactic-vpc/limitations-and-roadmap). * To serve a workload publicly through Datum's ALB, see [Compute and ALB](/docs/compute/compute-and-alb). # gVPC limitations and roadmap Source: https://datum.net/docs/galactic-vpc/limitations-and-roadmap Review the limitations of gVPC networks and the networking capabilities that Datum is working on. > [!NOTE] > > gVPC is in preview, and the `networking.datumapis.com/v1alpha` API can change. ## Limitations In addition to the [networking limitations that apply to all of Compute](/docs/compute/limitations-and-roadmap#networking), networks have the following limitations: * Each workload attaches to exactly one network through one network interface, and it can't move to another network. * Instance addresses aren't reachable from the internet. To serve traffic publicly, attach an [ALB](/docs/alb/overview) to the workload. * The `networkPolicy` field of a network interface has no effect. Don't rely on it to restrict traffic. Firewall rules are a planned capability. * You can't choose a network's address ranges or create your own subnets. * Networks carry Compute traffic only in the locations where Compute runs: `us-central-1`, `us-east-1`, and `us-west-1`. ## Planned capabilities Datum is working on the following networking capabilities: * **Firewall rules:** Security groups that control the traffic to and from your Instances. * **Private DNS:** Private DNS names, so Instances can find each other without IP addresses. Outbound internet access, including IPv4 destinations, and viewing networks in the Datum Cloud portal are already available. For details, see [Traffic and connecting](/docs/galactic-vpc/networks/traffic). For what Datum is building next across the platform, see the [Platform roadmap](/docs/platform/roadmap). > [!NOTE] > > Plans can change. This list isn't a commitment to deliver any feature or to meet any timeline. ## What's next * To learn which traffic a network carries, see [Traffic and connecting](/docs/galactic-vpc/networks/traffic). * To serve a workload publicly through Datum's ALB, see [Publish a workload](/docs/compute/compute-and-alb). * To run Instances in more locations, see [Placement and scaling](/docs/compute/placement-and-scaling). # gVPC limits and quotas Source: https://datum.net/docs/galactic-vpc/limits-and-quotas The fixed limits for gVPC networks: address space, networks per workload, network interfaces per Instance, MTU, and the locations that carry Compute traffic. This page lists the fixed limits for [gVPC](/docs/galactic-vpc/overview) networks. gVPC is still being built, and network quotas are not documented yet. > [!NOTE] > > gVPC is in preview, and the `networking.datumapis.com/v1alpha` API can change. ## Limits | Item | Limit | | - | - | | Address space for each network | A `/48` IPv6 prefix that Datum assigns | | Address space for each location on a network | A `/64` from the network's prefix | | Address block for each Instance | A `/96` from the location's `/64` | | Networks for each workload | 1 | | Network interfaces for each Instance | Exactly 1 | | Default MTU | `1440` | | Locations that carry Compute traffic | `us-central-1`, `us-east-1`, and `us-west-1` | For limits that apply to the workloads on a network, see [Compute limits and quotas](/docs/compute/limits-and-quotas). For features that networks don't support yet, see [Limitations and roadmap](/docs/galactic-vpc/limitations-and-roadmap). # Addressing Source: https://datum.net/docs/galactic-vpc/networks/addressing How Instances on a gVPC network get private IPv6 addresses, and how the address plan is laid out. Every Instance gets a private IPv6 address on the network that its workload attaches to. This page describes the address plan. To learn which traffic a network carries, see [Traffic and connecting](/docs/galactic-vpc/networks/traffic). > [!NOTE] > > [gVPC](/docs/galactic-vpc/overview) is in preview, and the `networking.datumapis.com/v1alpha` API can change. ## How Instances get addresses Datum assigns all addresses. You don't choose address ranges or plan subnets. The following table describes each level of the address plan: | Level | Address space | Details | | - | - | - | | Network | A `/48` IPv6 prefix | Datum assigns the prefix from the unique local range `fd00::/8` when you create the network. Prefixes don't overlap between networks or projects. The `IPV6PREFIX` column of `datumctl get networks` shows the prefix. | | Location | A `/64` from the network's prefix | Datum assigns one to each location where Instances on the network run. | | Instance | A `/96` block from the location's `/64` | Each Instance's network interface owns the whole block. The block's primary address is the Instance's private IPv6 address. | In the [Cloud Portal](https://cloud.datum.net), open a network under **VPC** > **Networks** to see the address plan for real: the network's `/48`, the `/64` for each location, and the `/96` for each Instance. ![A network Overview showing Network is ready, and Instances grouped by location, us-central-1, us-east-1, and us-west-1, each with its /64 and each Instance with its /96 address](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/galactic-vpc-network-addresses.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=08b1cb745848b1dc836354bb50594342) A network's address plan: a /48 for the network, a /64 for each location, and a /96 for each Instance Inside the Instance, the interface is named `eth0`, and it has IPv6 addresses only. Unique local addresses are private: they aren't routable on the internet. To find your Instances' addresses, run the following command: ```bash theme={null} datumctl compute instances --workload=WORKLOAD_NAME ``` Replace `WORKLOAD_NAME` with the name of your workload. The `INTERNAL IP` column shows each Instance's primary private IPv6 address. For all the addresses on an Instance's interface, run `datumctl compute instances describe INSTANCE_NAME`. ## What's next * To learn which traffic a network carries and how Instances connect, see [Traffic and connecting](/docs/galactic-vpc/networks/traffic). * To keep an Instance's address after scale-down, see [Keep addresses after scale-down](/docs/galactic-vpc/configuration#keep-addresses-after-scale-down). # Network services Source: https://datum.net/docs/galactic-vpc/networks/network-services Give a workload's Instances a stable name and named ports on a gVPC network, and send load balancer traffic to them. A network service gives a workload's Instances on one network a stable name and named ports. Datum keeps the service's membership current as Instances start, stop, and move between locations, and sends each request to the nearest location that has a ready Instance. Datum's [Application Load Balancer](/docs/alb/overview) reaches a workload through a network service. > [!NOTE] > > gVPC is in preview, and the `networking.datumapis.com/v1alpha` API can change. ## How network services work A network service is a `NetworkService` resource with two parts: * **A selector:** labels that an Instance's network interface must carry to belong to the service. Every Instance of a workload carries the workload's name as a label, so a service usually selects one workload. * **Ports:** the ports the Instances listen on, each with a name, such as `http` on port `8080`. A load balancer refers to a port by its name, not its number. Keep the following points in mind: * **One network:** A service covers Instances on one network. If the selector matches Instances on more than one network, the service serves none of them. * **Ready Instances only:** A service sends traffic only to Instances that report themselves ready. An Instance that is starting, failing, or shutting down doesn't receive traffic. * **Nearest location first:** Each request goes to the nearest location with a ready Instance. If that location has none, the request goes to the next nearest. * **Created before the workload:** You can create a service before its workload exists. The service starts serving when matching Instances are ready. When you publish a workload with `datumctl compute deploy --http-port`, the command creates a network service named after the workload. For more information, see [Compute and ALB](/docs/compute/compute-and-alb). ## Before you begin * Install `datumctl`, log in, and select a project. For more information, see [Install datumctl](/docs/datumctl/install). * Install the `network` plugin. For more information, see [Install the plugin](/docs/galactic-vpc/operations#install-the-plugin). * To create, change, or delete network services, you need the **Network Admin** role in the project. ## Create a network service To create a network service for a workload, run the following command: ```bash theme={null} datumctl network service create SERVICE_NAME \ --workload=WORKLOAD_NAME \ --port=PORT_NAME:PORT_NUMBER ``` Replace the following: * `SERVICE_NAME`: a name for the service. You can't change the name later. * `WORKLOAD_NAME`: the name of the workload whose Instances answer for the service. * `PORT_NAME`: a name for the port, such as `http`. Use lowercase letters, numbers, and hyphens. * `PORT_NUMBER`: the port that your Instances listen on, such as `8080`. To add more ports, repeat `--port`. A service takes up to 16 ports, and each name and number must be unique. By default, a load balancer connects to a port over HTTP/1.1. For a port that serves gRPC or another HTTP/2 app without TLS (h2c), use the long form of `--port` with `app-protocol=h2c`, such as `--port=name=grpc,port=9090,app-protocol=h2c`. The load balancer then sends every request on that port as HTTP/2. Most servers accept WebSockets only over HTTP/1.1, so serve WebSockets on a different port. To select Instances by label instead of by workload, use `--selector`, such as `--selector app=web`. The labels must be on the Instances' network interfaces, not only on the workload. ## Send load balancer traffic to a network service To send a load balancer's traffic to a network service, name the service and one of its port names: ```bash theme={null} datumctl alb create LOAD_BALANCER_NAME \ --network-service=SERVICE_NAME \ --port=PORT_NAME ``` For more information, see [Create a load balancer](/docs/alb/operations/overview#create-a-load-balancer). ## Check a network service To see what a service selects, its ports, and where it's serving, run the following command: ```bash theme={null} datumctl network service describe SERVICE_NAME ``` The output ends with a diagnosis. If the service isn't serving, the output also lists the Instances that aren't ready and what each one reports. The following table describes the most common reasons that a service isn't serving: | Reason | Meaning | What to do | | - | - | - | | `NoMatchingInterfaces` | No Instance matches what the service selects. | Start the workload, or fix the selector. The output lists the workloads in your project so that you can compare names. | | `MultipleNetworks` | The selector matches Instances on more than one network. | Narrow the selector so that it matches Instances on one network. | | `NoServingLocations` | The service has Instances, and none of them is ready. | Check the workload. Its Instances might be starting or failing. | ## Change a network service To change which workload a service selects, run the following command: ```bash theme={null} datumctl network service update SERVICE_NAME --workload=WORKLOAD_NAME ``` To change a service's ports, run the following command: ```bash theme={null} datumctl network service update SERVICE_NAME --port=http:8080 --port=metrics:9100 ``` `--port` replaces the whole port list, so name every port that the service should keep. A load balancer that refers to a port name that you remove stops reaching the service. ## Delete a network service To delete a network service, run the following command: ```bash theme={null} datumctl network service delete SERVICE_NAME ``` Before it asks you to confirm, the command lists the load balancers that send traffic to the service. Those load balancers stop reaching anything on that route. ## What's next * To manage networks and Instances' network interfaces from the command line, see [Operations](/docs/galactic-vpc/operations). * To publish a workload with a load balancer in one step, see [Compute and ALB](/docs/compute/compute-and-alb). * To learn how Instances connect privately, see [Traffic and connecting](/docs/galactic-vpc/networks/traffic). # Networks overview Source: https://datum.net/docs/galactic-vpc/networks/overview What a gVPC network is, how it relates to projects, locations, and workloads, and how the default network works. A *network* is the core [gVPC](/docs/galactic-vpc/overview) resource. You create networks in a project, attach Compute workloads to them, and each Instance receives a private IPv6 address on the network. > [!NOTE] > > gVPC is in preview, and the `networking.datumapis.com/v1alpha` API can change. ## How networks work A network belongs to a project, not to a location. The following list describes how a network relates to your projects, locations, workloads, and Instances: * **Networks belong to a project:** You create networks with the `Network` resource. You don't create a network for each location. When a workload runs Instances in a location, Datum gives the network address space in that location and assigns each Instance an address from it. * **One network for each workload:** All of a workload's Instances, in every [placement](/docs/compute/placement-and-scaling), attach to the same network. You choose the network when you create the workload. * **Many workloads on one network:** Several workloads can name the same network. For example, a frontend workload and a backend workload can share a network. * **Addresses are private IPv6:** Each network gets its own private IPv6 prefix from the unique local address range. Each Instance gets an address from that prefix. The addresses aren't reachable from the internet. For more information, see [Addressing](/docs/galactic-vpc/networks/addressing). * **Networks are isolated:** Each network is separate from every other network, including other networks in the same project and other customers' networks. * **Public traffic arrives through Datum's ALB:** To serve a workload on the internet, attach Datum's [Application Load Balancer](/docs/alb/overview) to it. The ALB reaches a workload through a [network service](/docs/galactic-vpc/networks/network-services), which gives the workload's Instances a stable name and named ports, and forwards each request to an Instance over the private network. For more information, see [Compute and ALB](/docs/compute/compute-and-alb). ## The default network You don't need to create a network before your first deploy. When you run `datumctl compute deploy` without `--network`, the command attaches the workload to a network named `default`. If the network doesn't exist, the command offers to create it. In scripts, pass `--yes` to create it without a prompt. Datum doesn't create the `default` network when it creates a project. If you apply a workload with `datumctl apply`, the network must already exist. `datumctl compute deploy -f` offers to create a missing network, as the flag-based deploy does. ## In this section > [card] [Addressing](/docs/galactic-vpc/networks/addressing) > > How Instances get private IPv6 addresses. > [card] [Traffic and connecting](/docs/galactic-vpc/networks/traffic) > > Which traffic a network carries and how Instances connect. > [card] [Network services](/docs/galactic-vpc/networks/network-services) > > Give a workload a stable name and named ports for a load balancer. # Traffic and connecting Source: https://datum.net/docs/galactic-vpc/networks/traffic Learn which traffic a gVPC network carries and how Instances connect to each other. A network carries private traffic between Instances. This page describes which kinds of traffic are possible and how Instances connect to each other. To learn how Instances get their addresses, see [Addressing](/docs/galactic-vpc/networks/addressing). > [!NOTE] > > [gVPC](/docs/galactic-vpc/overview) is in preview, and the `networking.datumapis.com/v1alpha` API can change. ## What traffic a network carries The following table describes which kinds of traffic are possible: | Traffic | Available | Details | | - | - | - | | Between Instances on the same network in the same location | Yes | Instances use their private IPv6 addresses. Any Instance can connect to any port on another Instance in the same network and location. | | Between Instances on the same network in different locations | Yes | A network spans locations, and Instances can connect to each other privately across locations using their private IPv6 addresses. | | Between different networks | No | Networks are isolated from each other. | | To or from other addresses through an Instance | No | A network only delivers a packet addressed to an Instance's own address, and only accepts a packet from an Instance's own address. An Instance can't route traffic for other addresses, so a router or VPN appliance needs an overlay, such as VXLAN or a UDP tunnel, between Instances. | | Inbound from the internet | Through Datum's ALB | Instance addresses aren't reachable from the internet directly. Public HTTP and HTTPS traffic reaches Instances through Datum's [Application Load Balancer](/docs/alb/overview). Raw TCP and UDP exposure is a preview limitation. | | Outbound to the internet | Yes | Instances can reach internet destinations, including services that have only IPv4 addresses. | | Lookups by name | Not in the preview | Private DNS names for Instances are on the [roadmap](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). Use IPv6 addresses in the preview. | | Firewall filtering | Not in the preview | Firewall rules to filter traffic between Instances on a network are on the [roadmap](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). | The workload API accepts a `networkPolicy` field on each network interface, but the field has no effect. Don't rely on it to restrict traffic. To limit who can reach a service, check credentials in your app, or put services that must stay apart on different networks. Firewall rules for networks are on the [roadmap](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). ## Connect to other Instances To connect from one Instance to another on the same network, use the peer's private IPv6 address. Keep the following points in mind: * Make your app listen on IPv6 (`::`). An app that listens only on `0.0.0.0` accepts IPv4 connections only, and Instances don't have IPv4 addresses. * In a URL, put an IPv6 address in square brackets, such as `http://[fd20:4a3b:91c2:1::1:0:2]:8080/`. * Private DNS names aren't available in the preview. To tell an app about its peers, pass their addresses in environment variables or a ConfigMap. ## What's next * To serve a workload publicly through Datum's ALB, see [Compute and ALB](/docs/compute/compute-and-alb). * To review what networks can't do, see [Limitations and roadmap](/docs/galactic-vpc/limitations-and-roadmap). # gVPC observability Source: https://datum.net/docs/galactic-vpc/observability See the state of your gVPC networks, the private addresses of your Instances, and the metrics Datum publishes for them. This page covers how to see what your [gVPC](/docs/galactic-vpc/overview) networks are doing: whether each network is ready, which private addresses your Instances have, and the metrics Datum publishes for them. ## Is a network ready? To list your networks with their address space and status, run the following command: ```text theme={null} $ datumctl get networks NAME IPV6PREFIX READY REASON AGE default fd20:0:36::/48 True Ready 45m api fd20:0:37::/48 True Ready 18m ``` `IPV6PREFIX` is the `/48` that Datum assigned to the network. A network is ready to use when `READY` is `True`. For more detail, including each condition and its reason, run `datumctl describe network NETWORK_NAME`. ## Which addresses do my Instances have? To see each Instance's private address on its network, run the following command: ```text theme={null} $ datumctl compute instances --workload=api NAME WORKLOAD LOCATION INTERNAL IP TYPE AGE STATUS api-us-central-us-central-1-0 api us-central-1 fd20:0:37:2:0:1::/96 sandbox 17m Available api-us-east-us-east-1-0 api us-east-1 fd20:0:37::2:0:0/96 sandbox 17m Available api-us-west-us-west-1-0 api us-west-1 fd20:0:37:1:0:1::/96 sandbox 17m Available ``` Each Instance owns the `/96` block shown in `INTERNAL IP`, taken from the network's prefix. The first address in the block is the Instance's private IPv6 address. Instances in different locations get addresses from different parts of the prefix, but they're all on the same network. See [Addressing](/docs/galactic-vpc/networks/addressing). ## Metrics Datum publishes the state of your networks and your Instances' addresses to your project's metrics store. To query them in your own tools, [export them](/docs/operations/metrics-export). | Metric | What it reports | | - | - | | `datum_cloud_network_status_condition` | `1` when a network condition is true. The `condition` label is the condition, such as `Ready`, and `reason` explains it. `resource_name` is the network's name. | | `datum_cloud_network_info` | One series for each network. | | `datum_cloud_compute_instance_network_interface_info` | One series for each Instance network interface, with its private address in `network_ip`. | To find networks that aren't ready, replace `PROJECT_ID`: ```promql theme={null} datum_cloud_network_status_condition{resourcemanager_datumapis_com_project_name="PROJECT_ID", condition="Ready"} == 0 ``` Datum doesn't publish traffic metrics for networks yet, such as bytes or packets between Instances. For CPU, memory, and logs of the Instances on a network, see [Compute observability](/docs/compute/observability). For traffic that reaches your network through a load balancer, see [ALB metrics](/docs/alb/observability#metrics). ## Follow the roadmap > [card] [gVPC limitations and roadmap](/docs/galactic-vpc/limitations-and-roadmap) > > See what gVPC doesn't support yet and what Datum is building next. # gVPC operations Source: https://datum.net/docs/galactic-vpc/operations Use the datumctl network plugin to view, change, and delete gVPC networks, network services, and Instances' network interfaces. This page covers day-to-day operations on [gVPC](/docs/galactic-vpc/overview) networks with the `network` plugin for `datumctl`. The plugin manages networks, network services, and the network interfaces that Instances hold on your networks. It writes the same resources as the manifests on [Configuration](/docs/galactic-vpc/configuration), and its `describe` output explains what's wrong and who needs to act. > [!NOTE] > > gVPC is in preview, and the `networking.datumapis.com/v1alpha` API can change. ## Install the plugin To install the plugin from the official Datum catalog, run the following command: ```bash theme={null} datumctl plugin install network ``` To check the installation, run the following command: ```bash theme={null} datumctl network version ``` `datumctl network version` works without a login, a project, or a network connection. For more information about plugins, see [Using plugins](/docs/datumctl/plugins/using-plugins). Without the plugin, you can still list and delete networks with `datumctl get networks` and `datumctl delete network NETWORK_NAME`. ## Manage networks The following table describes the network commands: | Command | What it does | | - | - | | `datumctl network create NAME` | Creates an IPv6-only network. Pass `--mtu` to change the MTU from the default of `1440`, and `--wait` to wait until the network is ready. | | `datumctl network list` | Lists networks, the locations they've reached, and how many Instances and services use them. Pass `-o wide` for each network's IPv6 prefix. | | `datumctl network describe NAME` | Shows a network's address families, MTU, IPv6 prefix, each location with its address range and Instance counts, the workloads and network services on it, and a diagnosis. | | `datumctl network update NAME --mtu MTU` | Changes the MTU, which is the only setting that you can change after you create a network. The change applies to every Instance on the network. | | `datumctl network delete NAME` | Deletes a network. | A new network isn't in any location. It reaches a location when a workload in that location attaches to it, so `describe` shows no locations until you deploy a workload: ```bash theme={null} datumctl network create backend datumctl compute deploy api --image=ghcr.io/example/api:1.4.2 --location=us-east-1 --network=backend datumctl network describe backend ``` A workload's network can't change after you create the workload. To move a workload to another network, deploy a new workload on that network and then delete the old one. For more information, see [Compute operations](/docs/compute/operations#update-a-workload). ## Delete a network Before you delete a network, delete the workloads that are attached to it. To delete a network, run the following command: ```bash theme={null} datumctl network delete NETWORK_NAME ``` Replace `NETWORK_NAME` with the name of the network. Before it asks you to confirm, the command lists the workloads that lose their network, the network services that stop serving, and the kept interfaces whose addresses are released. If Instances still hold addresses on the network, the network stays in deletion and reports `RangeOccupied` until they're gone. ## Manage network services The `datumctl network service` commands create, describe, change, and delete network services. For more information, see [Network services](/docs/galactic-vpc/networks/network-services). ```bash theme={null} datumctl network service create api --workload=api --port=http:8080 datumctl network service list datumctl network service describe api ``` ## Manage network interfaces Each Instance holds a network interface on its workload's network. To list the interfaces in your project, run the following command: ```bash theme={null} datumctl network interface list ``` To narrow the list, pass `--network`, `--workload`, or `--kept`. A workload with `reclaimPolicy: Retain` keeps an Instance's addresses after the Instance is removed. The kept interface holds the addresses until an Instance with the same name returns. For more information, see [Keep addresses after scale-down](/docs/galactic-vpc/configuration#keep-addresses-after-scale-down). To release a kept interface and give up its addresses permanently, run the following command: ```bash theme={null} datumctl network interface release INTERFACE_NAME ``` Replace `INTERFACE_NAME` with the name of the kept interface. You can't release an interface that a running Instance holds. After you release the addresses, update anything that refers to them, such as DNS records or firewall rules. ## Find out what's wrong `datumctl network describe` and `datumctl network service describe` end with a diagnosis that names the cause, the location or workload that it affects, and who needs to act: you, Datum, or nobody while a change is still in progress. Keep the following points in mind: * A new network with no locations isn't a fault. The network reaches a location when a workload there attaches to it. * A network service with no ready Instances is a workload problem, not a network problem. `datumctl network service describe` lists the Instances that aren't ready and what each one reports. For specific errors, see [Troubleshooting](/docs/galactic-vpc/troubleshooting). ## Scripts and automation * **Confirmation:** `delete`, `update`, `service delete`, and `interface release` ask you to type the resource's name. Without a terminal, they stop unless you pass `--yes`. * **Previews:** Commands that change resources accept `--dry-run`, which validates the change and discards it. * **Output:** Commands that read resources accept `-o table`, `-o wide`, `-o json`, `-o yaml`, or `-o name`. * **Exit codes:** Failures print a named exit code, such as `exit status 4 # NETWORK_NOT_FOUND`. The codes are `NETWORK_USAGE` (2), `NETWORK_FORBIDDEN` (3), `NETWORK_NOT_FOUND` (4), `NETWORK_CONFLICT` (5), `NETWORK_INVALID` (6), `NETWORK_UNAVAILABLE` (8), and `NETWORK_ABORTED` (9). * **Waiting:** If `create --wait` times out, the command exits with an error, but the network exists. Don't run `create` again; run `describe` instead. ## What's next * To create a network with a manifest and attach a workload, see [Configuration](/docs/galactic-vpc/configuration). * To give a workload a stable name for a load balancer, see [Network services](/docs/galactic-vpc/networks/network-services). * To fix network errors, see [Troubleshooting](/docs/galactic-vpc/troubleshooting). # gVPC Source: https://datum.net/docs/galactic-vpc/overview gVPC gives your project private IPv6 networks that span every Datum location your workloads run in. > [!NOTE] > > **Working with this product via an AI agent?** Datum publishes a [Networks skill](/docs/agents/skills/networks) that teaches agents to create and manage networks with `datumctl network`. [Datum MCP](/docs/agents/datum-mcp) gives agents live access through its `network` and `vpc` toolsets. gVPC is Datum's private networking for your workloads. You create a *network* in your project, attach [Compute](/docs/compute/overview) workloads to it, and each Instance receives a private IPv6 address on that network. One network spans every location that your workloads run in, so you don't create a separate network for each region. ![A gVPC network in the center with an attachment port for each thing connected to it. On the left, an app on your laptop reaches the network through a datum tunnel (planned). On the right, Compute Instances in Dallas and Frankfurt (available) and a physical connection to your data center (exploring) attach to the same network. Dashed paths show the laptop app reaching the Dallas Instance, Dallas reaching Frankfurt, and Frankfurt reaching the data center, all through the network.](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/gvpc-connect.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=a567a092379041ae65b3baa1f981b42c) One network, many attachments. Compute Instances attach today. Tunnels from your laptop and physical connections are on the roadmap. > [!NOTE] > > gVPC is in preview, and the `networking.datumapis.com/v1alpha` API can change. For the story behind gVPC, see [Meet me in the Galactic VPC](https://www.datum.net/blog/galactic-vpc-backbone/) on the Datum blog. gVPC runs on Datum's global IPv6 backbone. Datum plans the addresses, carries the traffic, and keeps each network separate from every other network. ## How it works A network belongs to a project, not to a location. You attach Compute workloads to a network, and every Instance of those workloads gets a private IPv6 address on it, in whichever location the Instance runs. Networks are isolated from each other, and public traffic reaches Instances only through Datum's [Application Load Balancer](/docs/alb/overview). For details, see [Networks overview](/docs/galactic-vpc/networks/overview). ## Features gVPC provides the following features: * **One network across locations:** A network is a single project-wide resource. Every workload that names it, in any location, attaches to the same network, and each Instance gets a private IPv6 address on it. * **Automatic addressing:** Datum assigns address space to each network and an address to each Instance. You don't plan subnets. * **Stable addresses when you need them:** Instances keep their addresses when Datum replaces them, and a workload can keep addresses reserved when it scales down. * **Network services:** A [network service](/docs/galactic-vpc/networks/network-services) gives a workload's Instances a stable name and named ports, so a load balancer can send traffic to them. * **Declarative:** Networks are Kubernetes-style resources that you manage with the [`datumctl network` plugin](/docs/galactic-vpc/operations), manifests, or GitOps tooling. To learn what Datum is working on next, see [Planned capabilities](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). ## What's next > [card] [Networks](/docs/galactic-vpc/networks/overview) > > How networks relate to projects, locations, and workloads. > [card] [Configuration](/docs/galactic-vpc/configuration) > > Create a network and attach a workload to it. > [card] [Network services](/docs/galactic-vpc/networks/network-services) > > Give a workload a stable name and named ports for a load balancer. > [card] [Operations](/docs/galactic-vpc/operations) > > Inspect, change, and delete networks and network services with the `network` plugin. > [card] [Limitations and roadmap](/docs/galactic-vpc/limitations-and-roadmap) > > Review networking limitations and planned capabilities. > [card] [Deploy an app](/docs/getting-started/first-workload) > > Deploy a workload that attaches to the `default` network. # gVPC troubleshooting Source: https://datum.net/docs/galactic-vpc/troubleshooting Find out why a gVPC network or Instance address isn't working, and fix common network errors. This page lists common [gVPC](/docs/galactic-vpc/overview) errors and how to fix them. gVPC is still being built, so this page will grow. > [!NOTE] > > gVPC is in preview, and the `networking.datumapis.com/v1alpha` API can change. Start with `datumctl network describe NETWORK_NAME` or `datumctl network service describe SERVICE_NAME`. Each ends with a diagnosis that names the cause and who needs to act. For more information, see [Operations](/docs/galactic-vpc/operations). ## Common problems | Error or reason | Meaning | What to do | | - | - | - | | `NetworkNotFound` | The network named in the workload's `networkInterfaces` doesn't exist, so no Instances start. | Create the network. See [Configuration](/docs/galactic-vpc/configuration#create-a-network). | | `permission to use the network was denied` | You don't have the `networking.datumapis.com/networks.use` permission on the network. | Ask a project administrator for the **Network Admin** role. See [Permissions](/docs/galactic-vpc/configuration#permissions). | | Instances never start, with `NetworkFailedToCreate` and `has not reached location` | The workload is attached to a network that is being deleted. A network that is being deleted can still report `Ready`. | Run `datumctl get network NETWORK_NAME -o yaml` and check for a `deletionTimestamp`. To see where the network exists, run `datumctl get networkcontexts,networkbindings`. Attach the workload to another network. | | `RangeOccupied` | A network you're deleting still has Instances holding addresses on it. | Delete the workloads attached to the network. See [Delete a network](/docs/galactic-vpc/operations#delete-a-network). | | A network is rejected for not including IPv6 | `spec.ipFamilies` doesn't include `IPv6`. | Keep the default, `[IPv6]`. | | Connections between locations stall | `spec.mtu` is larger than the default of `1440`. | Keep the default MTU. | | An Instance can't reach a peer | The app listens only on IPv4, or the URL doesn't bracket the IPv6 address. | Listen on `::` and use `http://[ADDRESS]:PORT/`. See [Traffic and connecting](/docs/galactic-vpc/networks/traffic#connect-to-other-instances). | | `NoMatchingInterfaces`, `MultipleNetworks`, or `NoServingLocations` | A network service isn't serving. | See [Check a network service](/docs/galactic-vpc/networks/network-services#check-a-network-service). | ## MTU gVPC networks use an MTU of `1440` bytes by default. The [Cloud Portal](https://cloud.datum.net) shows it on a network's **Configuration** tab, and `datumctl get network NETWORK_NAME -o yaml` shows it in `spec.mtu`. The MTU is smaller than the common Ethernet default of `1500`, because Datum wraps each packet in an extra IPv6 header to carry it between locations. See [Segment routing](/docs/platform/segment-routing). > [!WARNING] > > **Needs further investigation.** How the `1440` MTU affects traffic between Instances hasn't been fully documented yet. If large requests, file transfers, or TLS handshakes between Instances stall while small requests work, packet size is a likely cause: keep the default MTU, and check that the app and any tunnels inside the Instance don't assume `1500`. For an overlay or tunnel between Instances, such as VXLAN, an MTU of `1360` or less has worked in testing. Report what you find to [support@datum.net](mailto:support@datum.net). For problems with the workloads on a network, see [Compute troubleshooting](/docs/compute/troubleshooting). # Get Involved Source: https://datum.net/docs/get-involved Datum is an open source company; we are building in the open and encourage feedback, contributions, questions, bug reports etc. We're building in the open and would seriously love your feedback, questions, bug reports, ideas, challenges, and support. ## Discord Our real-time mode of conversation with users and community members is [through Discord](https://link.datum.net/discord). We aim to keep the vibe chill and the tone respectful, but we appreciate all kinds of ideas, links to cool articles or projects, and questions or support requests. ## GitHub discussions Whenever possible, we bring feature and bug conversations to [GitHub Discussions](https://github.com/orgs/datum-cloud/discussions). This helps us "walk the walk" by handling problems in an open forum, and benefits other users who might be experiencing similar things. This is also where we post changelog items, drawing attention to notable improvements or feature updates (these are also fed into [our website](https://www.datum.net/changelog/) for convenience). ## GitHub enhancements & milestones We track our planned work through [enhancements](https://github.com/datum-cloud/enhancements/issues), which function like epics. We regularly review and organize these into [milestones](https://github.com/datum-cloud/enhancements/milestones) to create a monthly "release" cycle. Milestones have names chosen to honor innovative or interesting people such as Dolores Huerta and John Blankenbaker. You can see previous [milestones here](https://github.com/datum-cloud/enhancements/milestones?state=closed). We ship all the time, but communicate broadly with a monthly rhythm that includes our email newsletter, community huddle, and meetups. ## Monthly community huddles Each month we host a public community meeting. These function like open office hours, usually with Datum engineers showing their work or a relevant guest presenting on an interesting topic. Feel free to stop in, ask questions, bring ideas, or just listen. * You can view all of our upcoming events, including the monthly huddle, [over here](https://www.datum.net/events/). * Registration through Luma (linked for each event) helps you keep it on your calendar! ## NYC & Boston Alt Cloud meetups Sometimes you just need to gather with other like-minded cloud operators to hang out, trade war stories, and learn something new. We host a meetup in NYC and in Boston three times in the Fall (Sep, Oct, Nov) and again in the Spring (Feb, Mar, Apr). Topics range from Unikernels to the security implications of OpenClaw. We start with drinks and light bites and end with a monthly "Mutual Aid" session where community members help each other with a request. ## Newsletter Our monthly email newsletter is a primary point of contact for product updates and company news, but also includes a summary of content we've published (blogs, guides, etc.) and things we're reading. You can subscribe via the Account Preferences of the Datum Cloud portal or at the bottom of our website. # Account Setup Source: https://datum.net/docs/getting-started/account-setup Create a Datum Cloud account, get your first organization and project, and invite team members to collaborate. The first step is to create a user account at [https://cloud.datum.net](https://cloud.datum.net). You can sign up with Google or GitHub, or with your email address and a passkey. For the steps, see [Create an account](/docs/account/create-account). Once you’ve created a user account, you can personalize your experience and access details by selecting your avatar or initials at the top right of your screen. ![User Account Preferences](https://mintcdn.com/datum-4926dda5/9pdr37W0ZlTfIFim/images/user-account-preferences.png?fit=max\&auto=format\&n=9pdr37W0ZlTfIFim\&q=85\&s=9838f51b59af74b226d825b0f06f2c3b) Find your profile in the top right of the portal ## Orgs & Projects We use a standard cloud data model that associates [Projects](/docs/projects/overview) within [Organizations](/docs/organizations/overview). All organizations use our **standard** quota limits, including support for collaborators, projects, and shared resources. When you finish signing up, Datum creates your first organization and your first project for you. You can rename them, or create more, at any time. ![The sign-up Complete screen, with checkmarks for setting up your organization and configuring your first project, and an Access the portal button](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/onboarding-complete.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=a512b9f4c8cbf250c5e74432f29911f1) Signing up creates your first organization and project > [!NOTE] > > Personal organizations have been deprecated as of July 14, 2026. All existing organizations have been automatically upgraded to standard quota limits — no action required. ## Payment method A payment method is required for every active organization to improve the security and stability of our platform. Datum is currently free during public beta, and your account won’t accrue any charges without prior notice. For how usage will be priced, see [pricing](https://www.datum.net/pricing). ### Confirming your contact details As part of adding your payment method, we’ll ask you to confirm: * Your **legal name**. * Any **company name** you want associated with an organization. ## Team management To collaborate with team members, you’ll need to create or join an Organization. Organizations support the following coarse-grained roles: * **Owners** - Full control over the organization and all resources. * **Editors** - Control over all resources in an organization, except for team members. * **Viewers** - Read-only access to all resources in an organization. For more information, see [Team management](/docs/organizations/team-management). ## Quotas Datum is in active development and is free during public beta, with reasonable limits on resources. Resource limits (e.g. quotas) can be found under the setting areas for Organizations and Projects. For more information, see [Usage](/docs/organizations/usage). ## Support If you run into a problem, including a quota that limits your work, email [support@datum.net](mailto:support@datum.net), or use the live chat at the top of the portal. [Discord](https://link.datum.net/discord) is also an option! # Deploy an app Source: https://datum.net/docs/getting-started/first-workload Deploy draw.io to Datum Compute and open it in your browser through an Application Load Balancer that Datum attaches for you. > [!NOTE] > > Compute is in preview. The `v1alpha` APIs can change, and Datum reviews Compute access requests manually. ## Before you begin * Create a Datum Cloud account, an organization, and a project. For more information, see [Account setup](/docs/getting-started/account-setup). * Install `datumctl`. For more information, see [Install](/docs/datumctl/install). * Run `datumctl login`. For more information, see [Logging in](/docs/datumctl/auth/logging-in). * Run `datumctl plugin install compute`. For more information, see [Using plugins](/docs/datumctl/plugins/using-plugins). * Select the project to deploy in with `datumctl ctx use your-org-id/your-project-id`. To confirm your account and project, run `datumctl whoami`. For more information, see [Contexts and scoping](/docs/datumctl/contexts-and-scoping). * Make sure that Compute access is active for the project. Run `datumctl compute access` and check that the `Status` line shows `Active`. If it doesn't, see [Request Compute access](/docs/compute/limits-and-quotas#request-access). ## Deploy draw\.io This page deploys [draw.io](https://www.drawio.com/), an open source diagramming app, using the official `docker.io/jgraph/drawio` image with the `general-purpose` runtime class, which runs standard container images without changes. The app fits in one Instance (1 vCPU and 2 GiB of memory), stores nothing on the Instance, and needs no passwords or config files. **1. Deploy draw\.io** To deploy draw\.io to Dallas (`us-central-1`) and publish it on a public URL, run the following command: ```bash theme={null} datumctl compute deploy drawio \ --image=docker.io/jgraph/drawio:31.4.6 \ --runtime-class=general-purpose \ --location=us-central-1 \ --http-port=8080 ``` The flags do the following: * `--image`: the container image to run, pinned to a draw\.io release. Compute requires the registry host in the image reference, such as `docker.io`. * `--runtime-class=general-purpose`: runs the image as a standard container in a lightweight virtual machine. Without this flag, Datum uses the `unikernel` class, which needs a [specially packaged image](/docs/compute/runtimes/unikernels). To compare the two, see [Choose a runtime class](/docs/compute/runtimes/overview). * `--location=us-central-1`: runs one Instance in Dallas. * `--http-port=8080`: attaches an ALB to the workload and publishes port `8080`, where draw\.io serves HTTP. The command asks you two questions: 1. The first time you deploy in a project, it asks whether to create a network named `default`. Enter `y`. 2. It prints a plan and asks `Apply? (Y/n)`. Enter `y`. The command then creates the workload, waits for the Instance to start, and publishes the URL. The end of the output is similar to the following: ```text theme={null} Rollout complete in 22s. Publishing... Backends 1 healthy across us-central-1 Edge programmed Certificate issued https://peril-oven-p53hn.datumproxy.net ``` The last line is the hostname of the ALB attached to the workload. Your hostname is different. The command also saves the workload definition to `workload.yaml` in the current directory. **2. Check that draw\.io is available** To list the workloads in your project, run the following command: ```bash theme={null} datumctl compute workloads ``` Continue when `HEALTH` is `Available` and `READY` is `1/1`. The `URL` column shows the ALB hostname from the deploy output. **3. Open draw\.io and draw a diagram** In your web browser, go to the URL from the deploy output. Datum's ALB serves draw\.io over HTTPS, and the app opens a blank diagram. If the browser can't find the site, wait a minute and try again. Click the rectangle in the **General** shape panel, double-click it, and type `Hello from Datum`. To keep it, click **File** > **Export as** > **PNG...**. The editor runs in your browser, so your diagrams stay with you even when Datum restarts or replaces the Instance. **4. Optional: See the Instance** To list the workload's Instances, run the following command: ```bash theme={null} datumctl compute instances --workload=drawio ``` The `INTERNAL IP` column shows the Instance's private IPv6 address on the `default` [gVPC](/docs/galactic-vpc/overview) network. The Instance has no public address, so traffic from the internet reaches draw\.io only through the ALB. ## Clean up 1. To delete the draw\.io workload, its Instance, and its ALB, run the following command and enter `y` to confirm: ```bash theme={null} datumctl compute destroy drawio ``` 2. Delete `workload.yaml` from the current directory. 3. Optional: If this tutorial created the `default` network and no other workload uses it, delete the network: ```bash theme={null} datumctl delete network default ``` ## What's next * To deploy your own image, check that your app listens on IPv6 and serves plain HTTP. During the preview, Instances have no persistent disks and no readiness checks. For details, see [Compute limitations](/docs/compute/limitations-and-roadmap#limitations). * To pass configuration and secrets to a workload, see [Compute configuration](/docs/compute/configuration). * To package your own app as a fast-starting unikernel, see [Unikernels](/docs/compute/runtimes/unikernels). * To run in more locations or with more Instances, see [Placement and scaling](/docs/compute/placement-and-scaling). * To learn how Datum's control plane turns your resources into running Instances, DNS, and load balancers, see [Platform API](/docs/platform/kubernetes). # Use ALB as your HTTP proxy Source: https://datum.net/docs/guides/alb-as-httpd Replace the NGINX, Apache, Caddy, or Traefik container in front of your app with Datum's Application Load Balancer. If your app runs behind NGINX, Apache httpd, Caddy, or Traefik, that proxy usually does four jobs: * Ends TLS * Routes requests by hostname and path * Spreads requests across copies of your app * Keeps unwanted traffic out On [Datum Compute](/docs/compute/overview), the [Application Load Balancer (ALB)](/docs/alb/overview) does all four at the edge, so you only have to deploy your app's container. The examples use NGINX syntax. The same approach applies to Apache httpd, Caddy, and Traefik. > [!NOTE] > > This guide covers the common cases. The [ALB as httpd reference](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md) has the rest: > > * A directive-by-directive mapping > * Limits > * Troubleshooting > * The results of the live tests behind this guide > > Its sections use the same headings as this guide. The tests ran on October 28, 2026, against preview APIs, so recheck edge cases before you rely on them. This guide covers: * [Deploying your app without a proxy container](#deploy-your-app-without-a-proxy-container) * [Translating common proxy configuration](#translate-common-proxy-configuration) * [Reviewing production settings](#production-review) * [Verifying the migration](#verify-the-migration) > [!NOTE] > > Compute is in preview. The `v1alpha` APIs can change, and Datum reviews Compute access requests manually. ## Before you begin * Create a Datum Cloud account, an organization, and a project. For more information, see [Account setup](/docs/getting-started/account-setup). * Install `datumctl` and run `datumctl login`. For more information, see [Install](/docs/datumctl/install). * Run `datumctl plugin install compute`, and make sure that Compute access is active for the project: run `datumctl compute access` and check that the `Status` line shows `Active`. For more information, see [Deploy an app](/docs/getting-started/first-workload#before-you-begin). * Run `datumctl plugin install alb`. This guide uses the `alb` plugin for most ALB changes. * Have a container image for your app, without the proxy container. This guide names the app workload `web` and a second workload `api`, and shows the hostnames from a test run, such as `tile-desire-9gpyk.datumproxy.net`. Replace them with your own names and hostnames everywhere they appear, including inside YAML files. This guide assumes your app serves HTTP itself, on a port you name. The ALB ends TLS and sends plain HTTP to that port. If your app needs its proxy to translate requests for it, see [Apps that need a gateway proxy](#apps-that-need-a-gateway-proxy) first. ## How proxy concepts map to ALB Datum uses a few terms that differ from proxy configuration: * **Workload**: your app on Datum Compute. It runs as one or more **Instances** in each location you choose. See [Compute](/docs/compute/overview). * **HTTPProxy**: the ALB resource you edit. It holds hostnames, routing rules, and backends, much like an NGINX `server` block. See [HTTPProxy](/docs/alb/httpproxy/overview). * **Backend**: where a rule sends requests, usually a Compute workload. This is the NGINX `upstream`. See [Load balancing and health checks](/docs/alb/configuration/load-balancing). * **Policy**: a separate resource attached to the ALB. In NGINX these are directives inside the config file. On Datum they sit beside the HTTPProxy. See [ALB configuration](/docs/alb/configuration/overview). Policies cover: * WAF * Sign-in * IP rules * Timeouts * Rate limits The following table maps the parts of a proxy configuration you use most: | In your proxy | On Datum | | - | - | | Virtual host (`server_name`) | `hostnames` on an HTTPProxy, or `datumctl alb hostname add` | | Location or route (`location`) | `rules` and `matches` on an HTTPProxy, or `datumctl alb route add` | | Upstream pool (`upstream`) | `backends` on a rule, usually a workload's network service | | TLS certificates (`ssl_certificate`) | Issued and renewed by Datum for each hostname | | Config test (`nginx -t`) | `datumctl diff -f FILE` and `datumctl apply -f FILE --dry-run=server` | | Config reload (`nginx -s reload`) | `datumctl apply -f FILE`, or a `datumctl alb` command | For Apache, Caddy, and Traefik equivalents, and for every NGINX directive, see [Directive mapping](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#directive-mapping) in the reference. Your ALB hostname is anycast. Each request reaches the nearest [Datum location](/docs/locations), where a managed fleet of Envoy proxies ends TLS and forwards plain HTTP to a healthy Instance. There is no proxy server for you to size, patch, or reload. Changes take up to three minutes to reach every location, and usually less than a minute. Check for the new behavior before you test anything else. For how to wait for a change from a script, see [Edge model](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#edge-model) in the reference. ### What stays in your app The ALB routes, protects, and balances requests. It doesn't store files or change response bodies, so keep these in your app or its image: * Static files * Response compression * Caching * Custom error pages For the full list, see [What stays in your app](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#what-stays-in-your-app) in the reference. ## Deploy your app without a proxy container `datumctl compute deploy --http-port` creates: * The workload * An ALB * The HTTPProxy that connects them The ALB it creates is minimal: * It serves plain HTTP alongside HTTPS. * It has no WAF. The steps after the deploy add both. For how the ALB attaches to a workload, see [Serve public traffic with ALB](/docs/compute/compute-and-alb). For test results, see [Deploy your app without a proxy container](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#deploy-your-app-without-a-proxy-container) in the reference. **1. Deploy the app with an ALB** To deploy your app to Dallas (`us-central-1`) with two Instances, run the following command: ```bash theme={null} datumctl compute deploy web \ --image=ghcr.io/acme/web:1.4.2 \ --runtime-class=general-purpose \ --location=us-central-1 \ --min=2 \ --http-port=8080 ``` Replace the image with your app's image, and `8080` with the port your app listens on. The workload attaches to the project's network named `default`. If the project has no network by that name, the command offers to create it. To use another network, add `--network=NAME`. To skip the prompts, for example in a script or CI, add `--yes`. Run at least two Instances in each location: a rollout replaces one Instance at a time, so a single Instance means a short outage on every deploy. The end of the output is similar to the following: ```text theme={null} Placement "default": locations=[us-central-1], min=2 Runtime class: general-purpose Network: default HTTP service: port 8080 → Datum-managed URL Datum terminates TLS; serve plain HTTP on this port. workload/web created Saved workload.yaml Waiting for rollout. Ctrl-C to detach (rollout continues in background). PLACEMENT LOCATION UPDATED READY OLD PHASE default us-central-1 0 0 1 Pending default us-central-1 1 1 1 Updating default us-central-1 2 2 0 Done Rollout complete in 22s. Publishing... Backends 2 healthy across us-central-1 Edge programmed Certificate issued https://tile-desire-9gpyk.datumproxy.net ``` The last line is the ALB's hostname. Your hostname is different. The command also saves the workload definition to `workload.yaml` in the current directory, replacing any file of that name. The ALB has the same name as the workload, so you can inspect it with the `alb` plugin: ```bash theme={null} datumctl alb describe web ``` ```text theme={null} Name: web Status: Active Status detail: The HTTPProxy has been programmed Default hostname: tile-desire-9gpyk.datumproxy.net Custom hostnames: none Force HTTPS: no Routes: / web:http (network-service) weight=1 (100%) Load balancing: least request (default) Health checks: off Traffic protection: off Basic auth: off Certificates: True (AllCertificatesReady) ``` `Force HTTPS: no` and `Traffic protection: off` are the two gaps the next steps close. **2. Redirect HTTP to HTTPS** To redirect plain HTTP requests to HTTPS, run the following command: ```bash theme={null} datumctl alb update web --force-https ``` ```text theme={null} Application load balancer "web" updated. ``` After a minute or two, a plain HTTP request gets a permanent redirect: ```bash theme={null} curl -sI http://tile-desire-9gpyk.datumproxy.net/pricing ``` ```text theme={null} HTTP/1.1 301 Moved Permanently location: https://tile-desire-9gpyk.datumproxy.net/pricing ``` For a redirect you write yourself, such as one with a different status code, see [HTTP to HTTPS redirect](/docs/alb/httpproxy/https-redirect). **3. Turn on traffic protection** Start the WAF in Observe mode. Observe mode logs requests the WAF would block but lets them through: ```bash theme={null} datumctl alb waf set web --mode Observe --paranoia 1 ``` ```text theme={null} Traffic protection on "web" set to Observe (paranoia 1). ``` After the WAF has seen real traffic and you have checked the logs, switch to Enforce mode: ```bash theme={null} datumctl alb waf set web --mode Enforce --paranoia 1 ``` In Enforce mode, a request that matches a WAF rule gets `403`. To see blocked requests, filter the access logs: ```bash theme={null} datumctl alb logs web --since 1h --code 403 ``` ```text theme={null} TIME METHOD STATUS PATH HOST DURATION 19:39:06 GET 403 /?id=1%27%20OR%20%271%27=%271 tile-desire-9gpyk.datumproxy.net 3ms ``` For how long to stay in Observe and how to tune rules, see [Roll out traffic protection safely](/docs/alb/operations/traffic-protection). **4. Add your domain** To serve the app on your own hostname, run the following command: ```bash theme={null} datumctl alb hostname add web app.example.com ``` Then run `datumctl alb describe web` and follow the hostname through its four steps: claimed, ownership verified, DNS record, and certificate. Datum issues and renews the certificate for you. For the DNS records to create, see [Serve traffic on your own domain](/docs/alb/operations/custom-domains). ## Translate common proxy configuration Each example shows the `datumctl` commands and, where NGINX has an equivalent block, the `nginx.conf` next to the HTTPProxy that replaces it. For the following, see [Translate common proxy configuration](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#translate-common-proxy-configuration) in the reference: * Headers * CORS * Client IPs * Regex matching ### Route paths to different apps Send `/api` to one workload and everything else to another. For more information, see [Path-based routing](/docs/alb/httpproxy/path-routing) and [Send paths to different origins](/docs/alb/operations/routes). Deploy the API as its own workload, on the same network as `web`: ```bash theme={null} datumctl compute deploy api \ --image=ghcr.io/acme/api:2.1.0 \ --runtime-class=general-purpose \ --location=us-central-1 \ --min=2 \ --http-port=8080 ``` This also gives `api` an ALB of its own, which [Require sign-in](#require-sign-in) uses later. Then add a route for `api` to the `web` ALB: ```bash theme={null} datumctl alb route add web --path /api --network-service api --port http ``` ```text theme={null} Route /api added to "web". ``` `compute deploy` creates a network service for each workload, named after the workload, with its HTTP port named `http`. The route sends requests to it. The `web` HTTPProxy now holds three rules: the HTTPS redirect from step 2, the default route, and the `/api` route. The following tabs show the NGINX equivalent and the HTTPProxy rules: ```nginx nginx.conf theme={null} upstream web { server web:8080; } upstream api { server api:8080; } server { location / { proxy_pass http://web; } location /api { proxy_pass http://api; } } ``` ```yaml HTTPProxy rules theme={null} spec: rules: - matches: - headers: - name: x-forwarded-proto type: Exact value: http path: type: PathPrefix value: / filters: - type: RequestRedirect requestRedirect: scheme: https statusCode: 301 - matches: - path: type: PathPrefix value: / backends: - networkService: name: web port: http - matches: - path: type: PathPrefix value: /api backends: - networkService: name: api port: http ``` To see the whole HTTPProxy, run `datumctl get httpproxy web -o yaml`. As in NGINX, the most specific match wins, whatever the order of the rules, so `/api/users` goes to `api` even though its rule comes second. A `PathPrefix` matches whole path segments: `/api` matches `/api/users` but not `/apiv2`. For more matching behavior, see [Route paths to different apps](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#route-paths-to-different-apps) in the reference. ### Redirect and rewrite URLs Remove a path prefix before the request reaches your app. For more information, see [URL rewrite](/docs/alb/httpproxy/url-rewrite). For redirects to a canonical hostname, see [Redirect and rewrite URLs](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#redirect-and-rewrite-urls) in the reference. By default, the ALB forwards the path unchanged, so the API above receives `/api/users`. To strip `/api`, add a `URLRewrite` filter to the `/api` rule. Start from the live HTTPProxy, not a new file. A file that you write from scratch replaces every rule on the ALB, including the HTTPS redirect from step 2. Save the live HTTPProxy to a file: ```bash theme={null} datumctl get httpproxy web -o yaml > httpproxy.yaml ``` In `httpproxy.yaml`, find the rule that matches `/api` and add the `filters` block, so that the rule looks like the `httpproxy.yaml` tab: ```nginx nginx.conf theme={null} location /api/ { proxy_pass http://api/; } ``` ```yaml httpproxy.yaml theme={null} - matches: - path: type: PathPrefix value: /api filters: - type: URLRewrite urlRewrite: path: type: ReplacePrefixMatch replacePrefixMatch: / backends: - networkService: name: api port: http ``` Where you ran `nginx -t` before a reload, compare the file with what's live and validate it on the server: ```bash theme={null} datumctl diff -f httpproxy.yaml datumctl apply -f httpproxy.yaml --dry-run=server ``` Because the file started from the live HTTPProxy, the diff shows only your change: ```text theme={null} @@ -57,6 +59,12 @@ name: api port: http weight: 1 + filters: + - type: URLRewrite + urlRewrite: + path: + replacePrefixMatch: / + type: ReplacePrefixMatch matches: - path: type: PathPrefix ``` Then apply it: ```bash theme={null} datumctl apply -f httpproxy.yaml ``` ```text theme={null} httpproxy.networking.datumapis.com/web configured ``` The first `apply` on an HTTPProxy that a `datumctl alb` or `compute deploy` command created prints a warning about a missing `last-applied-configuration` annotation. The annotation is added for you, and you can ignore the warning. ### Require sign-in Put sign-in in front of an app without running oauth2-proxy or a forward-auth container. For setup with a specific provider, see [Sign in with Google](/docs/alb/authentication/oidc-google) or [Per-user access with Auth0](/docs/alb/authentication/oidc-auth0). Sign-in is a `SecurityPolicy` with an `oidc` section, plus a Secret that holds your client secret. This example protects the `api` workload's own ALB. Save it as `oidc.yaml` and replace the following values: * `CLIENT_ID` and `CLIENT_SECRET`: the OAuth client from your identity provider. For Google, the client ID ends in `.apps.googleusercontent.com`. * `targetRefs` `name`: the HTTPRoute for the ALB. It has the same name as the ALB and the workload. To list them, run `datumctl get httproutes`. * `redirectURL`: your ALB's hostname from `datumctl alb describe api`, followed by `/oauth2/callback`. The Secret needs the `gateway-sync` label, or the ALB can't read it: ```yaml oidc.yaml theme={null} apiVersion: v1 kind: Secret metadata: name: api-oidc labels: networking.datumapis.com/gateway-sync: "true" type: Opaque stringData: client-secret: CLIENT_SECRET --- apiVersion: gateway.envoyproxy.io/v1alpha1 kind: SecurityPolicy metadata: name: api-oidc spec: targetRefs: - group: gateway.networking.k8s.io kind: HTTPRoute name: api oidc: provider: issuer: https://accounts.google.com clientID: CLIENT_ID clientSecret: name: api-oidc redirectURL: https://firefly-inlet-39383.datumproxy.net/oauth2/callback logoutPath: /logout ``` ```bash theme={null} datumctl apply -f oidc.yaml ``` ```text theme={null} secret/api-oidc created securitypolicy.gateway.envoyproxy.io/api-oidc created ``` A request without a session now goes to the identity provider: ```bash theme={null} curl -s -o /dev/null -w '%{http_code} %{redirect_url}\n' https://firefly-inlet-39383.datumproxy.net/ ``` ```text theme={null} 302 https://accounts.google.com/o/oauth2/v2/auth?client_id=CLIENT_ID&code_challenge=... ``` The ALB checks that a user signed in, but not who they are. It has no equivalent of the oauth2-proxy `--email-domain` flag, so restrict who can sign in with your identity provider. For more information, see [Access control](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#access-control) in the reference. ## Production review An app that worked behind its old proxy can still fail behind the ALB, because some defaults differ. Check each item before you move production traffic: * **Response timeout**: about 15 seconds by default, streaming included. Raise it for slow APIs, server-sent events, and LLM streams. See [Response timeout](#response-timeout). * **Uploads**: the WAF rejects large request bodies, even in Observe mode. See [WAF](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#waf) in the reference and [WAF configuration](/docs/alb/configuration/waf). * **WebSockets**: supported to Compute workloads. See [Timeouts, streaming, WebSockets, and gRPC](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#timeouts-streaming-websockets-and-grpc). * **Rate limits**: counted per edge node, not across the whole ALB. See [Rate limits](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#rate-limits). * **Client IP**: your app reads it from forwarded headers. See [Pass the client IP to your app](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#pass-the-client-ip-to-your-app). ### Response timeout A response that takes longer than about 15 seconds gets `504`, whatever your app does. To see it, request a route in your app that takes about 20 seconds. In the following example, `/slow` is such a route: ```bash theme={null} curl -s -o /dev/null -w '%{http_code} %{time_total}s\n' https://tile-desire-9gpyk.datumproxy.net/slow ``` ```text theme={null} 504 15.125767s ``` To raise the limit, attach a `BackendTrafficPolicy` to the ALB's Gateway, which has the same name as the ALB: ```yaml timeout.yaml theme={null} apiVersion: gateway.envoyproxy.io/v1alpha1 kind: BackendTrafficPolicy metadata: name: web-timeout spec: targetRefs: - group: gateway.networking.k8s.io kind: Gateway name: web timeout: http: requestTimeout: 120s ``` ```bash theme={null} datumctl apply -f timeout.yaml ``` ```text theme={null} backendtrafficpolicy.gateway.envoyproxy.io/web-timeout created ``` After the change reaches the edge, usually within a minute, the same 20-second request succeeds: ```text theme={null} 200 20.157289s ``` Unlike NGINX `proxy_read_timeout`, which limits the gap between reads, this limits the whole response. For streaming and WebSocket behavior, see [Timeouts, streaming, WebSockets, and gRPC](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#timeouts-streaming-websockets-and-grpc) in the reference. ### TLS and custom domains Datum issues and renews a certificate for each hostname on the ALB. These leave with the proxy container: * Certificate files * certbot * ACME settings To check certificate status, run `datumctl alb describe web` and read the `Certificates` line. For more information, see [Serve traffic on your own domain](/docs/alb/operations/custom-domains) and [Serve a wildcard hostname](/docs/alb/configuration/wildcard-hostnames). ### Health checks and failover The ALB has no active health checks. Its passive health checks watch real responses instead: once you turn them on, an Instance that returns a run of 5xx errors is taken out of rotation, separately on each edge node. For settings, see [Passive health checks](/docs/alb/configuration/load-balancing#passive-health-checks). For how Instances are replaced during an update, see [How rollouts work](/docs/compute/placement-and-scaling#how-rollouts-work). For test results, see [Load balancing and rollouts](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#load-balancing-and-rollouts) in the reference. To turn on passive health checks with the default settings, run the following command: ```bash theme={null} datumctl alb update web --health-checks ``` `datumctl alb describe web` then shows: ```text theme={null} Health checks: passive: eject after 5 consecutive 5xx for 30s, max 50% ejected ``` To change the thresholds, use `--consecutive-5xx` and `--base-ejection-time`. ### Headers and request limits * Your app sees the client's address in forwarded headers, not on the connection. * The WAF caps request size. * Responses go out uncompressed unless your app compresses them. For numbers, see [Limits and quotas](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#limits-and-quotas) in the reference and [ALB limits and quotas](/docs/alb/limits-and-quotas). These are the forwarding headers your app receives on each request: ```text theme={null} x-forwarded-for: 47.147.206.200 x-forwarded-proto: https x-envoy-external-address: 47.147.206.200 x-request-id: b7970a87-d84b-4503-aa96-6a7fd6bfc174 ``` ### Placement and scaling The ALB runs in every Datum location and scales with traffic, so you don't size it. Where you run Instances decides how close your app is to users. For more information, see [Placement and scaling](/docs/compute/placement-and-scaling). To see where your Instances run, run the following command: ```bash theme={null} datumctl compute instances --workload=web ``` ```text theme={null} NAME WORKLOAD LOCATION INTERNAL IP TYPE AGE STATUS web-default-us-central-1-0 web us-central-1 fd20:0:4d::1:0:0/96 sandbox 1m Available web-default-us-central-1-1 web us-central-1 fd20:0:4d::2:0:0/96 sandbox 53s Available 2 instances — 2 Available, 0 Pending, 0 Failed ``` ### Access metrics For the metrics Datum publishes for each ALB, see [Metrics](/docs/alb/observability#metrics). To send them to Grafana Cloud or another Prometheus remote-write endpoint, see [Metrics export](/docs/operations/metrics-export). ### Access logs `datumctl alb logs` shows recent requests for one ALB. `datumctl telemetry logs` shows the full Envoy fields, including the response flags that explain 502 and 503 errors. For more information, see [Access logs](/docs/alb/observability#access-logs). For the commands used in testing, see [Verify and debug](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#verify-and-debug) in the reference. ```bash theme={null} datumctl alb logs web --since 15m --limit 5 ``` ```text theme={null} TIME METHOD STATUS PATH HOST DURATION 19:36:31 GET 200 /?id=1%27%20OR%20%271%27=%271 tile-desire-9gpyk.datumproxy.net 105ms 19:34:40 GET 504 /slow tile-desire-9gpyk.datumproxy.net 15058ms 19:34:25 HEAD 301 /pricing - 3ms 19:34:25 GET 200 /users tile-desire-9gpyk.datumproxy.net 95ms 19:34:25 GET 200 /users tile-desire-9gpyk.datumproxy.net 99ms ``` The `PATH` column shows the path after any rewrite, so a request for `/api/users` appears as `/users`. To find which Instance served a failing request, add `--code` and `-o wide`: ```bash theme={null} datumctl alb logs web --since 15m --code 504 -o wide ``` ```text theme={null} TIME METHOD STATUS PATH HOST DURATION REQUEST ID UPSTREAM 19:34:40 GET 504 /slow tile-desire-9gpyk.datumproxy.net 15058ms 3c744717-f341-488e-8553-891cd862609a [fd20:0:4d::2:0:0]:8080 ``` `UPSTREAM` is the Instance's private address, which matches the `INTERNAL IP` column of `datumctl compute instances`. For every Envoy field, including `response_flags`, query the logs with the `telemetry` plugin. To install it, run `datumctl plugin install telemetry`. A query is a [LogQL](https://grafana.com/docs/loki/latest/query/) stream selector, and without a hostname it matches every ALB in the project. To see the `504` responses for one ALB, match its hostname with the `authority` label: ```bash theme={null} datumctl telemetry logs '{authority="tile-desire-9gpyk.datumproxy.net", response_code="504"}' --since 1h ``` To list every label you can match on, run `datumctl telemetry labels`. A `504` from the response timeout has `response_flags="UT"`, upstream timeout. ### Security review An ALB that `compute deploy` creates has no WAF. One that `datumctl alb create` creates starts with the WAF in Enforce mode. Confirm which you have: ```bash theme={null} datumctl alb waf describe web ``` With no WAF, the output is one line: ```text theme={null} Traffic protection: off ``` After step 3 of the deploy, it shows the mode and sensitivity: ```text theme={null} Traffic protection: Enforce Paranoia: 1 Policy: web ``` To turn the WAF on without blocking real users, see [Roll out traffic protection safely](/docs/alb/operations/traffic-protection). For sign-in options, see [Authentication](/docs/alb/authentication/overview). For test results, see [Protection](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#protection) and [Access control](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#access-control) in the reference. ### Rollback plan Keep the old proxy container deployable until the ALB has carried production traffic for a while. Lower your DNS TTL before cutover so you can point back quickly. See [Cut over](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#cut-over) in the reference. ## Apps that need a gateway proxy Some apps don't speak HTTP. They speak a gateway protocol, and the proxy in front of them translates: * **FastCGI**: PHP-FPM and similar process managers. NGINX reaches them with `fastcgi_pass`. * **uWSGI**: the uwsgi protocol used by many Python apps. NGINX reaches them with `uwsgi_pass`. * **CGI**: older scripts that the web server runs once per request. The ALB forwards only HTTP, and an Instance runs one container, so a proxy sidecar can't do the translation. You have two options. **Switch to an image that serves HTTP directly.** This is the simpler option. Examples: * PHP: FrankenPHP, or Apache with `mod_php` * Python: gunicorn or uvicorn Then follow this guide from the start. For details, see [Before you begin](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#before-you-begin) in the reference. **Run the gateway as its own workload.** Deploy your existing proxy, such as NGINX with `fastcgi_pass`, as one workload behind the ALB, and your app as another. Both attach to the same network, in the same or different locations. The gateway reaches your app Instances by their private IPv6 addresses. To find them, run the following command: ```bash theme={null} datumctl compute instances --workload=app ``` ```text theme={null} NAME WORKLOAD LOCATION INTERNAL IP TYPE AGE STATUS app-default-us-central-1-0 app us-central-1 fd20:0:4d::3:0:0/96 sandbox 3m Available 1 instances — 1 Available, 0 Pending, 0 Failed ``` The gateway connects to the address before the `/96`, in square brackets, such as `fastcgi_pass [fd20:0:4d::3:0:0]:9000;`. Keep the following points in mind: * Private DNS names aren't available in the preview, so pass the app Instances' addresses to the gateway in environment variables or a ConfigMap. * Addresses can change when Instances are replaced. To keep them, see [Keep addresses after scale-down](/docs/galactic-vpc/configuration#keep-addresses-after-scale-down). * Networks don't filter traffic between Instances yet. Any Instance on the network can reach your FastCGI or uWSGI port, so put these two workloads on a network of their own. For more information, see [Addressing](/docs/galactic-vpc/networks/addressing) and [Traffic and connecting](/docs/galactic-vpc/networks/traffic). ## Verify the migration Test through the ALB hostname while the old proxy still serves production, and switch DNS last. Start by checking that each path reaches the right app: ```bash theme={null} for p in / /api/users; do printf '%-12s ' "$p"; curl -s -o /dev/null -w '%{http_code}\n' "https://tile-desire-9gpyk.datumproxy.net$p" done ``` A `200` alone doesn't prove the right app answered. Check for something only that app returns, such as a version endpoint or a response header. The [verification checklist](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md#verify-the-migration) in the reference has: * A `curl` check for each route, redirect, and production-review item * Log and metric checks If the ALB isn't serving, see [When a load balancer is not serving](/docs/alb/troubleshooting). ## Clean up After cutover and your rollback window, retire: * The old proxy container * Its configuration * Any certificate renewal jobs If you deployed a test app to follow along, delete the policies you applied first. `compute destroy` doesn't remove them: ```bash theme={null} datumctl delete trafficprotectionpolicy web datumctl delete -f timeout.yaml datumctl delete -f oidc.yaml ``` Then destroy each workload: ```bash theme={null} datumctl compute destroy web -y ``` ```text theme={null} Workload: web Placements: 1 Locations: us-central-1 Min replicas: 2 URLs: https://tile-desire-9gpyk.datumproxy.net workload/web deleted. ``` `compute destroy` removes: * The workload * Its HTTPProxy * Its network service Run `datumctl compute destroy api -y` for the `api` workload. If you created a network only for this guide, delete it after both workloads are gone: ```bash theme={null} datumctl delete network NETWORK_NAME ``` For more information, see [Compute operations](/docs/compute/operations). ## What's next * [ALB as httpd reference](https://github.com/datum-labs/shorts/blob/main/gists/alb/alb-as-httpd-reference.md): every directive, limit, and test result behind this guide * [ALB overview](/docs/alb/overview) * [HTTPProxy](/docs/alb/httpproxy/overview) * [Web application firewall](/docs/alb/configuration/waf) * [Authentication](/docs/alb/authentication/overview) * [Placement and scaling](/docs/compute/placement-and-scaling) * [ALB limitations and roadmap](/docs/alb/limitations-and-roadmap) # Guides Source: https://datum.net/docs/guides/overview Step-by-step guides for common Datum tasks: replace your web server with an ALB, deploy an app, serve it on your own domain, require sign-in, and point DNS at a load balancer. ## Featured > [card] [Use ALB as your HTTP proxy](/docs/guides/alb-as-httpd) > > Replace the NGINX, Apache, Caddy, or Traefik container in front of your app with Datum's Application Load Balancer, and deploy only your app. ## More tasks * [Deploy an app](/docs/getting-started/first-workload) on Compute. * [Serve traffic on your own domain](/docs/alb/operations/custom-domains) with an Application Load Balancer. * [Require sign-in](/docs/alb/authentication/overview) before requests reach your app. * [Point a domain at a load balancer](/docs/domain-dns/point-to-alb) with Datum DNS. Have a guide you'd like to see? Suggest it in [GitHub discussions](https://github.com/orgs/datum-cloud/discussions/categories/feature-requests). # Introduction Source: https://datum.net/docs/index Datum is a modern edge cloud, backed by open source, focused on the builders and service providers of the AI era. Datum is an edge cloud where you can run containers close to your users, publish them behind a load balancer with HTTPS, connect your existing services through secure tunnels, and host DNS for your domains. * We are 100% focused on builders and modern, "alt cloud" service providers. * Our code is open source (AGPLv3), so you can run it anywhere if needed. * We prioritize agent-first and developer experiences. Datum is still being built. The product is evolving quickly, documentation is growing with it, and Datum is free during public beta (see [pricing](https://www.datum.net/pricing)). To see where each product stands, check the [Roadmap](/docs/platform/roadmap). ## Get started > [card] [Getting Started](/docs/getting-started/account-setup) > > Set up your account and Project, and run your own app. > [card] [Platform](/docs/platform/kubernetes) > > How Datum's edge cloud is built, from the control plane to the backbone. > [card] [datumctl](/docs/datumctl/overview) > > The CLI for people, scripts, and agents. ## Products > [card] [gVPC](/docs/galactic-vpc/overview) > > Private IPv6 networks for your Compute Instances, spanning every location they run in. > [card] [Compute](/docs/compute/overview) > > Run container images and unikernels as isolated Instances close to your users. > [card] [Application Load Balancer](/docs/alb/overview) > > Intelligent edge proxies and routing purpose-built for AI workloads and agent traffic. > [card] [Connectors](/docs/connectors/overview) > > Secure tunnels to bridge your existing infrastructure into Datum's edge cloud. > [card] [Domains and DNS](/docs/domain-dns/overview) > > Track domains, host authoritative DNS, and resolve private names inside your VPCs. > [card] [Locations](/docs/locations) > > Datum's global footprint of Regions and Availability Zones. ## Agents > [card] [Agents](/docs/agents/overview) > > Skills, MCP, llms.txt, and datumctl — everything AI agents need to work with Datum. ## Why Datum Our platform provides internet-scale primitives — intelligent edge proxies, scalable compute, private backbones, cloud onramps, peering, meet-me rooms — with an experience that is 100% programmatic, agent-first, and backed by AGPLv3 open source. We believe an edge cloud in the AI era is different from a traditional CDN, mainly because the use case is intelligence, not caching. We also think that open — open source and open ecosystems — is a critical requirement in a world that is increasingly fragmented and regulated. We serve builders and modern service providers, which we lovingly call [Alt Clouds](https://www.alt-cloud.org/). We aim to support thousands of them, so neutrality and transparency are important in everything we do. We started Datum because we saw three big trends: 1. Intelligence and data everywhere 2. Thousands of new providers 3. An era of increased regulation > [!NOTE] > > Want to learn more? Read Zac's blog post, [Every agent needs an edge](https://www.datum.net/blog/every-agent-needs-an-edge/). ## Background We founded Datum in late 2024. Our team includes infrastructure veterans who helped build companies like Packet, Voxel, SoftLayer, StackPath, Highwinds, Zscaler, and more. We're fortunate to have support from investors at Amplify Partners, Cervin Ventures, CRV, Encoded Ventures, ex/ante, Step Function VC, and Vine Ventures. To learn how Datum is built, see [Platform API](/docs/platform/kubernetes). To take part, see [Get involved](/docs/get-involved). # Locations Source: https://datum.net/docs/locations Datum's global footprint of Regions and Availability Zones, and how to check which locations are available to use in your project Datum’s infrastructure footprint is organized into points of presence (PoPs) that are packaged into Regions and Availability Zones (AZs). See our [Locations](https://www.datum.net/locations/) for a visual overview of all PoPs. Each Region represents a specific geographic and network boundary, while each AZ provides independent capacity within that Region. This structure supports predictable latency, fault isolation, and regulatory alignment. Similar to the large public clouds, Datum uses a country-anchored naming format with the following elements: | **Field** | **Description** | | - | - | | Geography | Two-letter ISO country code (example: US, DE, IN) | | Cardinal Direction | Directional indicator within the country: north, south, east, west, or central. | | Number | Region index within that location. | | Count | Availability Zone identifier: a, b, c, etc. | For example, here is how we describe the first region and availability zone in the eastern US. ```text theme={null} -- ``` `Example: us-east-1a` Regions and AZs are defined by distance and policy boundaries. * When Points of Presence in the same country are separated by more than about 5 milliseconds round-trip time (RTT), Datum creates a new Region. * If multiple PoPs exist in the same metro but operate independently (for example, separate network clusters or facilities), they become separate AZs. Increment the Count value to the next letter. * If PoPs are within about 5 milliseconds RTT and part of the same operational domain, they remain in the same Region. Example: us-east-1a, us-east-1b, us-east-1c. *Note: regulatory and export control requirements may restrict or define where data and workloads can reside. These boundaries take precedence over latency-based placement rules.* ## Regions | **Region Code** | **Metropolitan Area** | | - | - | | ae-north-1 | Dubai | | au-east-1 | Sydney | | br-east-1 | São Paulo | | ca-east-1 | Toronto | | cl-central-1 | Santiago | | de-central-1 | Frankfurt | | gb-south-1 | London | | in-west-1 | Mumbai | | jp-east-1 | Tokyo | | nl-west-1 | Amsterdam | | sg-central-1 | Singapore | | us-central-1 | Dallas | | us-east-1 | Ashburn | | us-east-2 | New York City | | us-west-1 | San Jose, California | | za-central-1 | Johannesburg | ## Check what's available to your project The table above is Datum's global footprint. What you can deploy into is a narrower, per-project question: Datum represents each usable location as a `Location` resource, and the ones inside your project's own control plane are the ones offered to you. `datumctl get locations` lists them: ```bash theme={null} datumctl get locations --project your-project-id ``` ```text theme={null} NAME CLASS CITY AGE READY REASON us-central-1 datum-managed DFW 20h us-east-1 datum-managed IAD 20h ``` Each row is a location you can place work in today. `NAME` is the identifier you reference elsewhere, and matches the region codes in the table above: `us-central-1` is Dallas, `us-east-1` is Ashburn. `CITY` is the airport-style code for the metro it serves. `CLASS` is the kind of capacity behind it, where `datum-managed` is Datum's own footprint. > [!TIP] > > This list reflects your active project scope. If a location you expect is missing, confirm you are pointed at the right project. See [Contexts & scoping](/docs/datumctl/contexts-and-scoping). For the full definition of a location, including its topology, request YAML or JSON: ```bash theme={null} datumctl get location us-central-1 --project your-project-id -o yaml ``` ```yaml theme={null} apiVersion: locations.miloapis.com/v1alpha1 kind: Location metadata: name: us-central-1 spec: locationClassRef: name: datum-managed topology: topology.datum.net/city-code: DFW topology.datum.net/region: us-central-1 ``` ## Region tracing Visit [https://edge.datum.net/](https://edge.datum.net/) to learn which Datum Region and availability zone you're reaching via anycast. If you're using Datum Tunnels, this will be your entrypoint into our network. You can also access our zones and locations programmatically: * JSON structured response [https://edge.datum.net/api/edge-info](https://edge.datum.net/api/edge-info) * Plaintext [https://edge.datum.net/api/trace](https://edge.datum.net/api/trace) ## Availability commitments Uptime percentages, service credits, and per-location availability commitments aren't documented yet. Datum is in active development; see [Account setup](/docs/getting-started/account-setup) for the current beta terms. # Activity Logs Source: https://datum.net/docs/operations/activity-logs Query and filter audit logs in the Datum Cloud portal to track changes across your organizations and projects ## Portal Access activity logs from the organization's settings page. ![Activity Logs](https://mintcdn.com/datum-4926dda5/FYsOAYxQDJnosEsI/images/activity-logs.png?fit=max\&auto=format\&n=FYsOAYxQDJnosEsI\&q=85\&s=4017077cd793a78d54e16b4972af5c3c) You can also query and filter audit logs for a project by navigating to a project's settings page. How long audit logs are retained isn't documented yet. ## Command line With `datumctl`, you can browse audit logs from the terminal and pipe them into `jq` or other tools. For example, to see the last 7 days of audit logs: ```bash theme={null} datumctl activity audit --start-time now-7d ``` By default, `datumctl activity audit` retrieves the last 24 hours of activity. You can use the `--filter` option to specify a [CEL expression](https://cel.dev/) to filter the audit logs returned. For commands and examples, see [Audit logs and events](/docs/datumctl/activity/audit-and-events) and [Change history and feed](/docs/datumctl/activity/change-history-and-feed). # Datum Assistant Source: https://datum.net/docs/operations/datum-assistant Ask Patch, the Datum Cloud assistant, about your resources in natural language, and let it diagnose problems for you. Patch uses diagnostic tools from the Datum services your project uses, so it can work out why a Compute workload isn't available, why a load balancer isn't serving, or why a network isn't ready. It never changes anything on its own: it proposes a change as a plan, shows you the exact manifests, and applies them only when you approve. Patch is available in the terminal with `datumctl assistant`. Conversations are kept by Datum, so you can pick one up from any machine you're signed in on. ```bash theme={null} datumctl plugin install assistant datumctl assistant chat "Why is the api-backend workload not available?" ``` Usage is metered by messages and tokens. See your organization's [Usage](/docs/organizations/usage). > [card] [datumctl AI assistant](/docs/datumctl/ai/assistant) > > Start a conversation, choose the project, and manage your conversations. To connect your own AI tools to Datum instead, see [Datum MCP](/docs/agents/datum-mcp) and the [Agents overview](/docs/agents/overview). # Metrics Export Source: https://datum.net/docs/operations/metrics-export The metrics Datum publishes for your project, and how to export them to Grafana Cloud or any Prometheus remote-write endpoint. > [!NOTE] > > **Working with this resource via an AI agent?** Datum publishes a [Metrics Export skill](/docs/agents/skills/metrics-export) that teaches agents the canonical patterns for this resource. Datum collects metrics from the services in your project and keeps them in your project's metrics store. The [Cloud Portal](https://cloud.datum.net) charts them on each resource's pages. To see them next to your other metrics, export them to [Grafana Cloud](https://grafana.com/) or any other endpoint that accepts Prometheus remote write. ## What Datum measures | Product | What you can measure | Reference | | - | - | - | | Application Load Balancer | Requests per second by location and response code, origin response time, and firewall events by outcome and rule | [ALB metrics](/docs/alb/observability#metrics) | | Compute | CPU and memory for each Instance, ready and desired Instances for each workload and placement, and traffic to published workloads | [Compute metrics](/docs/compute/observability#metrics) | | gVPC | Whether each network is ready, and each Instance's private address | [gVPC metrics](/docs/galactic-vpc/observability#metrics) | | Domains and DNS | Whether each zone and record set is serving, and whether each domain is verified | [Domains and DNS metrics](/docs/domain-dns/observability#metrics) | | Connectors | Traffic through the load balancers that route to each Connector | [Connector metrics](/docs/connectors/observability#metrics) | Metrics come in two kinds: * **Traffic and usage metrics** measure what your resources are doing, such as requests per second or CPU time. Their names start with the component that records them, such as `envoy_` for load balancer traffic, `coraza_` for the firewall, and `datum_compute_instance_` for Instance CPU and memory. * **Status metrics** report the state of your resources, such as whether a load balancer is programmed or a domain is verified. They follow a consistent pattern, so once you know one, you know them all: | Pattern | What it reports | | - | - | | `_info` | One series for each resource, with descriptive labels. Use it to count resources or join in labels. | | `_status_condition` | `1` when a condition is true and `0` when it's false, with `condition`, `reason`, and `status` labels. | | `_status_condition_last_transition_time` | When each condition last changed, as a Unix timestamp. | | `_created` | When the resource was created, as a Unix timestamp. | ### Labels on every series | Label | Meaning | | - | - | | `resourcemanager_datumapis_com_project_name` | Your project ID. Include it in every query, because resource names are only unique within a project. | | `resource_name` | On status metrics, the name of the resource. | | `label_topology_kubernetes_io_region` or `region` | On traffic and usage metrics, the Datum location, such as `us-west-1`. | Queries use [MetricsQL](https://docs.victoriametrics.com/victoriametrics/metricsql/), which also accepts standard PromQL. ## Export with the Cloud Portal You can set up an export policy in the Datum Cloud portal. Go to the project of your choice, find the **Metrics** tab, and follow the instructions. ![Grafana Access](https://mintcdn.com/datum-4926dda5/9pdr37W0ZlTfIFim/images/grafana-access.png?fit=max\&auto=format\&n=9pdr37W0ZlTfIFim\&q=85\&s=dd3607d7505fb2a29720e4cb9399d4d6) Step 1: Provide the Grafana instance URL ![Grafana Finish Setup](https://mintcdn.com/datum-4926dda5/9pdr37W0ZlTfIFim/images/grafana-finish-setup.png?fit=max\&auto=format\&n=9pdr37W0ZlTfIFim\&q=85\&s=f8e5ce9b6a19d6f8897568275ea04556) Step 2: Configure your export policy ## Export with datumctl An `ExportPolicy` defines: * **Sources**, which select the metrics to export. Each source needs a MetricsQL selector in `metrics.metricsql`, and exports only the matching series. To export everything in your project, use `{__name__=~".+"}`. The selector must be a plain series selector such as `{label="value"}`, not a function like `rate()`. Datum limits it to your project for you. * **Sinks**, which send the selected sources to a Prometheus remote-write endpoint. **1. Store your credentials in a Secret** Create a Secret with the credentials for your endpoint. It must be of type `kubernetes.io/basic-auth`, with `username` and `password` keys. For [Grafana Cloud](https://grafana.com/products/cloud/), the username is your instance ID and the password is an access token with permission to write metrics. ```yaml theme={null} apiVersion: v1 kind: Secret metadata: name: grafana-cloud-credentials type: kubernetes.io/basic-auth stringData: username: "123456" password: "glc_..." ``` ```bash theme={null} datumctl apply -f grafana-secret.yaml ``` **2. Create the export policy** This policy exports only load balancer traffic and firewall metrics. To export everything, set `metricsql` to `{__name__=~".+"}`. ```yaml theme={null} apiVersion: telemetry.miloapis.com/v1alpha1 kind: ExportPolicy metadata: name: grafana-cloud-export spec: sources: - name: alb-metrics metrics: metricsql: '{__name__=~"envoy_vhost_vcluster_upstream_rq.*|coraza_envoy_filter_request_events_total"}' sinks: - name: grafana-cloud sources: - alb-metrics target: prometheusRemoteWrite: endpoint: https://prometheus-prod-01-prod-us-east-0.grafana.net/api/prom/push authentication: basicAuth: secretRef: name: grafana-cloud-credentials batch: maxSize: 500 timeout: "5s" retry: maxAttempts: 3 backoffDuration: "1s" ``` Replace `endpoint` with your own remote-write URL. If you leave out `batch` or `retry`, the defaults are batches of 500 every `5s` and 3 attempts `5s` apart. ```bash theme={null} datumctl diff -f exportpolicy.yaml datumctl apply -f exportpolicy.yaml ``` **3. Check that it's exporting** ```bash theme={null} datumctl describe exportpolicy grafana-cloud-export ``` The policy's `Ready` condition is `True` once every sink is accepted. Each sink also has an `Accepted` condition in `status.sinks`. If it's `False` with reason `InvalidAuthentication`, the Secret is missing, isn't of type `kubernetes.io/basic-auth`, or lacks a `username` or `password` key. `Accepted` means the configuration is valid, not that the endpoint received data. To confirm delivery, query your endpoint for your project's metrics after a minute or two. ## Datum dashboards in Grafana Cloud To add prebuilt Datum dashboards in Grafana Cloud, go to **Dashboards**, select **New dashboard**, then select **Import dashboard**. Load the Datum dashboards using the following Grafana IDs. ### Datum Overview Grafana ID: **23939** Provides real-time visibility into your load balancers, including: * Total HTTP proxies, hostnames, and domains * Upstream requests per second (RPS) by region * Global response codes and latency percentiles * Domain and proxy health status ### Datum Proxy Grafana ID: **24261** Offers a real-time view of one load balancer's hostnames and regional traffic, including: * Regional RPS and upstream response codes * Latency percentiles (p50, p90, p99) * Domain and proxy status For additional help, consult the [Grafana Cloud documentation](https://grafana.com/docs/grafana-cloud/). # Project Suspension Source: https://datum.net/docs/operations/project-suspension What happens when a project is suspended, what is stopped versus retained, and how to appeal or self-remediate Project suspension is a reversible, non-destructive stop on a project. Datum can suspend a project for policy, billing, security, or compliance reasons. Nothing about your project is deleted — running work is stopped, and your configuration, IP allocations, and other resources are retained so the project can start back up on reinstatement. ## What happens when a project is suspended When a project is suspended: * **The project freezes.** You can read your project and its resources, but you can no longer create new resources or change existing configuration. Deleting resources is still allowed, so you can clean up or offboard while suspended. * **Everything running stops — but nothing is deleted by Datum.** Compute instances are stopped, published endpoints stop serving, open connections drop, and the project stops executing work. * **Your configuration and allocations are preserved.** Configuration, IP allocations, DNS records, and enabled-service state all remain in place. Think of suspension as a stop, not a deletion. The essential guarantee is **reversibility** — Datum never destroys your stored state to suspend a project. ## What's stopped vs. retained | Resource | Behavior | | - | - | | Compute instances | Stopped, then restarted fresh | | In-memory state and open connections | Not preserved | | Published endpoints | Stop serving | | DNS records | Retained | | Configuration | Retained | | IP allocations | Retained | | Enabled-service state | Retained | ## What to expect on reinstatement When a suspended project is reinstated: * The project **unfreezes** and accepts writes again. * Everything **starts back up automatically** — instances start fresh and endpoints serve again. No redeploy is required. * The full history of when and why the project was suspended and reinstated is retained for audit and appeal. Coming back up takes about as long as a normal start. Instances restart fresh rather than picking up mid-computation, so any in-memory state and open connections from before the suspension are gone. ## How to appeal or resolve a suspension How a suspension is lifted depends on the reason it was set. | Reason | How to resolve | | - | - | | **Billing** | Self-remediate — resolve the outstanding payment issue and the project is reinstated automatically. | | **Fraud, abuse, or compliance** | Appeal — Datum reviews the suspension before it can be lifted. | If your project's suspension is set for a billing reason, resolve the payment to reinstate it automatically. If it's set for fraud, abuse, or compliance, you can submit an appeal for review. Datum reviews appeals for these types of suspensions before lifting them. ## Compute instances Compute instances in a suspended project are **stopped, not hibernated**. Their [placement](/docs/compute/placement-and-scaling) and quota allocation are retained, so on reinstatement they start fresh with no redeploy. Compute Instances don't have persistent disks, so any files written at runtime are lost. In-memory state and open network connections do not survive a suspension. Design anything you run on Datum to tolerate a restart. # Secrets Source: https://datum.net/docs/operations/secrets Create and manage secret resources in a Datum project, from the Cloud Portal or with datumctl. ![Secrets](https://mintcdn.com/datum-4926dda5/9pdr37W0ZlTfIFim/images/secrets.png?fit=max\&auto=format\&n=9pdr37W0ZlTfIFim\&q=85\&s=6bcfb0ec852ccc95b5e69393b1e8fdf4) ## Create a Secret with datumctl A Secret is a Kubernetes-style resource, so you create it with a manifest and `datumctl apply`. To create a Secret, follow these steps: 1. Save the following manifest as `api-secrets.yaml`: ```yaml theme={null} apiVersion: v1 kind: Secret metadata: name: api-secrets type: Opaque stringData: db-password: PASSWORD ``` Replace `PASSWORD` with the value to store. 2. To create the Secret in your active project, run the following command: ```bash theme={null} datumctl apply -f api-secrets.yaml ``` To target a different project, add `--project your-project-id`. For more information, see [Contexts and scoping](/docs/datumctl/contexts-and-scoping). > [!WARNING] > > A manifest with `stringData` holds the secret value in plain text. Don't commit it to source control. ## List and inspect Secrets To list the Secrets in your project, run the following command: ```bash theme={null} datumctl get secrets ``` To inspect one Secret, run the following command: ```bash theme={null} datumctl describe secret api-secrets ``` ## Update or delete a Secret To change a Secret, edit the manifest and run `datumctl apply -f` again. To delete it, run the following command: ```bash theme={null} datumctl delete secret api-secrets ``` ## Use a Secret Compute workloads can read a Secret as environment variables or as files. For more information, see [Configuration](/docs/compute/configuration). # Organizations Source: https://datum.net/docs/organizations/overview Organizations hold your Projects, team members, billing, and quotas. Documentation is coming soon. > [!NOTE] > > **Coming soon.** Datum is still being built, and documentation for this topic is on the way. An organization is the top level of Datum's data model. It holds your [Projects](/docs/projects/overview), the team members who can work in them, your payment method, and the quotas that apply across them. This page will describe how to create and manage organizations, and how they relate to Projects. ## In this section * [Team management](/docs/organizations/team-management) — the Owner, Editor and Viewer roles. Steps for inviting members are coming soon. * [Usage](/docs/organizations/usage) — see what your organization uses and how it compares to its quotas. ## In the meantime * [Account setup](/docs/getting-started/account-setup) — create an account, an organization, and a Project. # Team Management Source: https://datum.net/docs/organizations/team-management Invite team members to an organization and assign them roles. Documentation is coming soon. > [!NOTE] > > **Coming soon.** Datum is still being built, and documentation for this topic is on the way. To collaborate with team members, you create or join an organization. This page will describe how to invite team members, change their roles, and remove them. ## Roles Organizations support the following coarse-grained roles: * **Owners** - Full control over the organization and all resources. * **Editors** - Control over all resources in an organization, except for team members. * **Viewers** - Read-only access to all resources in an organization. ## In the meantime * [Account setup](/docs/getting-started/account-setup) — create an account and an organization. * [Service accounts](/docs/projects/service-accounts) — non-human identities for automation. # Usage Source: https://datum.net/docs/organizations/usage See what your organization uses on Datum, and how that compares to its quotas. Datum is in active development and is free during public beta, with reasonable limits on resources. Resource limits (quotas) can be found under the settings for Organizations and [Projects](/docs/projects/overview). For how usage is priced, see [pricing](https://www.datum.net/pricing). ## See your usage To see your organization's usage in the [Cloud Portal](https://cloud.datum.net), select your organization, then select **Usage**. The page shows metered consumption across every project in the organization for the current billing cycle. To look at an earlier cycle or a single project, use the filters at the top. Usage is grouped by product. For example, Compute meters the CPU and memory allocated to your Instances and how long they run, and DNS meters active records and the queries Datum answers. ![The organization Usage page with a billing cycle filter, a project filter, and a usage summary table grouped by product: Assistant, Compute, and DNS](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/organization-usage.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=df78ff75172ab7309bf9b916e375f7ea) Metered usage across every project in an organization ## Related * [Compute limits and quotas](/docs/compute/limits-and-quotas) * [gVPC limits and quotas](/docs/galactic-vpc/limits-and-quotas) * [ALB limits and quotas](/docs/alb/limits-and-quotas) * If a quota constrains your work, contact [support@datum.net](mailto:support@datum.net). # Architecture Source: https://datum.net/docs/platform/architecture How Datum's edge cloud is put together, from the control plane to the backbone. Documentation is coming soon. > [!NOTE] > > **Coming soon.** Datum is still being built, and documentation for this topic is on the way. This page will describe how the Datum platform fits together: the Kubernetes-style control plane, the global network and its locations, and how products such as Compute, [gVPC](/docs/galactic-vpc/overview), the [Application Load Balancer](/docs/alb/overview), [Connectors](/docs/connectors/overview), and DNS are built on top of it. ## In the meantime * [Platform API](/docs/platform/kubernetes) — how Datum's platform and API are built on Kubernetes resources. * [Envoy](/docs/platform/envoy) — the proxy behind the Application Load Balancer. * [Locations](/docs/locations) — Datum's regions and availability zones. * [Overview](/docs/) — what Datum is and who it is for. ## Follow the roadmap > [card] [Platform roadmap](/docs/platform/roadmap) > > See what Datum is building now and what comes next. # Ecosystem Source: https://datum.net/docs/platform/ecosystem The partners and open source projects Datum builds on. Documentation is coming soon. > [!NOTE] > > **Coming soon.** Datum is still being built, and documentation for this topic is on the way. This page will describe the partners and open source projects that power Datum, and how the ecosystem around the platform fits together. ## In the meantime * [Architecture](/docs/platform/architecture) — how the platform is put together. * [Envoy](/docs/platform/envoy) — the edge proxy behind the [Application Load Balancer](/docs/alb/overview). * [Iroh relays](/docs/platform/iroh-relays) — the QUIC connectivity layer behind [Connectors](/docs/connectors/overview). ## Follow the roadmap > [card] [Platform roadmap](/docs/platform/roadmap) > > See what Datum offers today and what comes next. # Envoy Source: https://datum.net/docs/platform/envoy How Datum uses the Envoy proxy and Envoy Gateway as the data plane for the Application Load Balancer. [Envoy](https://www.envoyproxy.io/) is the open source, high-performance proxy that handles Layer 7 traffic on Datum. You use it as Datum's [Application Load Balancer (ALB)](/docs/alb/overview). The ALB is built on Envoy and configured through [Envoy Gateway](https://gateway.envoyproxy.io/) and the Kubernetes [Gateway API](https://gateway-api.sigs.k8s.io/). The ALB started life as Datum's "AI Gateway", and it's still built for AI and agent traffic as much as for websites and APIs. ```mermaid theme={null} %%{init: {"flowchart": {"subGraphTitleMargin": {"top": 6, "bottom": 18}}}}%% flowchart TB B["① browser · West Coast
anycast HTTPS · app.example.com"] A["① AI agent · Central US
anycast HTTPS · api.example.com"] M["① mobile app · East Coast
anycast HTTPS · api.example.com"] subgraph west["us-west-1 · SJC"] EW["② Envoy · ALB
TLS · WAF · routes · auth
no Instance in this location"] end subgraph central["us-central-1 · DFW"] EC["② Envoy · ALB
TLS · WAF · routes · auth"] end subgraph east["us-east-1 · IAD"] EE["② Envoy · ALB
TLS · WAF · routes · auth"] end subgraph vpc["Your gVPC network"] ID["③ Instance · Dallas
private address :8080"] IA["③ Instance · Ashburn
private address :8080"] end B --> EW A --> EC M --> EE EW -.->|"④ nearest healthy Instance"| ID EC -->|HTTP · private| ID EE -->|HTTP · private| IA classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 class B,A,M,EW,EC,EE,ID,IA box style west fill:#EDEDEC,stroke:#8A8E99,color:#B5605E style central fill:#EDEDEC,stroke:#8A8E99,color:#B5605E style east fill:#EDEDEC,stroke:#8A8E99,color:#B5605E style vpc fill:#EDEDEC,stroke:#8A8E99,color:#B5605E linkStyle 0,1,2,4,5 stroke:#5A5F6B,stroke-width:1.4px linkStyle 3 stroke:#B5605E,stroke-width:1.6px,stroke-dasharray:6 5 ``` ## How Datum runs Envoy Datum runs Envoy as a shared fleet, deployed in every Datum location: * **One fleet, every customer.** Every location runs a pool of Envoy proxies that serves the ALBs of every customer at once. Datum doesn't run a separate proxy for each ALB, so creating an ALB doesn't wait for new infrastructure to start. * **Close to your users.** ALB hostnames are anycast. A client's request arrives at the nearest Datum location, and Envoy in that location terminates TLS and handles it. To see which location you reach, visit [edge.datum.net](https://edge.datum.net/). * **Scales with the network.** Each new Datum location adds Envoy capacity close to the users it serves, and Datum adds proxies within a location as traffic grows. You don't size, place or scale a load balancer yourself. * **Configured centrally.** You declare an ALB once, in your project. Datum's network services operator turns your resources into Gateway API resources, and Envoy Gateway pushes the resulting configuration to Envoy in every location. ## Into your private network Envoy is how public traffic reaches your [Compute](/docs/compute/overview) Instances, which have private addresses only on your [gVPC](/docs/galactic-vpc/overview) network. For each request, Envoy opens a new connection to a healthy Instance, preferring one in the same location. If the location has no healthy Instance, Envoy sends the request to the nearest location that has one, over Datum's backbone. Datum terminates TLS at Envoy and connects to your Instances over plain HTTP on your private network. For how to publish a workload, see [Compute and ALB](/docs/compute/compute-and-alb). Because the fleet is shared, Envoy must never confuse your network with another customer's, even when both use the same private addresses. Datum handles this below Envoy, in the [Galactic](/docs/platform/galactic) network: * On each Envoy node, a sidecar keeps a Linux VRF, a separate routing table, for each customer network that Envoy connects to. * When Envoy connects to one of your Instances, its connection is bound to your network's VRF. The packet leaves the node wrapped in an [SRv6](/docs/platform/segment-routing) header that identifies your network. * On the Instance's node, Datum unwraps the packet into your network's routing table only, and delivers it. Envoy itself only routes HTTP. Isolation between customers comes from the network, not from rules in each proxy. An ALB can also send traffic to a public origin, such as an existing website, over HTTPS on the internet. The same Envoy fleet serves both kinds of origin. ## Where Envoy fits When you create an ALB, you describe what you want with Datum resources, and Datum's [network services operator](https://github.com/datum-cloud/network-services-operator) turns them into Envoy configuration: * **[HTTPProxy](/docs/alb/httpproxy/overview)** is the main resource you write: hostnames, routes, origins, redirects, rewrites, and header changes. Datum's operator turns it into a Gateway API `Gateway` and `HTTPRoute`, and Envoy Gateway programs Envoy from them. * **[TrafficProtectionPolicy](/docs/alb/configuration/waf)** adds the [Coraza](https://www.coraza.io/) web application firewall with the [OWASP Core Rule Set](https://coreruleset.org/), which runs inside Envoy. It inspects requests before they reach your origin and responses before they leave Datum for the client (outbound traffic). * **Certificates and DNS** for the ALB's hostnames are managed by Datum alongside the Envoy configuration. For the full picture of the resources behind a load balancer, see [What an ALB is made of](/docs/alb/overview#what-an-alb-is-made-of). ## What Envoy gives you Envoy handles the protocols and traffic features of the ALB, including: * HTTP/1.1, HTTP/2, gRPC, and WebSockets. * TLS termination, with certificates that Datum issues and renews. * Path, header, and query matching, with weighted traffic across several origins. * Redirects, URL rewrites, and request and response header changes. * Access logs for every request, including the ones the firewall blocks. See [ALB observability](/docs/alb/observability). ## Using Envoy Gateway resources directly Some advanced features use Envoy Gateway resources (`gateway.envoyproxy.io/v1alpha1`) that you apply to your project with `datumctl`, alongside your `HTTPProxy`: | Resource | Used for | Guide | | - | - | - | | `SecurityPolicy` | Basic authentication and OIDC sign-in | [Basic auth](/docs/alb/authentication/basic-auth), [OIDC with Google](/docs/alb/authentication/oidc-google), [Auth0](/docs/alb/authentication/oidc-auth0) | | `Backend` and `BackendTrafficPolicy` | Failover between a primary and a fallback origin | [Load balancer failover](/docs/alb/configuration/failover) | Because these are standard Envoy Gateway and Gateway API resources, the upstream [Envoy Gateway documentation](https://gateway.envoyproxy.io/docs/) is a useful reference. Not every upstream feature is available on Datum, so start from the Datum guides. ## Related * [ALB overview](/docs/alb/overview) * [HTTPProxy](/docs/alb/httpproxy/overview) * [Platform API](/docs/platform/kubernetes) — why Datum's platform and APIs are built on Kubernetes. * [Galactic](/docs/platform/galactic) — the network underneath gVPC. * [Iroh Relays](/docs/platform/iroh-relays) — how tunnels reach Datum's edge. # Galactic Source: https://datum.net/docs/platform/galactic Galactic is the SRv6 network that gVPC runs on. Learn how it works, how Datum runs it as a service, and how it lets you connect many kinds of things to one private network. Galactic is the networking software underneath [gVPC](/docs/galactic-vpc/overview). It builds private networks on Datum's IPv6 backbone with segment routing ([SRv6](/docs/platform/segment-routing)), and it keeps each network separate from every other one. Galactic is open source under the AGPL-3.0 license, at [github.com/datum-cloud/galactic](https://github.com/datum-cloud/galactic). You don't install or run Galactic to use gVPC. Datum runs it for you. This page explains what happens underneath, so you can understand how your traffic moves and what the network can grow into. ## The Galactic stack Galactic has four layers. Kubernetes resources describe every layer, so the whole network is configured the same way the rest of Datum is. ![Four stacked layers. Attach: pods, VMs, ingress addresses and internet egress. Control: BGP EVPN routes through route reflectors. Datapath: VRF isolation and eBPF programs in each node's kernel. Underlay: the eBGP transit network. A side panel lists the Kubernetes resources that configure them.](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/galactic-stack.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=a76bdf8527ee4d81f3e6f0a0f8d232b6) Galactic's four layers, and the Kubernetes resources that configure them | Layer | What it does | Main components | | - | - | - | | Attach | Connects something to a network: a container, a VM, a public ingress address, or an internet egress point. | `galactic-cni`, `galactic-gateway`, `galactic-nat` | | Control | Tells every node where each address on each network lives. | `galactic-router` with an embedded BGP speaker | | Datapath | Wraps, forwards and unwraps packets in each node's Linux kernel. | eBPF programs, one VRF per network on each node | | Underlay | Carries the wrapped packets between nodes and locations. | `fabric-router` (FRR) running eBGP | ## How a connection works When something attaches to a network, Galactic gives it an address and tells every other node in that network how to reach it. The following figure follows one attachment from start to finish. ![Two compute nodes, Node A and Node B, connected through a route reflector and an SRv6 underlay. Control-plane arrows go from the attach chain to the router to the route reflector to Node B. A dashed data path goes from the pod on Node B through the underlay to the pod on Node A.](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/galactiv-attach-e2e.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=cfd54acc6906fe2c8bb1b9ef05e8dd58) Steps 1 to 5 publish a new attachment's address. Steps 6 to 8 deliver a packet to it from another node. 1. The attach plugins create the node's side of the connection. They place it in the network's own routing table (a VRF) and register it with the node's eBPF datapath. 2. They record the new address as a `BGPAdvertisement` resource. 3. The node's router turns that resource into a BGP EVPN route. The route carries the node's SRv6 address for this network. 4. Route reflectors pass the route to every node. Only nodes that host the same network import it. 5. Those nodes install the route in their copy of the network's routing table. 6. When a workload sends a packet, its node wraps the packet in an outer IPv6 header addressed to the destination node's SRv6 address. 7. The backbone forwards the wrapped packet like any other IPv6 packet. It never sees your network's addresses. 8. The destination node's eBPF program unwraps the packet and hands it to the right network's routing table, which delivers it. The figure shows Galactic in general, so it uses both IPv4 and IPv6 addresses. gVPC networks use IPv6 only. See [Addressing](/docs/galactic-vpc/networks/addressing). ### The address carries the instructions Each SRv6 address that Galactic uses tells the network where the packet goes, what to do when it gets there, and which private network it belongs to. ![A bar divided into fields: uSID Block 48 bits, Node-ID 16 bits, Function 4 bits, Argument 12 bits, Slot 16 bits and Padding 32 bits.](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/sid-anatomy.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=3989f978f669a106cedd93c49e3d8f3a) The 128-bit layout of a Galactic SRv6 address * **Block and Node-ID** identify the destination node. The backbone routes on this 64-bit prefix only. * **Function** says what the node does with the packet, such as unwrapping it into a private network. * **Argument** identifies which private network on that node receives the packet. Because isolation is encoded in the address and enforced in the kernel, two networks can't reach each other even when they share a node. Isolation comes from the topology, not from a shared VLAN with rules on top. ## How Datum runs Galactic as a service gVPC is Galactic run as a managed service. You work with a small set of resources, such as a network and the workloads attached to it. Datum operates everything else. What Datum runs for you: * **The backbone.** Datum runs the underlay routers in every location, the routes between locations, and the configuration of each router. Configuration changes roll out through the API in stages, so a bad change can be stopped before it reaches the whole fleet. * **The routing control plane.** Every node runs a Galactic router. Datum runs the route reflectors that connect them, so a new attachment becomes reachable from every location without anyone touching a router. * **The edges.** Datum runs the edge nodes that bring public traffic into a network, through the [Application Load Balancer](/docs/alb/overview) and anycast ingress addresses. They also take traffic out to the internet. * **Identity and addressing.** Datum allocates network identities and address space centrally, so two networks never collide. This is why you don't plan subnets. * **Visibility.** Datum monitors every BGP session, route and datapath counter. Its operators can ask any router for its view of the network, and can run ping and traceroute from inside the backbone. What you do: * Create a network. * Attach things to it. The rest of this page describes what "attach things to it" can grow to include. ## One network, many attachments > [!NOTE] > > This section describes where gVPC is headed. Only the attachments marked **Available** below work today. Plans can change, and this isn't a commitment to deliver any feature or to meet any timeline. In Galactic, everything that joins a network does it the same way. It gets a place in the network's routing table and an address, and it announces that address with a `BGPAdvertisement`. A container, a virtual machine, a public ingress address and an internet egress point already join this way. The rest of the network doesn't need to know what kind of thing an attachment is. So adding a new kind of connection doesn't need a new network. It needs a new way to attach. That makes a gVPC network work like a [virtual meet-me room](https://www.datum.net/blog/galactic-vpc-backbone/): a single private place where all your connections meet, whatever they are and wherever they come from. | You want to connect | How it attaches | Status | | - | - | - | | Datum Compute Instances | Each Instance gets a private address on the network | Available | | Public HTTP and HTTPS traffic | Through the [Application Load Balancer](/docs/alb/overview) | Available | | The internet, outbound | Through egress points on Datum's edge, IPv6 and IPv4 | Available | | Laptops, desktop apps and containers anywhere | Through a [Connector](/docs/connectors/overview) tunnel | Planned | | Your own servers and VPNs | Through a WireGuard peer or a VPN | Exploring | | Your data center or another cloud's VPC | Through a BGP session, a VLAN, or a carrier interconnect | Exploring | ## Words we use for this These terms describe gVPC in plain language. Each one corresponds to something the platform does. | We say | It means | | - | - | | **One network, anywhere** | A network belongs to your project, not to a location. It spans every location where something is attached to it. | | **Attach it** | Anything that joins a network, whether an Instance, a tunnel or a data center, joins the same way and gets an address on it. | | **Private by default** | Nothing on a network is reachable from the internet unless you publish it, for example through an ALB. | | **Isolation is the topology** | Networks are separated by the routing itself, in every node's kernel, not by rules that someone has to keep correct. | | **A network per anything** | Networks cost nothing to plan, so you can create one per customer, per environment, per agent or per workload. | | **Datum runs the backbone** | You don't manage routers, routes, tunnels or addresses. You declare a network, and Datum keeps it working. | ## What's next > [card] [gVPC](/docs/galactic-vpc/overview) > > Create a network and attach workloads to it. > [card] [Traffic and connecting](/docs/galactic-vpc/networks/traffic) > > Learn which traffic a network carries today. > [card] [Networking roadmap](/docs/galactic-vpc/limitations-and-roadmap) > > Review what gVPC can't do yet and what's planned. > [card] [Galactic on GitHub](https://github.com/datum-cloud/galactic) > > Read the source code and architecture documents. # Iroh Relays Source: https://datum.net/docs/platform/iroh-relays Datum runs Iroh relays in its locations. Learn what the relays do, where they run, and how Datum tunnels use them to connect over QUIC. Datum runs relays for [Iroh](https://www.iroh.computer/), the QUIC-based peer-to-peer networking stack behind Datum [Tunnels](/docs/connectors/tunnels). A relay helps two endpoints find each other and connect, even when both sit behind NAT or a firewall, and carries their traffic until they can talk directly. ![Three endpoints across the top: your workstation running a datum tunnel, your alt cloud running an iroh tunnel, and an AWS Lambda function with an iroh endpoint. Each connects over HTTPS to the Iroh relay in its nearest Datum location: us-west-1 in San Jose, us-central-1 in Dallas, and us-east-1 in Ashburn. A dashed path shows the workstation reaching the alt cloud through the Dallas relay, and a second dashed path shows the direct QUIC connection they switch to after holepunching.](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/iroh-relays.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=3e68e22e591bb853cf7e26422e4ac475) A workstation, an alt cloud and an AWS Lambda function each connect out to the relay in their nearest Datum location. The workstation reaches the alt cloud through the alt cloud's relay first, then directly. ## What a relay does Every Iroh endpoint has a key pair, and its public key is its address. Endpoints dial each other by key, not by IP address, so a connection keeps working when a laptop changes networks or a server moves. A relay does three things for the endpoints that use it: * **Gives every endpoint a home.** Each endpoint keeps one outbound connection to its nearest relay, its *home relay*, over HTTPS on port 443. Because the connection is outbound, the endpoint needs no public address and no open inbound port. * **Makes the first connection work at once.** When one endpoint dials another, the first packets go through the other endpoint's home relay. The connection works right away, whatever NAT or firewall is in the way. * **Helps the endpoints go direct.** While traffic flows through the relay, both sides try to holepunch a direct UDP path. When that works, traffic moves to the direct path, and the relay stays available as a fallback. Relays never see your data. The connection between two endpoints is QUIC, encrypted end to end with TLS 1.3, so a relay forwards only encrypted packets that it can't read. ## Why QUIC QUIC is the transport under HTTP/3. Iroh uses it for every connection between endpoints because it gives tunnels: * **Encryption by default.** TLS 1.3 is part of the protocol, not a layer on top. * **Many streams on one connection.** Requests share a connection without one slow request blocking the others. * **Connections that survive network changes.** A connection can move between paths, such as from relayed to direct, or from Wi-Fi to cellular, without starting over. * **UDP underneath.** UDP is what makes holepunching through NAT possible. ## Where relays run Datum runs an Iroh relay in each of these locations: | Location | City | Relay | | - | - | - | | `us-west-1` | San Jose (SJC) | `iroh-relay.us-west-1.datumconnect.net` | | `us-central-1` | Dallas (DFW) | `iroh-relay.us-central-1.datumconnect.net` | | `us-east-1` | Ashburn (IAD) | `iroh-relay.us-east-1.datumconnect.net` | More locations will run relays as Datum grows. For every Datum location, see [Locations](/docs/locations). To check that a relay is up from your network, request its health endpoint: ```bash theme={null} curl https://iroh-relay.us-west-1.datumconnect.net/healthz ``` A healthy relay returns `{"status":"ok", ...}` with its version. ## How tunnels use the relays When you start a tunnel, the tunnel agent picks its relays for you. It starts from Datum's list of relays, measures the latency to each one, and keeps the closest. You don't configure a relay to use a tunnel. Tunnels are managed with the `connect` plugin, which is in preview. It isn't in the official catalog yet, so install it from GitHub: ```bash theme={null} datumctl plugin install datum-cloud/connect ``` The plugin runs a small daemon on your machine that holds the device's Iroh endpoint and its connection to the relays. Everything is private by default: a service you share is reachable only by other Connectors in your project, unless you choose otherwise. | To | Run | | - | - | | Share a service on your machine with your project | `datumctl connect serve localhost:8080` | | Share it publicly, through Datum's edge | `datumctl connect serve localhost:8080 --public` | | Share it with one specific Connector | `datumctl connect serve localhost:22 --allow TEAMMATE_CONNECTOR` | | Reach a service that another Connector shares | `datumctl connect dial SERVER_CONNECTOR:22 --bind 2222` | | Join a [gVPC](/docs/galactic-vpc/overview) network | `datumctl connect join NETWORK_NAME` | | See your Connector, services, and forwards | `datumctl connect status` | | Check the daemon and networking without changing anything | `datumctl connect doctor` | The first `serve` walks you through setup: it starts the daemon, uses your `datumctl login`, and enrolls the device in your current project. To enroll without sharing anything, or to reconnect later, run `datumctl connect up`. On a server without a browser, pass a service account's credentials with `datumctl connect up --credentials-file PATH`. Connectors find each other by name or public key, never by IP address. `dial` opens a port on your own machine, so in the example above, `ssh -p 2222 localhost` reaches the other Connector's SSH server through the tunnel, directly or through a relay. Public traffic reaches a `--public` service through Datum's edge. A request arrives at an [Application Load Balancer](/docs/alb/overview), and Datum's Iroh gateway forwards it to your Connector by its endpoint ID. Your service answers on `localhost`, and nothing on your machine is exposed to the internet. The [Desktop Apps](/docs/desktop-apps) use the same tunnels. > [!NOTE] > > To pin a tunnel to specific relays, for example while you troubleshoot, set `DATUM_CONNECT_RELAY_URLS` to a comma-separated list of relay URLs before you start the daemon. ## Where this is going Iroh connects any two endpoints, wherever they run. With the `connect` plugin, a workstation can already reach a service on another Connector by key, and a device can join a gVPC network. Datum's relays are built to grow that further: servers in an [alt cloud](https://www.alt-cloud.org/) and short-lived functions, such as on AWS Lambda, reaching your network and each other with no public addresses or open ports. > [!NOTE] > > This section describes where Datum is headed. Plans can change, and this isn't a commitment to deliver any feature or to meet any timeline. ## What's next > [card] [Tunnels](/docs/connectors/tunnels) > > Datum's Iroh-powered Connector. > [card] [Desktop Apps](/docs/desktop-apps) > > Expose a local environment to the internet over a tunnel. > [card] [Locations](/docs/locations) > > Every Datum location and region. > [card] [Platform roadmap](/docs/platform/roadmap) > > See what Datum is building now and what comes next. # Isolation Source: https://datum.net/docs/platform/isolation How Datum isolates and operates every Instance: its own virtual machine, its own gVPC network, secure defaults, and the operating model behind gVPC-connected Compute. Isolation is the idea Datum builds everything else on: each customer's work is kept apart from every other customer's, at the virtual machine, the network, and the control plane, and Datum operates that separation for you. Compute is where you see it most. Every Compute Instance runs in its own virtual machine on your own [gVPC](/docs/galactic-vpc/overview) network, and the platform places, starts, and watches it in every location. This page explains the isolation and operating ideas underneath [Compute](/docs/compute/overview): how a workload becomes running Instances, how each Instance is kept apart from every other customer's, and how that stays efficient. For how to use Compute, see the [Compute documentation](/docs/compute/overview). ## From workload to Instances You describe what to run in a *workload*: a container image, a runtime class, an instance type, and *placements* that say which locations to run in and how many Instances to keep in each. Datum turns it into running Instances in every location you named: ```mermaid theme={null} flowchart TB W["① Workload
image · runtime class · instance type
placements in one or more locations"] subgraph east["us-east-1"] DE["② WorkloadDeployment
how many Instances to run here"] GE["③ Gates
quota granted · network ready"] IE["④ Instances
started by the runtime provider"] end subgraph central["us-central-1"] DC["② WorkloadDeployment"] GC["③ Gates"] IC["④ Instances"] end subgraph west["us-west-1"] DW["② WorkloadDeployment"] GW["③ Gates"] IW["④ Instances"] end S["⑤ Status flows back
datumctl compute workloads · Cloud Portal"] W --> DE --> GE --> IE W --> DC --> GC --> IC W --> DW --> GW --> IW IE -.-> S IC -.-> S IW -.-> S classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 class W,DE,GE,IE,DC,GC,IC,DW,GW,IW,S box style east fill:#EDEDEC,stroke:#8A8E99,color:#B5605E style central fill:#EDEDEC,stroke:#8A8E99,color:#B5605E style west fill:#EDEDEC,stroke:#8A8E99,color:#B5605E linkStyle 0,1,2,3,4,5,6,7,8 stroke:#5A5F6B,stroke-width:1.4px linkStyle 9,10,11 stroke:#B5605E,stroke-width:1.4px,stroke-dasharray:5 4 ``` 1. **You declare a workload** in your project's control plane, with `datumctl compute deploy`, a manifest, or the [Cloud Portal](https://cloud.datum.net). 2. **Datum splits it by location.** Each placement becomes a *WorkloadDeployment* in each of its locations, which tracks how many Instances that location should run. 3. **Each Instance waits at its gates.** An Instance doesn't start until your project's quota grants its resources and its private network is ready. An Instance that would go over quota waits, and reports why, rather than starting. 4. **A runtime provider starts the Instance.** Each location runs a provider for each runtime class. The provider starts the Instance in its own virtual machine and attaches it to your [gVPC](/docs/galactic-vpc/overview) network. 5. **Status flows back.** Each location reports its Instances' health to your project. To see it, run `datumctl compute workloads describe WORKLOAD_NAME`. Each location runs its own Instances, so a problem in one location doesn't stop Instances in the others. ## Isolation at the virtual machine Many container platforms run every customer's containers on one shared Linux kernel, separated by kernel features. A single kernel bug can then cross from one customer to another. Datum doesn't do that. Every Instance, in every runtime class, gets its own virtual machine and its own kernel: ```mermaid theme={null} flowchart TB LB["Application Load Balancer
the only public way in"] N1["Your gVPC network
private IPv6 only"] N2["Their network"] C1["Your container
hardened defaults: few capabilities,
no privilege escalation, seccomp"] C2["Another customer's container"] V1["Your Instance's own virtual machine
and its own kernel"] V2["Their Instance's own virtual machine
and its own kernel"] HV["Hypervisor on a shared Datum host
no kernel is shared between customers"] LB -->|"HTTP · private"| N1 N1 <-->|"your network only"| V1 N2 <-.->|"their network only"| V2 C1 -->|"runs inside"| V1 C2 -.->|"runs inside"| V2 V1 --> HV V2 -.-> HV classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 classDef mine fill:#E6E6E6,stroke:#B5605E,stroke-width:1.6px,color:#222 classDef other fill:#F6F6F4,stroke:#8A8E99,color:#5f6168 class LB,HV box class N1,C1,V1 mine class N2,C2,V2 other linkStyle 0,1,3,5 stroke:#B5605E,stroke-width:1.6px linkStyle 2,4,6 stroke:#8A8E99,stroke-width:1.2px,stroke-dasharray:5 4 ``` The only thing two customers' Instances share is the hypervisor underneath. Inside its virtual machine, an Instance can do things that would be risky on a shared kernel, such as running as root or adding Linux capabilities, because they only affect its own virtual machine. ## Runtime classes A *runtime class* is a way of running an Instance. Each class is a published contract about its isolation, which images it runs, and how fast its Instances start. Datum offers two: * **General purpose** runs any standard Linux container image, unchanged, inside a lightweight virtual machine built with [Kata Containers](https://katacontainers.io). Datum's [kata-provider](https://github.com/datum-labs/kata-provider) uses Cloud Hypervisor, which reserves far less memory for each virtual machine than QEMU, so more Instances fit on each host. * **Unikernel** runs your app as a [unikernel](/docs/compute/runtimes/unikernels): a single-purpose virtual machine that contains your app and only the operating system code it uses, built on [Unikraft](https://unikraft.org/). Each class has its own provider, released and operated separately, so a fault in one class can't take down the other. To read the contracts, run `datumctl get runtimeclasses`, or `datumctl get runtimeclass CLASS_NAME -o yaml` for the full detail. To compare the classes, see [Choose a runtime class](/docs/compute/runtimes/overview). ## Secure by default Isolation is the foundation, and gVPC carries it to the network. On top of both, every Instance starts with defaults you don't have to remember to set: * **Few privileges.** A `general-purpose` container starts with a short list of Linux capabilities, enough for common images such as nginx or Postgres to start, instead of the full set. It can't gain new privileges after it starts, and the runtime's default seccomp profile filters its system calls. A container that needs more asks for it explicitly in its workload, where you can see and review it. * **No public address.** An Instance has a private IPv6 address on your [gVPC](/docs/galactic-vpc/overview) network and nothing else. Public traffic reaches it only through an [Application Load Balancer](/docs/alb/overview), with TLS and a [web application firewall](/docs/alb/configuration/waf) in front. * **Only your network.** Your Instances can reach each other across locations, and never another customer's network, even when both use the same private addresses. Isolation comes from the routing itself. See [Segment routing](/docs/platform/segment-routing). * **Secrets stay secret.** Workloads read credentials from Secrets. Describing a workload shows where a value comes from, never the value itself. * **Actions are recorded.** Changes to workloads, and every [shell session](/docs/compute/instance-shell) into an Instance, are recorded in your project's [activity logs](/docs/operations/activity-logs). Shell sessions are encrypted all the way to the location that runs the Instance. * **Quota is enforced before anything runs.** Every Instance claims its resources from your project's quota before it starts, so one runaway workload can't consume more than your project allows. ## Efficient without trading away isolation Virtual machines used to mean slow starts and wasted memory. Compute avoids both: * **Millisecond starts.** Once its image is on the host, a unikernel Instance boots in tens of milliseconds. A unikernel carries only the code your app uses, so there's very little to boot and very little memory to hold. * **Idle Instances give memory back.** The platform parks an idle unikernel Instance into a snapshot and restores it on its next request. The Instance keeps its state without holding the host's memory while it waits. * **Dense, lightweight virtual machines.** The `general-purpose` class uses a lightweight hypervisor, so each virtual machine costs little memory beyond what your app uses. * **Right-sized Instances.** Every Instance has a named *instance type* from one catalog, so what you ask for, what your quota reserves, and what the Instance gets always match. To see the sizes available to your project, run `datumctl get instancetypes`. * **Scale where it's needed.** Each placement scales on its own, in its own location. Add Instances in the locations that are busy without growing the ones that aren't. See [Placement and scaling](/docs/compute/placement-and-scaling). * **The right tier for each job.** Run bursty, short-lived services on the unikernel class, and long-running services that need full Linux on `general-purpose`, in the same project and on the same network. ## How isolation and operations build gVPC-connected Compute Compute is these ideas put together. Isolation keeps each Instance and each network apart, and Datum's operating model places Instances, gates them on quota and network readiness, and reports their status back, in every location. The result is Compute whose Instances are connected to each other through your [gVPC](/docs/galactic-vpc/overview) and to no one else: * **[Locations](/docs/locations):** Instances run in Datum locations that serve Compute. * **[gVPC](/docs/galactic-vpc/overview):** every Instance gets a private IPv6 address on a network that spans locations. Datum doesn't start an Instance until its network is ready, and the network keeps it apart from every other customer's. * **[Application Load Balancer](/docs/alb/overview):** public HTTP and HTTPS traffic reaches Instances through an ALB. * **[Platform API](/docs/platform/kubernetes):** workloads are declarative resources that you manage with `datumctl`, manifests, or GitOps tooling. > [card] [Compute documentation](/docs/compute/overview) > > Runtimes, placement and scaling, configuration, operations, and more. # Platform API Source: https://datum.net/docs/platform/kubernetes Datum's platform and its API are built on Kubernetes, and every resource is a native Kubernetes construct. Learn how datumctl, kubectl and your other tools talk to it, and how controllers turn your resources into DNS, Envoy, networks and Compute in every location. Datum's platform and its API are built on Kubernetes, and the resources you work with are native Kubernetes constructs. Not Kubernetes as a container scheduler, but the part that makes Kubernetes powerful: an API server where everything is a typed resource, and controllers that keep reality matching what those resources declare. You don't need to know Kubernetes to use Datum, but it helps to understand what's under the hood. Everything you create on Datum, from a DNS zone to a load balancer to a Compute workload, is a Kubernetes-style resource served by the Platform API. Controllers pick each one up and make it real in Datum's locations. ```mermaid theme={null} %%{init: {"flowchart": {"subGraphTitleMargin": {"top": 6, "bottom": 14}, "wrappingWidth": 240}}}%% flowchart TB subgraph you["Your tools and agents"] CLI["datumctl"] KC["kubectl · Helm · k9s
credentials from datumctl"] GO["GitOps
Flux · Argo CD"] AG["AI agents
MCP"] UI["Cloud Portal"] end subgraph milo["api.datum.net · Milo"] API["① Kubernetes-style API server
discovery · OpenAPI · RBAC
admission policy · audit"] ORG["Organization control plane
Organizations · Projects · quotas"] PCP["② Project control plane
HTTPProxy · Workload · Network
DNSZone · Secrets"] end subgraph ctrl["③ Controllers"] NSO["network-services-operator
Networks · HTTPProxy"] CMP["Compute controllers
Workloads · Instances"] DNC["DNS controllers
zones · records"] end subgraph cells["④ Cells in every Datum location"] ENV["Envoy Gateway → Envoy
ALB"] GAL["Galactic
gVPC networks"] KATA["kata-provider → Kata VMs
general-purpose Instances"] UNI["unikernel provider
unikernel Instances"] DNS["Authoritative DNS
anycast"] end CLI --> API KC --> API GO --> API AG --> API UI --> API API --> ORG API --> PCP PCP -->|watch| NSO PCP -->|watch| CMP PCP -->|watch| DNC NSO --> ENV NSO --> GAL CMP --> KATA CMP --> UNI DNC --> DNS cells -.->|"⑤ status"| PCP classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 class CLI,KC,GO,AG,UI,API,ORG,PCP,NSO,CMP,DNC,ENV,GAL,KATA,UNI,DNS box style you fill:#EDEDEC,stroke:#8A8E99,color:#B5605E style milo fill:#EDEDEC,stroke:#8A8E99,color:#B5605E style ctrl fill:#EDEDEC,stroke:#8A8E99,color:#B5605E style cells fill:#EDEDEC,stroke:#8A8E99,color:#B5605E linkStyle 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14 stroke:#5A5F6B,stroke-width:1.4px linkStyle 15 stroke:#B5605E,stroke-width:1.6px,stroke-dasharray:6 5 ``` 1. **One API.** Every tool talks to the same Kubernetes API server at `api.datum.net`. It handles discovery, schemas, authentication, authorization, admission policy and audit logging for every request. 2. **Your resources live in a project.** Organizations and projects each get their own control plane. Your `HTTPProxy`, `Workload`, `Network` and `DNSZone` resources live in your project's control plane, isolated from every other project. 3. **Controllers watch your resources.** Each product has controllers that watch for changes to its resources, such as the [network services operator](https://github.com/datum-cloud/network-services-operator) for networks and load balancers. 4. **Controllers make it real in every location.** Controllers reconcile your resources into the clusters that run Datum's data plane in each location. That's where Envoy, Galactic, DNS servers and your Compute Instances actually run. 5. **Status flows back.** What happened in each location is written back to your resource's status, which is what `datumctl get` and the [Cloud Portal](https://cloud.datum.net) show you. ## Datum's control plane: Milo Datum's control plane is [Milo](https://github.com/datum-cloud/milo), an open source, multi-tenant control plane for service providers. Milo uses the Kubernetes API server as a generic control plane, without the container-scheduling parts it doesn't need. It keeps what matters for a platform: * **API discovery and schemas,** through the Discovery API and OpenAPI v3, so clients learn what exists and how to use it. * **Declarative validation and policy,** with admission policies written in CEL. * **Authentication and authorization,** with OIDC and role-based access control. * **Audit logs** of who did what and when. See [Activity logs](/docs/operations/activity-logs). * **Event-driven reconciliation,** through watches and controllers, instead of polling and scripts. * **Extensibility,** with custom resource definitions (CRDs) for most types and aggregated API servers where a type needs custom behavior. Organizations, projects and quotas are resources too. Your organization has its own control plane for organization-wide resources, and each project has a control plane for the resources you build with. ## How your tools reach it Because the API follows Kubernetes conventions, many tools can use it, and they all see the same resources: * **[datumctl](/docs/datumctl/overview)** is Datum's CLI. Commands such as `datumctl get`, `apply`, `diff`, `explain` and `api-resources` are Kubernetes verbs, sent to your organization's or project's control plane. * **[kubectl, Helm and k9s](/docs/datumctl/kubectl-interop)** work against the same control planes. `datumctl auth update-kubeconfig` writes a kubeconfig entry that gets its credentials from your `datumctl` session. * **[GitOps tools](/docs/datumctl/gitops)** such as Flux and Argo CD apply your resources from Git, the same way they would to a Kubernetes cluster. * **AI agents** use the [Datum MCP server](/docs/agents/datum-mcp) or `datumctl`. Well-known Kubernetes patterns make it easier for models to work with Datum accurately. * **The Cloud Portal** reads and writes the same resources. Under the hood, a project's resources live under a path like the following, which [`datumctl api-proxy`](/docs/datumctl/api-proxy) exposes on your machine: ```text theme={null} https://api.datum.net/apis/resourcemanager.miloapis.com/v1alpha1/projects/your-project-id/control-plane/apis/networking.datumapis.com/v1alpha/... ``` To see every resource type your project offers, run `datumctl api-resources`. To read the schema of one, run `datumctl explain`. For more information, see [Discovering resources](/docs/datumctl/discovering-resources). ## Controllers turn resources into infrastructure Each product is a set of resources plus the controllers that reconcile them. Controllers watch your project's control plane, and act in the clusters that run each Datum location: | You create | Controllers | What runs in each location | | - | - | - | | `HTTPProxy`, `TrafficProtectionPolicy` | [network services operator](https://github.com/datum-cloud/network-services-operator) and [Envoy Gateway](https://gateway.envoyproxy.io/) | [Envoy](/docs/platform/envoy), as your [ALB](/docs/alb/overview) | | `Network` | network services operator and [Galactic](/docs/platform/galactic) | Your [gVPC](/docs/galactic-vpc/overview) network | | `Workload` with the `general-purpose` runtime class | Compute controllers and [kata-provider](https://github.com/datum-labs/kata-provider) | Each Instance as a container in its own virtual machine, with [Kata Containers](https://katacontainers.io) | | `Workload` with the `unikernel` runtime class | Compute controllers and the [unikernel](/docs/compute/runtimes/unikernels) provider | Each Instance as a unikernel | | `DNSZone`, `DNSRecordSet` | DNS controllers | Records on Datum's anycast [authoritative DNS](/docs/domain-dns/dns) | Each runtime class has its own provider, released and operated separately, so a fault in one class can't take down the other. For example, kata-provider only acts on Instances of the `general-purpose` class. It runs each one as a Kubernetes Pod in a Kata virtual machine, so every Instance gets its own kernel. See [Runtimes](/docs/compute/runtimes/overview). ## Why Kubernetes-native We took a bet on building the platform and its API on Kubernetes for the following main reasons. ### Ecosystem It's aligned with the largest ecosystem for cloud operators. If you're a cloud native SRE or platform engineer, working with Datum capabilities exposed as resources should feel familiar, and the tools you already use, such as kubectl, Helm, k9s, Flux and Argo CD, already speak it. ### LLMs Because Kubernetes patterns are so well established, LLMs work with Datum more accurately. ### Semi-free lunch By modeling our platform on Kubernetes-style APIs, we get a lot of battle-hardened machinery at modest cost: discovery, validation, access control, audit, watches and garbage collection. It isn't completely free. A provider-grade control plane still has to handle multi-tenancy, storage limits and search, and Datum builds dedicated services where the core API shouldn't stretch. > [!NOTE] > > For more information, see [Building a distributed control plane for modern service providers](https://www.datum.net/blog/control-plane-for-modern-service-providers/) by Scot Schuchert-Wells, and [From cage nuts to Kubernetes](https://www.datum.net/blog/from-cage-nuts-to-kubernetes/) by Tom Daly. # Roadmap Source: https://datum.net/docs/platform/roadmap What Datum offers today, what is in preview, and where to follow and influence what comes next. Datum is still being built. We are building a public cloud that is free during public beta, with usage-based [pricing](https://www.datum.net/pricing), and working closely with design partners on managed cloud and BYOC approaches. This page summarizes where each product stands. ## Available today Our public cloud is opinionated, but powered by key partners. We call these our "house brand" services: * **[Application Load Balancer](/docs/alb/overview):** an [Envoy](/docs/platform/envoy) edge proxy and AI gateway, with a built-in [web application firewall](/docs/alb/configuration/waf). * **[Connectors](/docs/connectors/overview):** a way to orchestrate connections, starting with QUIC tunnels powered by [Iroh](/docs/platform/iroh-relays). * **[Domains and DNS](/docs/domain-dns/overview):** domain tracking and verification, plus global authoritative DNS. * **Essentials:** [metrics export](/docs/operations/metrics-export), [activity logs](/docs/operations/activity-logs), [secrets](/docs/operations/secrets), and team collaboration. ## In preview * **[Compute](/docs/compute/overview):** isolated Instances that run container images or fast-starting [unikernels](/docs/compute/runtimes/unikernels) in Datum locations. * **[gVPC](/docs/galactic-vpc/overview):** private IPv6 networks for your Instances that span every location they run in, on Datum's [SRv6](/docs/platform/segment-routing) backbone. ## What we're working on * **More Connectors:** developer options such as Tailscale tailnets and WireGuard VPNs, and telco options such as AWS Direct Connect, Equinix Fabric, and Megaport onramps. * **vMMR:** a virtual "meet me room" for interacting with diverse connections. * **Networking for Instances:** firewall rules and private DNS. See [gVPC planned capabilities](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). ## Product roadmaps Each product has its own limitations and roadmap page: * [gVPC](/docs/galactic-vpc/limitations-and-roadmap) * [Compute](/docs/compute/limitations-and-roadmap) * [Application Load Balancer](/docs/alb/limitations-and-roadmap) * [Connectors](/docs/connectors/limitations-and-roadmap) * [Domains and DNS](/docs/domain-dns/limitations-and-roadmap) ## Follow along and get involved To see what has already shipped, read the [changelog](/docs/changelog). The best place to track our roadmap is our [website and milestones](https://www.datum.net/roadmap/). To suggest a feature, visit our [GitHub discussions](https://github.com/orgs/datum-cloud/discussions/categories/feature-requests). For more ways to take part, see [Get involved](/docs/get-involved). ### Enhancement proposals Datum designs major changes in the open, in the [datum-cloud/enhancements](https://github.com/datum-cloud/enhancements) repository on GitHub. It holds proposals for significant architectural changes, cross-cutting work such as security and performance, and long-term roadmap items that span several releases. Each proposal starts as an issue, is refined with maintainers and the community, and becomes an enhancement document. Proposals move from `Backlog` to `In Progress` to `Completed`. To follow the design behind what's coming, browse the [open enhancements](https://github.com/datum-cloud/enhancements/issues). To propose one of your own, open a new issue there. For smaller ideas and feature requests, use [GitHub discussions](https://github.com/orgs/datum-cloud/discussions/categories/feature-requests) instead. > [!NOTE] > > Plans can change. This page isn't a commitment to deliver any feature or to meet any timeline. # Segment Routing Source: https://datum.net/docs/platform/segment-routing How segment routing works, starting from one packet and building up to how Datum uses SRv6 to run private networks, load balancers and connections across every location. Datum's backbone uses IPv6 segment routing (SRv6) to carry traffic between locations. Segment routing is usually something only network engineers talk about, but the idea behind it is simple, and once it clicks, a lot of what Datum does makes sense. This page starts with one packet and builds up from there. You don't need to know any of this to use Datum. It explains what happens underneath [gVPC](/docs/galactic-vpc/overview), the [Application Load Balancer](/docs/alb/overview), and the rest of the platform. ## Start simple: the packet carries its route In most networks, every router makes its own decision. A packet arrives, the router looks up the destination address, and it sends the packet one hop closer. The packet has no say in how it gets there. To make some traffic take a different path, or to keep one customer's traffic apart from another's, every router along the way has to be told about it and has to remember it. Segment routing flips this around. The sender writes the route into the packet, as an ordered list of instructions called *segments*. Each router reads the current instruction, called the *active segment*, carries it out, and moves on to the next one. The routers in the middle don't need to know anything about the traffic. They just follow the instructions. ```mermaid theme={null} flowchart LR S["Sender
writes the route:
① go to B
② go to D"] A(("A")) B(("B")) C(("C")) D(("D")) R["Receiver"] S --> A A ==>|"① go to B"| B B ==>|"② go to D"| D A -.-|"shortest path, not used"| C C -.- D D --> R classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 classDef node fill:#EDEDEC,stroke:#5A5F6B,color:#222 class S,R box class A,B,C,D node linkStyle 0,5 stroke:#5A5F6B,stroke-width:1.4px linkStyle 1,2 stroke:#B5605E,stroke-width:2.2px linkStyle 3,4 stroke:#8A8E99,stroke-width:1px,stroke-dasharray:3 4 ``` In this example, the shortest path from A to D goes through C. But the sender wants the packet to go through B, so it writes two segments: "go to B", then "go to D". A sends the packet toward B. When it arrives, B sees that its segment is done and moves to the next one, "go to D". None of the routers had to be configured for this one packet. A segment can say several kinds of things: * **Go to a node.** Reach this router by the shortest path. This is a *node segment*. * **Use a link.** Leave through this specific link, even if it isn't the shortest way. This is an *adjacency segment*. * **Do something.** When you get here, carry out a function, such as delivering the packet into a particular private network. Because the route lives in the packet, the network in the middle stays simple. It doesn't hold state for each customer, each connection, or each special path. That's what lets segment routing scale. ## SRv6: segments are IPv6 addresses There are two ways to write segments into a packet. The older way uses MPLS labels. SRv6, which Datum uses, writes each segment as an IPv6 address. That has a big advantage: any router that can forward IPv6 can carry SRv6 traffic, because to most of the network an SRv6 packet is just an IPv6 packet. An SRv6 address isn't only a destination. Its bits are split into parts that tell the network where the packet goes and what to do when it gets there: ![A bar divided into fields: uSID Block 48 bits, Node-ID 16 bits, Function 4 bits, Argument 12 bits, Slot 16 bits and Padding 32 bits.](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/sid-anatomy.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=3989f978f669a106cedd93c49e3d8f3a) The 128-bit layout of the SRv6 addresses Datum uses * **Block and Node-ID** say which node the packet goes to. Datum's backbone routes on this 64-bit prefix only. * **Function** says what that node does with the packet, such as unwrap it and deliver it into a private network. * **Argument** says which private network on that node receives it. Datum uses a compact form of SRv6 called *micro-segments* (uSID), which packs instructions tightly into each address. ## Expand the scope: private networks on a shared backbone Now put this to work. Two customers each run an Instance in Dallas and another in Ashburn, on their own [gVPC](/docs/galactic-vpc/overview) networks. They happen to use exactly the same private addresses. How does Datum keep their traffic apart on shared infrastructure? ```mermaid theme={null} flowchart TB Y1["Your Instance in Dallas
fd20:20:3::5"] O1["Another customer's Instance in Dallas
also fd20:20:3::5"] W["Dallas node wraps each packet in an outer IPv6 header
to: Ashburn node · unwrap · which network"] BB["Datum backbone
forwards on the node part only · knows no customers"] U["Ashburn node unwraps the packet
into the network named in the address"] Y2["Your Instance in Ashburn"] O2["Their Instance in Ashburn"] Y1 -->|"your network"| W O1 -.->|"their network"| W W --> BB BB --> U U -->|"your network"| Y2 U -.->|"their network"| O2 classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 classDef other fill:#F6F6F4,stroke:#8A8E99,color:#5f6168 class Y1,W,BB,U,Y2 box class O1,O2 other linkStyle 0,2,3,4 stroke:#B5605E,stroke-width:1.8px linkStyle 1,5 stroke:#8A8E99,stroke-width:1.2px,stroke-dasharray:5 4 ``` 1. When your Instance in Dallas sends a packet to your Instance in Ashburn, the Dallas node wraps it in an outer IPv6 header. The outer destination is an SRv6 address that means "the Ashburn node, unwrap this, into *your* network". 2. Datum's backbone forwards the packet on the node part of that address, like any other IPv6 packet. The routers in between never see your private addresses and don't know you exist. 3. The Ashburn node reads the function and argument, unwraps the packet, and delivers it inside your network only. The other customer's packets travel the same way, with a different network in the address. Even though both customers use the same private addresses, and even though their traffic shares the same nodes and links, a packet can only ever be delivered into the network its address names. Isolation comes from the routing itself, enforced in each node's kernel, not from firewall rules that someone has to keep correct. This is also why networks on Datum cost nothing to plan. Adding a network doesn't add state to the backbone. It's one more value that an address can carry. For more on how Galactic builds these networks, see [Galactic](/docs/platform/galactic). ## Expand again: one backbone for the whole platform The same mechanism carries more than traffic between Instances. Anything that needs to reach your private network uses it: ```mermaid theme={null} flowchart TB NET["Clients on the internet"] ENV["Envoy · ALB
Datum edge, every location"] CO["Connectors
your sites and clouds · planned"] BB["SRv6 backbone
plain IPv6 between every Datum location"] CI["Your Compute Instances
on your gVPC network, every location"] EG["Internet egress"] NET --> ENV ENV -->|"SRv6 into your network"| BB CO -.->|"SRv6 into your network"| BB BB <-->|"SRv6 between locations"| CI CI -->|"SRv6 to the edge"| EG classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 classDef planned fill:#F6F6F4,stroke:#8A8E99,stroke-dasharray:3 4,color:#5f6168 class NET,ENV,BB,CI,EG box class CO planned linkStyle 0,1,3,4 stroke:#5A5F6B,stroke-width:1.4px linkStyle 2 stroke:#8A8E99,stroke-width:1.2px,stroke-dasharray:5 4 ``` * **Load balancers.** Envoy, the proxy behind every [ALB](/docs/alb/overview), reaches your Instances by wrapping its connection in an SRv6 address for your network. One shared fleet of proxies serves every customer without ever reaching the wrong network. See [Envoy](/docs/platform/envoy). * **Compute Instances.** Instances reach each other across locations over the same backbone, with no VPNs or peering to set up. See [gVPC traffic](/docs/galactic-vpc/networks/traffic). * **[Connectors](/docs/connectors/overview) and internet egress.** Outbound traffic to the internet already joins your network this way. Connections from your own sites and clouds are planned to follow. See [gVPC planned capabilities](/docs/galactic-vpc/limitations-and-roadmap#planned-capabilities). Because every product speaks SRv6, adding a new way to connect to your network doesn't need a new network. It needs a new place that can write the right address. ## Where it's going: choosing the path So far, packets have followed the shortest path between nodes. But the shortest path isn't always the fastest. Segment routing can steer traffic along a different path by adding segments, just like the first example on this page. ```mermaid theme={null} flowchart LR DAL["Dallas"] CHI["Chicago"] ASH["Ashburn"] DAL -->|"shortest path"| ASH DAL ==>|"① go to Chicago"| CHI CHI ==>|"② go to Ashburn"| ASH classDef box fill:#E6E6E6,stroke:#8A8E99,color:#222 class DAL,CHI,ASH box linkStyle 0 stroke:#8A8E99,stroke-width:1.2px,stroke-dasharray:5 4 linkStyle 1,2 stroke:#B5605E,stroke-width:2.2px ``` Datum is working on latency-aware routing for gVPC. The platform would continuously measure delay between nodes, and steer traffic onto the path that's actually fastest rather than the one with the fewest hops. You would be able to express intent, such as "prefer low latency", for your workloads, and see the results in metrics. > [!NOTE] > > This section describes where Datum is headed. Plans can change, and this isn't a commitment to deliver any feature or to meet any timeline. ## Learn more * [Segment Routing introduction](https://www.segment-routing.net/tutorials/2016-09-27-segment-routing-introduction/), from segment-routing.net. * [Introduction to Segment Routing](https://packetpushers.net/blog/introduction-to-segment-routing/), from Packet Pushers. * [Meet me in the Galactic VPC](https://www.datum.net/blog/galactic-vpc-backbone/), on why Datum built Galactic on SRv6, from the Datum blog. * [RFC 8402: Segment Routing Architecture](https://www.rfc-editor.org/rfc/rfc8402) and [RFC 8986: SRv6 Network Programming](https://www.rfc-editor.org/rfc/rfc8986), the standards behind it. ## Related * [Galactic](/docs/platform/galactic) — the open source network software that builds Datum's private networks on SRv6. * [gVPC](/docs/galactic-vpc/overview) — private IPv6 networks for your workloads. * [Envoy](/docs/platform/envoy) — how load balancers reach your private network. # ZeroTrust Source: https://datum.net/docs/platform/zerotrust How Datum applies zero trust: private by default, an identity and a certificate for everything, and no trust based on where a connection comes from. Zero trust means nothing is trusted because of where it sits on a network. Every person, service, and connection proves who it is, and every connection is encrypted, whether it crosses the internet or stays inside Datum. Datum builds this into the platform, so you don't have to assemble it yourself. ## Private by default The safest endpoint is one that isn't on the public internet. On Datum, your resources start private: * **[Compute](/docs/platform/isolation) Instances have no public address.** They have a private IPv6 address on your [gVPC](/docs/galactic-vpc/overview) network, isolated from every other customer's network by the routing itself. See [Segment routing](/docs/platform/segment-routing). * **The only public way in is one you choose.** You publish an app through an [Application Load Balancer](/docs/alb/overview), with a web application firewall in front of it. * **Tunnels expose nothing.** A [Datum tunnel](/docs/connectors/tunnels) connects a service on your machine to Datum without opening an inbound port. See [Iroh relays](/docs/platform/iroh-relays). To read more about the thinking behind this, see [Private by default](https://www.datum.net/blog/private-by-default/) on the Datum blog. ## A certificate for everything Datum issues and renews certificates automatically, so encryption is never an afterthought: * **Every load balancer hostname gets a certificate.** A load balancer's generated hostname is served with a Datum-managed certificate from the moment it's created. When you add a custom hostname, Datum issues a certificate for it once DNS is in place. HTTPS is enforced by default. * **Every tunnel endpoint is a key pair.** In [Iroh](https://www.iroh.computer/), which powers Datum tunnels, an endpoint's address *is* its public key. Connections are dialed by key, not by IP address, and encrypted end to end with TLS 1.3 over QUIC. Datum's relays forward only encrypted packets that they can't read. * **Every shell session is encrypted.** An [Instance shell](/docs/compute/instance-shell) is encrypted all the way to the location that runs the Instance. ## An identity for every actor Every request to Datum's API comes from a known identity, and each identity can do only what it's been granted: * **People** sign in to the [Cloud Portal](https://cloud.datum.net) and `datumctl`. `datumctl` uses short-lived tokens that refresh automatically, so there's no long-lived password or key on your machine. * **Machines** use [service accounts](/docs/projects/service-accounts), which authenticate with a signed credential instead of shared passwords or long-lived tokens. You can rotate or disable their keys without redeploying. * **AI agents** connect through the [Datum MCP server](/docs/agents/datum-mcp) with OAuth, with the same permissions as the person or service account they act for. * **Roles** limit what each identity can do, in each organization and project. * **Every action is recorded.** Changes, and who made them, appear in your [activity logs](/docs/operations/activity-logs). ## At your edge You can apply the same thinking to your own apps. A load balancer can require sign-in before a request ever reaches your origin, with [OIDC](/docs/alb/authentication/oidc-google) or [basic authentication](/docs/alb/authentication/basic-auth), and its [web application firewall](/docs/alb/configuration/waf) blocks common attacks. ## Related * [Isolation](/docs/platform/isolation) — how each Instance is contained in its own virtual machine and network. * [Iroh relays](/docs/platform/iroh-relays) — how tunnels connect without exposed ports. * [Segment routing](/docs/platform/segment-routing) — how private networks stay separate on a shared backbone. # Projects Source: https://datum.net/docs/projects/overview Projects group the resources you create on Datum within an organization. A Project belongs to an [organization](/docs/organizations/overview) and holds the resources you create, such as load balancers, workloads, networks, and DNS zones. Most `datumctl` commands run against a Project. ## Projects in the Cloud Portal To see your organization's projects, select your organization in the [Cloud Portal](https://cloud.datum.net), then select **Projects**. Each project shows its ID, which you use with `datumctl`, such as in `datumctl ctx use your-org-id/your-project-id`. To create a project, select **Add project**. An organization can have up to 10 projects. To request more, contact [support@datum.net](mailto:support@datum.net). ![The Projects page listing a Default project with its project ID, an Add project button, and an Understanding Projects panel](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/projects-list.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=bdd82eaf7a377dcbc60c2fc2bd9343ab) Your organization's projects ## In this section * [Service accounts](/docs/projects/service-accounts) — non-human identities for applications, CI/CD pipelines, and agents. ## Related * [Account setup](/docs/getting-started/account-setup) — create an organization and a Project. * [Contexts and scoping](/docs/datumctl/contexts-and-scoping) — choose which organization or Project `datumctl` works in. # Service Accounts Source: https://datum.net/docs/projects/service-accounts Create non-human identities for applications, CI/CD pipelines, and services to authenticate with Datum APIs Service accounts give non-human workloads — CI/CD pipelines, backend services, and scripts — a stable cryptographic identity to call Datum APIs. They authenticate using RSA key pairs rather than shared passwords or long-lived user tokens. ## When to use service accounts | Use case | Example | | - | - | | CI/CD pipelines | GitHub Actions deploying resources on every merge | | Kubernetes workloads | A pod calling the Datum API using a projected secret | | Backend services | A service authenticating as itself, not as a user | | Local automation | Scripts that run without a browser-based login | ## Create a service account The Cloud Portal offers two starting points. **CI/CD Pipeline** generates a credentials file to store as a secret in GitHub Actions, GitLab CI, or another pipeline, with setup instructions. **Service** gives a backend service or workload a stable identity to call Datum APIs, with credentials you mount as a Kubernetes Secret or environment variables. ![The Add a Service Account tab, with two cards: CI/CD Pipeline and Service, each with a Create a service account button](https://mintcdn.com/datum-4926dda5/Xc9AnQUL-oEENeSS/images/service-account-types.png?fit=max\&auto=format\&n=Xc9AnQUL-oEENeSS\&q=85\&s=c568a2dfab7b172a12e150a0d03073a7) Choose the kind of service account to create 1. Open your project in the [Datum Cloud portal](https://cloud.datum.net). 2. Navigate to **Service Accounts** in the left sidebar. 3. Select **Create service account**. 4. Enter a lowercase name (e.g. `ci-deploy`) and an optional display name. ![Service account creation form](https://mintcdn.com/datum-4926dda5/TEJDitxXSc4uEVSS/images/image.png?fit=max\&auto=format\&n=TEJDitxXSc4uEVSS\&q=85\&s=63d5a70deedcf2035fb889253c4766cb) 5. Select a key type: * **Datum-managed** — Datum generates an RSA key pair. The private key is shown **once** at creation time. Download it as a JSON credentials file before closing the wizard. * **User-managed** — You provide your own RSA public key in PEM format. Datum stores only the public key and never sees your private key. ![Key type selection](https://mintcdn.com/datum-4926dda5/TEJDitxXSc4uEVSS/images/image-1.png?fit=max\&auto=format\&n=TEJDitxXSc4uEVSS\&q=85\&s=1c7557d696c6b4e37dcd9c0beb6e5236) 6. Select **Create**. Once the account's identity email is provisioned, your credentials are ready. ![Service account created, credentials available to download](https://mintcdn.com/datum-4926dda5/TEJDitxXSc4uEVSS/images/image-2.png?fit=max\&auto=format\&n=TEJDitxXSc4uEVSS\&q=85\&s=db81205c074e3d3eb57a8a73c109f65b) > [!NOTE] > > For datum-managed keys, the private key is only displayed once. Download the credentials file before closing the wizard — it cannot be recovered later. ## Manage keys Open a service account and navigate to the **Keys** tab to add or revoke keys. * **Add a key** — create an additional datum-managed or user-managed key at any time. * **Revoke a key** — immediately invalidates the key. Any tokens issued with it will stop working at expiry. To rotate keys with zero downtime: add the new key, update your workloads to use it, then revoke the old key. ## Assign roles Service accounts are IAM principals like any other member. Assign roles from the **Roles** tab on the account's detail page, or from **Settings → Members** in your project or organization. ![Policy Bindings form showing role assignments](https://mintcdn.com/datum-4926dda5/TEJDitxXSc4uEVSS/images/image-3.png?fit=max\&auto=format\&n=TEJDitxXSc4uEVSS\&q=85\&s=fd208ebf6322e8c8a4c68c75f9d7e4c0) ## Disable or delete an account * **Disable** — suspends authentication without removing the account or its keys. Re-enable at any time from the **Overview** tab. * **Delete** — permanently removes the account and revokes all associated keys. This cannot be undone. ## Using credentials After creating a service account, you'll have a credentials file to use for authentication. ### Credentials file format When you create a datum-managed key, Datum returns a JSON credentials file. For its fields, see [The credentials file](/docs/datumctl/auth/service-accounts#the-credentials-file). **Treat this file like a password** — store it in a secrets manager and never commit it to source control. ### datumctl Log in with the credentials file, then run commands as the service account: ```bash theme={null} datumctl login --credentials /path/to/credentials.json datumctl get organizations ``` Service-account login doesn't prompt you to pick a context, so pass `--organization` or `--project` on each command, or set a context. For custom environments and more, see [Service accounts in datumctl](/docs/datumctl/auth/service-accounts#log-in-with-a-credentials-file). For a pipeline, see [Automating datumctl in CI/CD](/docs/datumctl/cicd/automating). ### GitHub Actions Store the contents of the credentials file as a repository or organization secret, such as `DATUM_SA_CREDENTIALS`, and pass it to the `datumctl` setup action. The action logs in as the service account, so later steps run `datumctl` without a login command. For the full workflow, see [GitHub Actions](/docs/datumctl/cicd/github-actions). ### Kubernetes Create a Kubernetes secret from the credentials file: ```text theme={null} kubectl create secret generic datum-credentials \ --from-file=credentials.json=/path/to/credentials.json ``` Mount it into your pod, then log in with the mounted file before running `datumctl`: ```yaml theme={null} volumeMounts: - name: datum-credentials mountPath: /var/secrets/datum readOnly: true volumes: - name: datum-credentials secret: secretName: datum-credentials ``` ```bash theme={null} datumctl login --credentials /var/secrets/datum/credentials.json ```