Automation & Integrations•6 min read•Multi-Cloud

Ephemeral Provisioners & Cloud-Init Webhooks

Bootstrap Linux compute instances automatically on first boot using cloud-init and token-authenticated CIEL Provisioners without manual inventory registration.

When scaling out infrastructure or launching ephemeral worker nodes across cloud providers (such as Hetzner, AWS, DigitalOcean, GCP, or private Proxmox clusters), manually registering IP addresses, selecting credentials, and assigning inventory groups for every new machine creates bottlenecks.

CIEL Ephemeral Provisioners invert this process: virtual machines trigger their own provisioning workflows upon first startup. By embedding a lightweight HTTP call into a cloud-init script or bootstrap image, nodes configure themselves agentlessly over SSH the moment they boot.

Ephemeral Server Provisioners and Automated Bootstrapping Endpoints

1. How Inbound Provisioning Works

Inbound provisioning allows compute nodes to register and pull configurations without exposing full dashboard privileges to external systems:

Interactive Architecture Diagram(Mermaid)
Rendering diagram...

Architecture Principles:

  • Direct-Target Execution (No Inventory Bloat): Provisioners inject the target host as an ephemeral direct execution. The machine does not clutter your persistent project inventories unless explicitly saved by a task.
  • Provider-Agnostic: Compatible with any platform supporting cloud-init, custom startup scripts, or SaaS billing webhooks.
  • Agentless Execution: Leverages native Linux SSH and Python runtimes without installing custom agent daemons on target nodes.

2. Configuring a Provisioner in the Dashboard

Under Provisioners (/provisioners in the sidebar under RUNS & JOBS), click + New Provisioner to open the configuration drawer:

1. Provisioner Name & Description

Set an operational title (for example, Hetzner Cloud Ephemeral Worker or AWS Spot Autoscaler Node).

2. Target Playbook

Choose the Ansible playbook to run against the newly booted machine. This playbook defines package installations, security hardening, user creation, or service setup.

3. Execution Environment

Select between Podman (runs the automation inside an isolated container sandbox) or Native (executes directly via the local host subprocess).

4. Default SSH Key

Assign a default private key credential from your Key Vault. If an incoming payload does not specify an ssh_key_id, CIEL falls back to this key to connect to the node.

5. Variable Sets

Attach workspace variable sets to the provisioner. Variables are merged with your standard workspace precedence rules and exposed to the target playbook during execution.

6. Inbound Caller Security

  • Allowed Caller IPs / CIDRs: Restrict incoming trigger requests to trusted IP ranges (such as your cloud provider metadata service, VPC gateway, or SaaS billing server). Leave empty to accept signed requests from any IP (0.0.0.0/0).
  • Required Extra Variables: Enforce mandatory JSON payload keys (such as cluster_id or environment). Requests missing any required key are rejected with HTTP 422 Unprocessable Entity before launching the runner.
  • Rate Limiting: Define maximum allowed requests per minute (default: 30 req/min) to prevent accidental loops or denial-of-service attempts.
  • Bearer Secret: A 48-character cryptographic secret string. Incoming requests must supply this secret via the Authorization: Bearer <secret> header or X-Provision-Secret header.

3. Interactive Installer Snippet Workbench

Clicking the </> Snippet button on any provisioner row opens the Installer Snippet Workbench. This dialog generates ready-to-run integration code tailored to your server environment:

Interactive Provisioner Snippet Dialog with Multi-Language Code Generators

Workbench Features:

  • Parameter Inputs: Customize target parameters directly in the modal, including Test Target IP, SSH Port, SSH User (default root), and Authentication Type (Password vs SSH Key ID).
  • Multi-Format Code Generation:
    • cURL: Single-line HTTP request for command-line testing or CI/CD pipelines.
    • Bash Script: Resilient script with automatic IP discovery (hostname -I or ifconfig.me).
    • Python 3: Script utilizing standard urllib.request without external dependencies, suitable for custom image builders and provisioning daemons.
  • Syntax-Highlighted Preview: Code block with endpoint badge and one-click copy functionality.

4. Production Cloud-Init Recipe

When launching a new cloud server, supply the following script in the User Data / Cloud-Init configuration field:

#cloud-config(yaml)
#cloud-config
# Automated CIEL node bootstrap on first boot

runcmd:
  - |
    # 1. Resolve the primary network IP of this virtual machine
    TARGET_IP=$(hostname -I | awk '{print $1}')
    
    # 2. Trigger the CIEL Provisioner endpoint
    curl -f -X POST "https://ciel.your-company.com/api/provision/trigger/YOUR_PROVISIONER_TOKEN" \
      -H "Content-Type: application/json" \
      -H "Authorization: Bearer YOUR_PROVISIONER_SECRET" \
      -d "{
        \"target\": {
          \"address\": \"${TARGET_IP}\",
          \"port\": 22,
          \"username\": \"root\",
          \"auth_type\": \"ssh_key\",
          \"ssh_key_id\": 1
        },
        \"extra_variables\": {
          \"node_role\": \"worker\",
          \"bootstrapped_via\": \"cloud-init\"
        }
      }"
Note

Ensure your CIEL Controller is reachable over the network by the booted VPS (via a public domain, reverse proxy with TLS, or private overlay network like WireGuard or Tailscale).


5. Security Guardrails & Target IP Quarantine

Because provisioner endpoints accept external inbound triggers, CIEL implements active security defenses:

Single-Shot IP Quarantine (Lockouts)

If an incoming request repeatedly targets an invalid IP, encounters SSH authentication failures, or attempts to target restricted addresses (such as 127.0.0.1, localhost, or link-local metadata addresses like 169.254.169.254), CIEL automatically quarantines the target IP:

  • While quarantined, subsequent trigger attempts against that IP return HTTP 423 Target IP Locked.
  • Navigate to the Target IP Quarantine tab to review blocked addresses, failure counts, and timestamps.
  • Administrators can click Unblock on individual records or Clear All to reset the quarantine table.
  • Administrators can also create manual quarantine records with custom expiration periods.

Creator Permission Verification

When a trigger arrives, the background worker validates that the user who created the provisioner remains active and retains owner, maintainer, or operator permissions within the project. If the creator is deactivated, triggers are rejected with HTTP 403 Forbidden.

Credential Rotation

If a provisioner secret or token is exposed, click the three-dots action menu (...) and select Regenerate secret key. CIEL immediately issues a new URL token and Bearer secret, invalidating the old credentials without modifying playbook or variable bindings.


6. Audit Trail & Trigger Logs

Every provisioning attempt is tracked under the Trigger Activity & Logs tab:

  • Caller IP & User Agent: Identifies the source triggering the endpoint.
  • Status & Rejection Reason: Detailed records for accepted runs, IP whitelist rejections (rejected_ip), missing secret errors (unauthorized), rate-limit violations (rate_limited), and payload validation failures (invalid_payload).
  • Direct Job Links: Accepted requests include direct links to real-time ANSI terminal output in the Jobs viewer.

7. External References

For related documentation and upstream tools: