Cloud

Stacklane creates cloud instances on Proxmox VE. Each region is one Proxmox cluster. Customers choose a region, an image (operating system) and a flavor (size), and Stacklane clones a template on the cluster. Only staff with the owner or admin role can manage the cloud settings.

Before you start#

You need:

Proxmox API token#

Stacklane signs in with a Proxmox API token. It sends the header Authorization: PVEAPIToken=<token ID>=<secret>. The token ID has the form user@realm!tokenname, for example stacklane@pve!panel. Stacklane rejects a token ID without @ and !.

What the token must be allowed to do#

Give the token enough permissions for every call in this table, on the nodes, the storage, the bridge and the VMs that Stacklane manages.

Stacklane actionProxmox API calls
Test connectionGET /version, GET /nodes
Choose a nodeGET /nodes (status and free memory)
Find the templateGET /cluster/resources?type=vm
Reserve a VM IDGET /cluster/nextid
Create an instancePOST /nodes/{node}/qemu/{template}/clone (full clone to the region storage and target node), POST /nodes/{node}/qemu/{vmid}/config, PUT /nodes/{node}/qemu/{vmid}/resize
Power actionsPOST /nodes/{node}/qemu/{vmid}/status/{start,stop,shutdown,reboot}, GET .../status/current
Wait for a taskGET /nodes/{node}/tasks/{upid}/status
FirewallPUT .../firewall/options, GET, POST and DELETE .../firewall/rules
Monitoring graphsGET /nodes/{node}/qemu/{vmid}/rrddata
SnapshotsPOST .../snapshot, POST .../snapshot/{name}/rollback, DELETE .../snapshot/{name}
VolumesGET and PUT /nodes/{node}/qemu/{vmid}/config (add and remove scsiN disks)
Delete an instancePOST .../status/stop, DELETE /nodes/{node}/qemu/{vmid}?purge=1&destroy-unreferenced-disks=1

Stacklane waits up to 15 minutes for a Proxmox task. After that the task fails with "the operation timed out".

TLS#

Keep Verify the TLS certificate of the cluster on when the cluster has a valid certificate. Turn it off only when the cluster uses a self-signed certificate on a network you trust.

Templates#

Stacklane creates each instance as a full clone of a template. Prepare one template VM per image in each region:

  1. Install the operating system with cloud-init support.
  2. Install the QEMU guest agent. Stacklane turns on the guest agent option (agent=1) on every instance.
  3. Put the boot disk on scsi0. Stacklane resizes scsi0 to the disk size of the flavor. Proxmox cannot make a disk smaller, so keep the template disk as small as, or smaller than, your smallest flavor.
  4. Add a cloud-init drive.
  5. Convert the VM to a template in Proxmox.
  6. Note the VM ID of the template.

The template can be on any node of the cluster. Stacklane finds it in the cluster resources.

When Stacklane creates an instance, it sets these options on the clone:

OptionValue
coresThe vCPUs of the flavor.
memoryThe memory of the flavor in MiB.
ciuserThe default user of the image.
cipasswordThe password the customer entered, if any.
sshkeysThe SSH keys the customer selected.
net0virtio,bridge=<region bridge>,firewall=1
ipconfig0ip=<address>/<prefix>,gw=<gateway> from the IPv4 pool, or ip=dhcp when the pool has no free address.
nameserverThe nameservers of the region.
onboot1
agent1

Then Stacklane starts the instance.

Regions#

Open Regions in the Cloud group and click Add region.

FieldRules and meaning
NameRequired. Up to 60 characters. Customers see this name.
SlugRequired. Lowercase letters, digits and hyphens, up to 40 characters. Must be unique.
CountryTwo-letter country code. Customers see it next to the region name.
API endpointThe URL of the cluster, for example https://pve.example.net:8006.
API token IDFor example stacklane@pve!panel.
API token secretRequired for a new region. Leave it empty to keep the current secret. When you change the endpoint to another host, you must enter the secret again.
StorageThe Proxmox storage for instance disks and volumes, for example local-lvm.
Network bridgeThe bridge for the instance network, for example vmbr0.
Preferred nodeOptional. Leave it empty to place each instance on the online node with the most free memory. When you set a node, all instances go to that node.
NameserversIP addresses, separated by spaces, that instances use as DNS resolvers. Default: 1.1.1.1 9.9.9.9.
Verify the TLS certificate of the clusterOn by default.
Statusactive or disabled.

Click Save. Then click Test connection under Cluster health. Stacklane shows the Proxmox version and the nodes with their CPU and memory use. If the test fails, the error message from Proxmox is shown.

When a region is disabled, customers do not see it and cannot create new instances in it. Existing instances in the region keep working.

Image templates#

On the region page, under Image templates, enter the template VM ID for each image you offer in this cluster. The VM ID must be between 100 and 999,999,999. Leave the field empty to stop offering the image in this region.

Customers only see the images that have a template in the selected region.

IPv4 pool#

Under IPv4 pool, add the public addresses that Stacklane gives to instances.

  1. In Addresses, enter single addresses or ranges, separated by commas or new lines. For example:

    203.0.113.10-203.0.113.50
    203.0.113.60, 203.0.113.61
    
  2. Enter the Prefix length, from 8 to 32, for example 24.

  3. Enter the Gateway.

  4. Click Add addresses.

You can add at most 4,096 addresses in one step. The page shows how many addresses are assigned. You can only remove an address that is not assigned to an instance.

Each new instance gets the first free address of the pool. When an instance is deleted, or its creation fails, the address becomes free again. When the pool is empty, the instance is created with DHCP and the portal shows no public address.

Images#

Open Images in the Cloud group. An image describes an operating system. You link it to a template per region on the region page.

FieldRules and meaning
NameRequired. Up to 60 characters, for example Ubuntu 24.04.
SlugRequired. Lowercase letters, digits and hyphens. Must be unique.
FamilyLowercase letters, digits and hyphens, for example ubuntu.
VersionRequired. Up to 30 characters, for example 24.04.
Default userThe cloud-init user, for example ubuntu or debian. Lowercase letters, digits, _ and -, up to 32 characters.
Statusactive or disabled. Customers only see active images.
Sort orderA number that sets the order in lists.

Flavors#

Open Flavors in the Cloud group. A flavor is an instance size with a monthly price.

FieldRules
NameRequired. Up to 60 characters.
SlugRequired. Lowercase letters, digits and hyphens. Must be unique.
vCPUs1 to 256.
Memory (MiB)256 to 4,194,304.
Disk (GB)1 to 65,536.
Monthly price0 to 1,000,000.00, without VAT.
CurrencyMust be the billing currency. See Billing.
Statusactive or disabled. Customers can only choose active flavors.
Sort orderA number that sets the order in lists.

Customers see the monthly price, the hourly price (monthly price ÷ 730) and the monthly price including their VAT rate. Usage is billed per hour. See Billing.

Limits and volume price#

Open Limits in the Cloud group. These limits apply to every organization.

FieldDefaultRangeMeaning
Maximum instances100 to 10,000Instances per organization.
Maximum vCPUs400 to 100,000Total vCPUs of all instances of an organization.
Maximum memory (MiB)81,9200 to 1,000,000,000Total memory of all instances of an organization.
Snapshots per server30 to 50Snapshots per instance.
Volumes per server40 to 16Extra volumes per instance.
Volume price (cents per GB per month)100 to 100,000Without VAT, billed per hour.

Every instance that is not deleted counts toward the quota, also an instance with an error. When a new instance would go above a limit, the customer gets the error quota_exceeded. Customers see their usage and limits when they create an instance.

Firewall, snapshots and volumes#

Customers manage these features themselves in the portal. This section explains what happens on the cluster.

Firewall#

When a customer saves firewall rules, Stacklane replaces all rules of the VM on Proxmox and turns on the VM firewall with these defaults:

So inbound traffic that matches no rule is blocked, and outbound traffic is allowed unless a rule blocks it. A customer can have up to 50 rules per instance. Each rule has a direction (in or out), an action (accept or drop), and optionally a protocol (tcp, udp or icmp), a port or port range (for TCP and UDP only), a source address or network, and a comment.

Snapshots#

A snapshot is a Proxmox VM snapshot. Customers can take a snapshot, restore it and delete it. The number of snapshots per instance is limited by Snapshots per server. Snapshots are not billed separately.

Volumes#

A volume is an extra disk on the region storage. Stacklane adds it as scsi1, scsi2 and so on. A volume is 1 to 4,096 GB. The number of volumes per instance is limited by Volumes per server. When a customer deletes a volume, Stacklane removes the disk from the VM and deletes it from the storage.

How provisioning works#

Changes to instances are not made during the API request. Stacklane puts a task in a queue and answers with 202 Accepted. A background worker runs the tasks one after the other.

  1. The customer creates an instance. Stacklane checks the quota, saves the instance with the status creating and adds an instance.create task.
  2. The worker takes the task. It reserves an address from the pool, clones the template, sets the options, resizes the disk and starts the VM.
  3. On success the instance gets the status running. On failure it gets the status error, and the address goes back to the pool.

Other actions work the same way. Each one is a task of its own kind:

Task kindStarted by
instance.createCreating an instance.
instance.powerStart, power off, shut down, reboot. Also used by Stacklane to power off instances of overdue accounts.
instance.deleteDeleting an instance.
instance.firewallSaving firewall rules.
instance.snapshotTaking, restoring or deleting a snapshot.
instance.volumeAdding or deleting a volume.

A task has the state pending, running, finished or error. An instance can only run one task at a time. A second action while a task is pending or running returns 409 with the code busy.

When a task fails#

Stacklane does not retry failed tasks.

Find the error#

Common errors#

ErrorMeaningWhat to do
cannot reach the hypervisor: ...Stacklane cannot connect to the API endpoint.Check the endpoint, the firewall between Stacklane and the cluster, and the TLS setting. Use Test connection.
the hypervisor rejected the request: 401 ... or 403 ...The token is wrong or misses a permission.Check the token ID, the secret and the token's permissions.
template <id> was not found in this regionNo VM with that ID is marked as a template in the cluster.Fix the template VM ID on the region page.
image is not available in this regionThe image has no template in the region.Add the template VM ID on the region page.
no node has enough free capacityNo online node has enough free memory, or the preferred node is offline.Add capacity, or clear or change the preferred node.
task failed on the hypervisor: ...A Proxmox task ended with an error, for example a failed clone.Look at the task log in Proxmox for the details.
the operation timed outA Proxmox task took more than 15 minutes.Check the cluster and the storage.
interrupted by a restartStacklane was restarted while the task was running.Check the state of the VM in Proxmox and repeat the action.
region credentials cannot be decryptedThe secret key of the server changed.Restore the original secret key, or enter the token secret again.

Repair the instance#

For testing without a Proxmox cluster, cloud.simulate = true in the configuration file uses a built-in simulator. See Configuration.

API endpoints#

All endpoints need the owner or admin staff role.

MethodPathDescription
GET, POST/api/admin/cloud/regionsList or create regions.
GET, PATCH/api/admin/cloud/regions/{id}Read or update a region.
POST/api/admin/cloud/regions/{id}/checkTest the connection to the cluster.
GET, PUT/api/admin/cloud/regions/{id}/imagesList or set template VM IDs. Body for PUT: {"image_id": 1, "template_vmid": 9000}. Use null to remove.
GET, POST/api/admin/cloud/regions/{id}/addressesList or add pool addresses. Body for POST: {"addresses": "...", "prefix": 24, "gateway": "203.0.113.1"}.
DELETE/api/admin/cloud/regions/{id}/addresses/{address}Remove a free address by its ID.
GET, POST/api/admin/cloud/flavorsList or create flavors.
PATCH/api/admin/cloud/flavors/{id}Update a flavor.
GET, POST/api/admin/cloud/imagesList or create images.
PATCH/api/admin/cloud/images/{id}Update an image.
GET, PUT/api/admin/cloud/settingsRead or save limits and the volume price.