Nick Ali

LXC - LXD

Creating an LXC stack easily

I’ve been futzing around using virtual machines with LXC. When learning, it’s a tedious process to set up everything up so if you make a mistake, you can tear it all down easily.

So I wrote the following script to create a profile, associated network and storage, for a minimal Ubuntu virtual machine. And when done, get rid of it all.

Right now it assumes you have an image with the alias ubuntu-24.04-minimal-x86_64.

To create the stack, run ./stack.sh --create <prefix>.

All the resources will be named <prefix>-<resource>.

To delete the stack, run ./stack.sh --delete <prefix>.

You should only use the delete if you have created the stack with the script.

#!/bin/bash
 
set -e
 
create_resources() {
  local prefix=$1
 
  # Create profile
  echo "Creating profile: ${prefix}-profile"
  lxc profile create "${prefix}-profile"
 
  # Set instance options
  echo "Setting instance options (limits.cpu=2, limits.memory=2GB) for ${prefix}-profile"
  lxc profile set "${prefix}-profile" limits.cpu=2 limits.memory=2GB
 
  # Create bridged network
  local network_name="br-${prefix}"
  echo "Creating network: ${network_name}"
  lxc network create "${network_name}"
 
  # Add network device to profile
  echo "Adding network device eth0 to ${prefix}-profile"
  lxc profile device add "${prefix}-profile" eth0 nic nictype=bridged parent="${network_name}"
 
  # Create ZFS storage pool
  local storage_pool="${prefix}-pool"
  echo "Creating storage pool: ${storage_pool}"
  lxc storage create "${storage_pool}" zfs size=10GB
 
  # Add root disk device to profile
  echo "Adding root disk device to ${prefix}-profile"
  lxc profile device add "${prefix}-profile" root disk path=/ pool="${storage_pool}"
 
  # Launch container with profile
  local container_name="${prefix}-container"
  echo "Launching container: ${container_name} with profile ${prefix}-profile"
  lxc launch ubuntu-24.04-minimal-x86_64 "${container_name}" --profile "${prefix}-profile"
 
  # Execute post-launch commands (install packages)
  # echo "Executing commands in ${container_name} to install packages (telnet, iputils-ping, dnsutils, deborphan)"
  # lxc exec "${container_name}" -- bash -c 'apt-get update; apt-get upgrade -y; apt-get install -y telnet iputils-ping dnsutils deborphan; exit'
 
  echo "Resources created successfully."
}j
 
delete_resources() {
  local prefix=$1
 
  # Fetch container name
  local container_name="${prefix}-container"
 
  # Check if container is running and stop it
  if lxc list --format csv -c ns | grep "^${container_name},RUNNING"; then
    echo "Stopping container: ${container_name}"
    lxc stop "${container_name}" --force
  fi
 
  # Delete container
  echo "Deleting container: ${container_name}"
  lxc delete "${container_name}" || true  # Continue even if deletion fails
 
  # Fetch network name
  local network_name="br-${prefix}"
 
  # Detach network from instances and profiles
  echo "Detaching network from instances and profiles: ${network_name}"
  lxc network detach-profile "${network_name}" "${prefix}-profile" || true  # Continue even if detachment fails
 
  # Delete network
  echo "Deleting network: ${network_name}"
  lxc network delete "${network_name}" || true  # Continue even if deletion fails
 
  # Fetch storage pool name
  local storage_pool="${prefix}-pool"
 
  # Delete storage pool
  echo "Deleting storage pool: ${storage_pool}"
  lxc storage delete "${storage_pool}" || true  # Continue even if deletion fails
 
  # Delete profile
  echo "Deleting profile: ${prefix}-profile"
  lxc profile delete "${prefix}-profile" || true  # Continue even if deletion fails
 
  echo "Resources with prefix ${prefix} deleted successfully."
}
 
# Main script logic
if [ "$1" == "--create" ]; then
  create_resources "$2"
elif [ "$1" == "--delete" ]; then
  delete_resources "$2"
else
  echo "Usage: $0 --create|--delete <prefix>"
  exit 1
fi

Setting up LXC network where the containers can’t connect to the host

Let’s say your default network name is lxdbr0.

Let’s create a network for the isolated containers:

lxc network create lxdbriso \
    ipv4.address=10.185.204.1/24 \
    ipv4.nat=true ipv6.address=none bridge.external_interfaces=eth0

Now let’s create a container:

lxc launch images:ubuntu/jammy c1 -n lxdbriso`

Next, turn on isolation for the newly created container:

lxc config device set c1 eth0 security.port_isolation=true

Set up ACLs that we can apply to the host and isolated network.

lxc network set lxdbr0 \
    security.acls.default.egress.action=allow \
    security.acls.default.ingress.action=allow
lxc network set lxdbriso \
    security.acls.default.egress.action=allow \
    security.acls.default.ingress.action=allow

Create the ACL.

lxc network acl create external-only

Have the ACLs reject each other.

lxc network acl rule add external-only egress \
    destination=10.6.53.0/24 \
    action=reject
lxc network acl rule add external-only egress \
    destination=10.185.204.0/24 \
    action=reject
lxc network acl rule add external-only egress \
    destination=192.168.100.0/24 \
    action=reject

Replace the 10.6.53.0 with the IPv4 listed for lxdbr0 when you run lxc network list. The 10.185.204.0 is the IP range we defined when creating lxdbriso above.

I also don’t want the containers to be access anything on the LAN. So the 192.168.100.0/24 is block any LAN IP ranges. You can add as many as you need. Apply the ACLs.

lxc network set lxdbr0 security.acls=external-only
lxc network set lxdbriso security.acls=external-only

If lxdbr0 already has security.acls defined, you can add more like:

lxc network set lxdbr0 security.acls=external-only,previous-acl

Open up a c1 shell: lxc exec c1 bash and try to ping the host and other machines on the LAN. You shouldn’t be able to. Ping something out on the Internet and it should be successful.

If you want to add containers that cannot talk to each other, create a new network for each container and duplicate the lxdbriso commands.

On the other hand if want the containers to be able to talk to each other (and not the host), set the network to lxdbriso when creating the containers.

Most of this was taken from a linuxcontainers.org forum post.

Memory & CPU Limits

If you want to limit the CPUs and memory:

lxc config set sso-container limits.cpu 2
 
# Shutdown instance before changing memory limits
lxc config set sso-container limits.memory 4096MB

Overwritten DNS

If you make changes to the DNS and it reverts when you reboot, put the nameservers in /run/systemd/resolve/resolv.conf.

Error: runc create failed: unable to start container process

You might get an error like this:

docker: Error response from daemon: failed to create task for container: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: error during container init: error mounting "proc" to rootfs at "/proc": mount proc:/proc (via /proc/self/fd/6), flags: 0xe: permission denied: unknown.

Make sure nesting is turned for the continainer:

lxc config set <container name> security.nesting true

Related