Custom network configuration with cloud-init

cloud-init may be used for custom network configuration of containers.

Before trying to use it, however, first determine which image source you are about to use as not all container images have cloud-init package installed. At the time of writing, images provided at images.linuxcontainers.org do not have the cloud-init package installed, therefore, any of the configuration options mentioned in this guide will not work. On the contrary, images provided at cloud-images.ubuntu.com have the necessary package installed and also have a templates directory in their archive populated with

cloud-init-meta.tpl
cloud-init-user.tpl
cloud-init-vendor.tpl
cloud-init-network.tpl

and others not related to cloud-init.

Templates provided with container images at cloud-images.ubuntu.com have the following in their metadata.yaml:

/var/lib/cloud/seed/nocloud-net/network-config:
  when:
    - create
    - copy
  template: cloud-init-network.tpl

Therefore, either when you create or copy a container it gets a newly rendered network configuration from a pre-defined template. cloud-init uses the network-config file to render /etc/network/interfaces.d/50-cloud-init.cfg when you first start a container. It will not react to any changes if you restart a container afterwards unless you force it.

The default behavior is to use a DHCP client on a container's eth0 interface.

In order to change this you need to define your own network configuration using user.network-config key in the config dictionary which will override the default configuration (this is due to how the template is structured).

The allowed values follow /etc/network/interfaces syntax in case of Ubuntu images.

For example, to configure a specific network interface with a static IPv4 address and also use a custom nameserver use

config:
  user.network-config: |
    version: 1
    config:
      - type: physical
        name: eth1
        subnets:
          - type: static
            ipv4: true
            address: 10.10.101.20
            netmask: 255.255.255.0
            gateway: 10.10.101.1
            control: auto
      - type: nameserver
        address: 10.10.10.254

A container's rootfs will contain the following files as a result:

/var/lib/cloud/seed/nocloud-net/network-config
/etc/network/interfaces.d/50-cloud-init.cfg

The former will be the same as the value provided in user.network-config, the latter will be a file in /etc/network/interfaces format converted from the network-config file by cloud-init (if it is not check syslog for cloud-init error messages).

/etc/network/interfaces.d/50-cloud-init.cfg should then contain

# This file is generated from information provided by
# the datasource.  Changes to it will not persist across an instance.
# To disable cloud-init's network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
auto lo
iface lo inet loopback
    dns-nameservers 10.10.10.254

auto eth1
iface eth1 inet static
    address 10.10.101.20
    gateway 10.10.101.1
    netmask 255.255.255.0

You will also notice that /run/resolvconf/resolv.conf or /etc/resolv.conf which is pointing to it will contain the desired dns server after boot-up.

# Dynamic resolv.conf(5) file for glibc resolver(3) generated by resolvconf(8)
#     DO NOT EDIT THIS FILE BY HAND -- YOUR CHANGES WILL BE OVERWRITTEN
nameserver 10.10.10.254

Implementation Details

cloud-init allows you to seed instance configuration using the following files located at /var/lib/cloud/seed/nocloud-net:

user-data (required)
meta-data (required)
vendor-data (optional)
network-config (optional)

The network-config file is written to by lxd using data provided in templates that come with an image. This is governed by metadata.yaml but naming of the configuration keys and template content is not hard-coded as far as lxd is concerned - this is purely image data that can be modified if needed.

NoCloud data source documentation
The source code for NoCloud data source
A good reference on which values you can use are unit tests for cloud-init
cloud-init directory layout

A default cloud-init-network.tpl provided with images from the "ubuntu:" image source looks like this:

{% if config\_get("user.network-config", "") == "" %}version: 1
config:
    - type: physical
      name: eth0
      subnets:
          - type: {% if config_get("user.network_mode", "") == "link-local" %}manual{% else %}dhcp{% endif %}
            control: auto{% else %}{{ config_get("user.network-config", "") }}{% endif %}

The template syntax is the one used in the pongo2 template engine. A custom config_get function is defined to retrieve values from a container configuration.

Options available with such a template structure:

Use DHCP by default on your eth0 interface;
Set user.network_mode to link-local and configure networking by hand;
Seed cloud-init by defining user.network-config.