LXC (Linux Containers) is an operating system-level virtualization method for running multiple isolated Linux systems (containers) on a single control host. LXC does not provide a virtual machine, but rather provides a virtual environment that has its own process and network space.
It is similar to other OS-level virtualization technologies on Linux such as OpenVZ and Linux-VServer, as well as those on other operating systems such as FreeBSD jails and Solaris Containers.
LXC relies on the Linux kernel cgroups functionality that became available in version 2.6.29, developed as part of LXC. It also relies on other kinds of namespace-isolation functionality, which were developed and integrated into the mainline Linux kernel.
The main benefit of this type of virtualization technology is the lack of hypervizor/guest overhead. This achieves close to bare-metal performance in most cases.
In this blog I'll show you how to configure LXC container and run Apache in it.
1. Prepare the host.
Install the main lxc package, bootstrap scrips and the bridge utility.
On Redhat/Fedora/Centos run:
File: gistfile1.sh
------------------
[root@host1 ~]# yum install lxc bridge-utils febootstrap
On Debian/Ubuntu run:
File: gistfile1.sh
------------------
[root@host1 ~]# apt-get install lxc bridge-utils debootstrap libcap-dev
Create the directory for the container:
File: gistfile1.sh
------------------
[root@host1 ~]# mkdir /lxc
[root@host1 ~]# ln -s /lxc /var/lib/
[root@host1 /lxc]# cd /lxc
[root@host1 /lxc]# mkdir mycontainer share bin
2. Enable cgroup.
For more information on cgroups read my previous article -
here
File: gistfile1.sh
------------------
[root@host1 /lxc]# mkdir /cgroup
[root@host1 /lxc]# echo "none /cgroup cgroup defaults 0 0" >>/etc/fstab
[root@host1 /lxc]# mount /cgroup
[root@host1 /lxc]# mkdir /cgroup/mycontainer
3. Configure bridging.
On Redhat like systems perform the following:
File: gistfile1.sh
------------------
On Redhat like systems perform the following:
[root@host1 /lxc]# chkconfig NetworkManager off
[root@host1 /lxc]# chkconfig network on
[root@host1 /lxc]# service NetworkManager stop
[root@host1 /lxc]# service network start
[root@host1 /lxc]# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
TYPE=Ethernet
HWADDR=aa:bb:cc:dd:ee:ff:11:22
ONBOOT=yes
BRIDGE=br0
USERCTL=no
IPV6INIT=no
[root@host1 /lxc]# vi /etc/sysconfig/network-scripts/ifcfg-br0
HOSTNANE=host1
DEVICE=br0
ONBOOT=yes
BOOTPROTO=static
DELAY=0
TYPE=Bridge
IPADDR=192.168.0.100
NETWORK=192.168.0.0
NETMASK=255.255.255.0
GATEWAY=192.168.0.1
MTU=1500
DNS1=192.168.0.1
IPV6INIT=no
USERCTL=no
[root@host1 /lxc]# vi /etc/sysconfig/network-scripts/ifup-post
if [ $DEVNAME = "br0" ]
then
/usr/sbin/brctl setfd br0 0
fi
[root@host1 ~]# service network restart
On Debian/Ubuntu:
File: gistfile1.sh
------------------
[root@host1 /lxc]# vi /etc/network/interfaces
auto lo
iface lo inet loopback
auto br0
iface br0 inet static
address 192.168.0.100
netmask 255.255.255.0
gateway 192.168.0.1
bridge_ports eth0
bridge_stp off
bridge_maxwait 5
post-up /usr/sbin/brctl setfd br0 0
4. Building the container.
There are several ways to do this:
Using the lxc tools like lxc-debian or
lxc-fedora.
Using the debootstrap or febootstrap scirpts.
Converting an existing openvz containers.
Using libvirt as shown in this post .
To create the LXC container we need a directory containing the file system - rootfs, that we created in step 1 - /lxc - and a configuration file.
To create the config file for the container we can use a template file, adopted from the lxc.conf man page:
File: gistfile1.sh
------------------
[root@host1 /]# cd /lxc
[root@host1 /lxc]# cat mycontainer.cfg
lxc.utsname = mycontainer
lxc.tty = 4
lxc.network.type = veth
lxc.network.link = br0
lxc.network.name = eth0
# on ESX VM veth does not work, you must use:
# lxc.network.type = phys
# lxc.network.name = eth1
# lxc.network.link = eth1
lxc.network.flags = up
lxc.network.mtu = 1500
lxc.network.ipv4 = 192.168.0.101/24
lxc.rootfs = /lxc/ubuntu
lxc.mount = /lxc/ubuntu.fstab
lxc.cgroup.devices.deny = a
# Allow any mknod (but not using the node)
lxc.cgroup.devices.allow = c *:* m
lxc.cgroup.devices.allow = b *:* m
# /dev/null and zero
lxc.cgroup.devices.allow = c 1:3 rwm
lxc.cgroup.devices.allow = c 1:5 rwm
# consoles
lxc.cgroup.devices.allow = c 5:1 rwm
lxc.cgroup.devices.allow = c 5:0 rwm
lxc.cgroup.devices.allow = c 4:0 rwm
lxc.cgroup.devices.allow = c 4:1 rwm
# /dev/{,u}random
lxc.cgroup.devices.allow = c 1:9 rwm
lxc.cgroup.devices.allow = c 1:8 rwm
lxc.cgroup.devices.allow = c 136:* rwm
lxc.cgroup.devices.allow = c 5:2 rwm
# rtc
lxc.cgroup.devices.allow = c 254:0 rwm
lxc.cgroup.cpuset.cpus = 0
lxc.cgroup.cpu.shares = 1024
lxc.cgroup.memory.limit_in_bytes = 512M
lxc.cgroup.memory.memsw.limit_in_bytes = 512M
# restrict capabilities
lxc.cap.drop = audit_control
lxc.cap.drop = audit_write
lxc.cap.drop = mac_admin
lxc.cap.drop = mac_override
lxc.cap.drop = mknod
lxc.cap.drop = setpcap
lxc.cap.drop = sys_admin
lxc.cap.drop = sys_boot
lxc.cap.drop = sys_module
lxc.cap.drop = sys_rawio
lxc.cap.drop = sys_time
The most important lines are:
File: gistfile1.sh
------------------
lxc.utsname = container_name
lxc.network.ipv4 = your_ip
lxc.rootfs = /lxc/rootfs.container_name
lxc.fstab = /lxc/fstab.container_name
We also need the following fstab file to define our chroot like environment:
File: gistfile1.sh
------------------
[root@host1 /lxc]# cat mycontainer.fstab
none /lxc/mycontainer/dev/pts devpts defaults 0 0
none /lxc/mycontainer/proc proc defaults 0 0
none /lxc/mycontainer/sys sysfs defaults 0 0
none /lxc/mycontainer/var/lock tmpfs defaults 0 0
none /lxc/mycontainer/var/run tmpfs defaults 0 0
#/etc/resolv.conf /lxc/mycontainer/etc/resolv.conf none bind 0 0
#/home /lxc/mycontainer/home none bind 0 0
/lxc/share /lxc/mycontainer/share none bind 0 0
/lxc/bin /lxc/mycontainer/lxc/bin none bind,ro 0 0
/cgroup/ubuntu /lxc/mycontainer/lxc/cgroup none bind,ro 0 0
/lib/modules /lxc/mycontainer/lib/modules none bind,ro 0 0
To install a minimum Ubuntu OS in the container we can use debootstrap or lxc-ubuntu as mentioned earlier:
File: gistfile1.sh
------------------
[root@host1 /lxc]# cd /lxc/mycontainer
[root@host1 /lxc/mycontainer]# PKGS=gpgv,kbd,binutils,language-pack-en
[root@host1 /lxc/mycontainer]# debootstrap --variant=minbase --arch amd64 --include=$PKGS lucid ubuntu
or
File: gistfile1.sh
------------------
[root@host1 /lxc/mycontainer]# lxc-ubuntu -p /lxc/mycontainer --trim -r lucid
5. Modify the container OS.
The basic Ubuntu installation needs to be cleaned some in order for it to execute correctly in the container. Things like modules, and start scripts needs to be removed/altered.
File: gistfile1.sh
------------------
[root@host1 /lxc]# cd /lxc
[root@host1 /lxc]# rm mycontainer/var/lib/dpkg/info/udev.postinst
[root@host1 /lxc]# rm mycontainer/var/lib/dpkg/info/plymouth.postinst
[root@host1 /lxc]# rm -rf mycontainer/lib/modules/*
[root@host1 /lxc]# cp /etc/passwd /etc/shadow /etc/group mycontainer/etc/
[root@host1 /lxc]# cat <<EOD> mycontainer/etc/apt/sources.list
deb http://archive.ubuntu.com/ubuntu/ lucid main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ lucid-updates main restricted universe multiverse
deb http://archive.ubuntu.com/ubuntu/ lucid-security main restricted universe multiverse
EOD
[root@host1 /lxc]# mkdir -p mycontainer/lxc/routing mycontainer/lxc/cgroup mycontainer/lxc/bin mycontainer/share
[root@host1 /lxc]# gw=$(route -n | awk '/^0.0.0.0/{print $2}')
[root@host1 /lxc]# echo "route add default gw $gw" >mycontainer/lxc/routing/default
chroot into mycontainer and configure the container:
File: gistfile1.sh
------------------
[root@host1 /lxc]# chroot /lxc/mycontainer /bin/bash
[root@host1 /]# export LANG=C
[root@host1 /]# apt-get update
[root@host1 /]# apt-get install -y adduser apt-utils iproute iptables netbase rsyslog sudo vim
[root@host1 /]# apt-get install -y ssh lsof wget
[root@host1 /]# apt-get install -y iputils-ping mtr-tiny dnsutils bind9-host
[root@host1 /]# apt-get install -y ia32-libs libterm-readline-gnu-perl dialog
[root@host1 /]# apt-get dist-upgrade -y
[root@host1 /]# cat <<EOD> /var/lib/locales/supported.d/en
en_US UTF-8
en_US.Latin1 ISO-8859-1
en_US.Latin9 ISO-8859-15
en_US.ISO-8859-1 ISO-8859-1
en_US.ISO-8859-15 ISO-8859-15
en_US.UTF-8 UTF-8
en_GB.UTF-8 UTF-8
EOD
[root@host1 /]# dpkg-reconfigure locales
[root@host1 /]# /usr/sbin/update-rc.d -f umountfs remove
[root@host1 /]# /usr/sbin/update-rc.d -f hwclock.sh remove
[root@host1 /]# /usr/sbin/update-rc.d -f hwclockfirst.sh remove
[root@host1 /]# rm /etc/init.d/hwclock*
[root@host1 /]# cd etc
[root@host1 /etc]# rm -f mtab
[root@host1 /etc]# ln -s ../proc/mounts mtab
[root@host1 /etc]# cd init
[root@host1 /etc/init]# rm -f console* control* hwclock* module* network-interface*
[root@host1 /etc/init]# rm -f plymouth* procps* tty{4,5,6}.conf udev* upstart* ufw* mountall*
[root@host1 /etc/init]# cd /dev
[root@host1 /dev]# rm mixer* *midi* audio* dsp* smpte* mpu* sequencer sndstat
Exit the chroot and run:
File: gistfile1.sh
------------------
[root@host1 /dev]# exit
[root@host1 /lxc]# cp -a --parents \
/root/.bashrc \
/etc/resolv.conf \
/etc/profile /etc/profile.d \
/etc/bash.bashrc \
/etc/timezone \
/etc/localtime \
/lxc/mycontainer/
[root@host1 /lxc]# echo ubuntulxc > /lxc/mycontainer/etc/hostname
[root@host1 /lxc]# cat <<EOD > /lxc/mycontainer/etc/fstab
tmpfs /dev/shm tmpfs defaults 0 0
EOD
[root@host1 /lxc]# cat <<EOD > /lxc/mycontainer/etc/hosts
127.0.0.1 localhost
[root@host1 /lxc]# cat <<EOD > /lxc/mycontainer/lib/init/fstab
# /lib/init/fstab: static file system information.
#
# These are the filesystems that are always mounted on boot, you can
# override any of these by copying the appropriate line from this file into
# /etc/fstab and tweaking it as you see fit. See fstab(5).
#
# <file system> <mount point> <type> <options> <dump> <pass>
/dev/root / rootfs defaults 0 1
none /proc proc nodev,noexec,nosuid 0 0
none /proc/sys/fs/binfmt_misc binfmt_misc nodev,noexec,nosuid,optional 0 0
none /sys sysfs nodev,noexec,nosuid 0 0
none /sys/fs/fuse/connections fusectl optional 0 0
none /sys/kernel/debug debugfs optional 0 0
none /sys/kernel/security securityfs optional 0 0
none /spu spufs gid=spu,optional 0 0
none /dev devtmpfs,tmpfs mode=0755 0 0
none /dev/pts devpts noexec,nosuid,gid=tty,mode=0620 0 0
none /dev/shm tmpfs nosuid,nodev 0 0
none /tmp none defaults 0 0
none /var/run tmpfs mode=0755,nosuid,showthrough 0 0
none /var/lock tmpfs nodev,noexec,nosuid,showthrough 0 0
none /lib/init/rw tmpfs mode=0755,nosuid,optional 0 0
EOD
[root@host1 /lxc]# rm -f /lxc/mycontainer/etc/init/rc-sysinit
[root@host1 /lxc]# cat << 'EOF' > /lxc/mycontainer/init/rc-sysinit
#!/bin/bash
rm -f $(find /var/run -name '*pid'
rm -f /var/lock/apache/*
route add default gw 192.168.0.1
exit 0
EOF
[root@host1 /lxc]# chmod a+x /lxc/mycontainer/etc/init/rc-sysinit
6. Start the container.
To create and start the LXC run:
File: gistfile1.sh
------------------
[root@host1 /lxc]# lxc-create -f /lxc/mycontainer/mycontainer.cfg -n ubuntu
[root@host1 /lxc]# lxc-start -n ubuntu -d
To list the containers and see their state:
File: gistfile1.sh
------------------
[root@host1 /lxc]# lxc-ls
ubuntu
[root@host1 /lxc]# lxc-info -n ubuntu
RUNNING
You should now be able to access the container with either lxc-console or ssh:
File: gistfile1.sh
------------------
[root@host1 /lxc]# ssh root@192.168.0.100
or
File: gistfile1.sh
------------------
[root@host1 /lxc]# lxc-console -n ubuntu
To stop the container:
File: gistfile1.sh
------------------
[root@host1 /lxc]# lxc-stop -n ubuntu
And to destroy the container run:
File: gistfile1.sh
------------------
[root@host1 /lxc]# lxc-destroy -n ubuntu
While in the container you can install and start services as usual.
Resources:
http://en.wikipedia.org/wiki/LXC
https://help.ubuntu.com/community/LXC
http://blog.bodhizazen.net/linux/lxc-configure-ubuntu-lucid-containers/
http://lxc.teegra.net
