Apache CloudStack KVM Test Environment

Apache CloudStack


In this step-by-step tutorial we expect to run the CloudStack Management Server inside the CentOS host, not inside the devcloud-kvm. This changes a bit the network diagram and also the way the machine are deployed. Below an exhibit of how it will look like once you are done.

Why so many layers?

We are currently running the CenTOS (Host) VM inside our VMware zone, which is deployed in our Apache CloudStack 4.2.0 BetaCloud environment.

In case you do not have a VMware based hypervisor and still want to try CloudStack + KVM,  you can still do everything explained below in your own centOS based computer.

Configure nested virtualisation on CentOS 6.4

Before you start, just check if your CentOS 6.4 server is read for nested virtualisation. In order to do that, check if the file /sys/module/kvm_intel/parameters/nested exists.
On a default CentOS 6 system this file will not exist. Fortunately, we have the xen-c6 project (http://dev.centos.org/centos/6/xen-c6/) that features a kernel that supports the nested parameter!

If your CentOS 6.4 still has a default Kernel, proceed with the steps below:

  1. Add the xen-c6 repo to your system
    1. sudo su -
    2. cd /etc/yum.repos.d
    3. c. wget http://dev.centos.org/centos/6/xen-c6/xen-c6.repo
    4. d. yum repolist
  1. Install the new kernel
    1. yum --enablerepo xen-c6 install kernel kernel-firmware
    2. vi /boot/grub/grub.conf : add "kvm-intel.nested=1" to the end of the kernel line
    3. Reboot the machine
    4. verify proper operation:
      1. cat /sys/module/kvm_intel/parameters/nested
      2. The output should be "Y"

Installing KVM, libvir and virt-manager

  1. Install qemu-kvm
    1. yum install qemu-kvm qemu-kvm-tools
  2. Install libvirt and virt-manager
    1. yum install libvirt virt-manager

Configure libvirt

  1. Edit /etc/libvirt/libvirtd.conf
    1. listen_tls = 0
    2. listen_tcp = 1
    3. unix_sock_group = "ekho"
      1. Or the user name you will use to connect to qemu-kvm via virt-manager
    4. unix_sock_ro_perms = "0777"
    5. unix_sock_rw_perms = "0770"
    6. unix_sock_dir = "/var/run/libvirt"
    7. auth_unix_ro = "none"
    8. auth_unix_rw = "none"
    9. auth_tcp = "none"
  2. Edit /etc/libvirt/qemu.conf

Look for the "user" and "group" parameters and change the following:

    1. From user = "root" to user = "ekho" <-- your own username here.
    2. From group = "root" to group = "ekho" <-- your own username here.
    3. service libvirtd restart

Configure devcloud-kvm

  1. Setup devcloud-kvm
    1. Download the CentOS based devcloud-kvm
      1. http://marcus.mlsorensen.com/cloudstack-extras/devcloud-kvm.tar
    2. Extract devcloud-kvm.tar
      1. tar -xvf devcloud-kvm.tar
      2. cd into devcloud-kvm directory
    3. Define the networks
      1. sudo virsh net-define network-devcloud-kvm-0.xml
      2. sudo virsh net-define network-devcloud-kvm-1.xml
      3. sudo virsh net-start devcloud-kvm-0
      4. sudo virsh net-start devcloud-kvm-1
      5. sudo virsh net-autostart devcloud-kvm-0
      6. sudo virsh net-autostart devcloud-kvm-1
    4. Open devcloud-kvm.xml and change the path of the qcow2 file
      1. e.g. /home/ekho/devcloud-kvm/devcloud-kvm.qcow2
    5. Define the VM and start it
      1. sudo virsh define devcloud-kvm.xml
      2. sudo virsh start devcloud-kvm
  2. Check if the VM is running
    1. sudo virsh list --all
      1. Output should be

Id           Name                         State
1            devcloud-kvm             running

Now you can SSH into the devcloud-kvm machine via one of the two available IPs:

  1. SSH into the VM
    1. ssh root@
    2. ssh root@
      1. The password is password

Building Apache CloudStack

  1. Create RPM packages
    1. git clone https://github.com/apache/cloudstack.git
    2. cd cloudstack/packaging
    3. ./package.sh -d centos63
    4. cd dist/rpmbuild/RPMS/x86_64
    5. rm -f cloudstack-baremetal-agent*
    6. rpm -Uvh cloudstack*
  2. Setup Database and Management Server
    1. cloudstack-setup-databases cloud:password@localhost --deploy-as root
    2. cloudstack-setup-management
      1. Wait 30 seconds so any db upgrades can complete
    3. mysql -u cloud -p < /home/ekho/cloudstack/tools/devcloud-kvm/devcloud-kvm.sql
      1. Before executing this, check the management server IP address specified in the devcloud-kvm.sql. It might not match your IP.
    4. service cloudstack-management restart

Install the cloudstack-agent on the KVM host

Whilst a Xen Server based hypervisor has the CloudStack Agent running inside the Secondary Storage VM, with a KVM based hypervisor it works a bit different: the CloudStack Agent is expected to be running inside the KVM host itself.

In order to achieve that, we have to copy the two RPMs that were generated by the steps on the topic 6.

  1. Copy the RPM files to the KVM host
    1. cd /home/ekho/cloudstacl/dist/rpmbuild/RPMS/x86_64
    2. scp cloudstack-agent* root@
    3. scp cloudstack-common* root@
  2. Connect to the KCM host and install the RPM
    1. ssh root@
      1. The password is password
    2. yum install cloudstack-common-4.6.0-SNAPSHOT.el6.x86_64.rpm cloudstack-agent-4.6.0-SNAPSHOT.el6.x86_64.rpm
  3. Start the cloustack-agent service
    1. service install cloudstack-agent start

Configure the Secondary Storage

We need to allow connections to the NFS export we have in the secondary storage. In order to achieve that, go to your secondary storage (VM with NFS export) and follow the steps below:

  1. Configure portmap
    1. yum --enablerepo="*" install nmap
    2. nmap [ip_of_your_sec_storage] -p 111
    3. vim /etc/hosts.allow
      1. add "ALL: ALL" to the file
    4. Restart the services
      1. /etc/init.d/portmap restart
      2. /etc/init.d/nfs restart
      3. /etc/init.d/nfslock restart
    5. Check if the portmapper, nfs and nfs lock are running fine
      1. rpcinfo -p
  2. Export you secondary storage directory
    1. vim /etc/exportfs
    2. Add the following line to the file
      1. /opt/storage/secondary *(rw,async,no_subtree_check,no_root_squash)
    3. Refresh the exports
      1. exportfs

Apache CloudStack packaging dependencies

All dependencies depicted below can be easily installed with "yum"

  1. Management Server dependencies
    1. Git
    2. Java JDK 1.7
    3. Apache Maven3
    4. Tomcat6
    5. ws-commons-util
    6. gcc
    7. glibc-devel
    8. MySQL-python
    9. Python 2.7
    10. And more...
      1. install all the other in one go:
        1. sudo yum install ipset jakarta-commons-daemon-jsvc jsvc vconfig dhcp httpd syslinux tftp-server xinetd ipmitool mysql-connector-java python-paramiko jakarta-commons-daemon-jsvc jsvc


You might need to disabled SELinux, but first try with it enabled.

  1. Edit /etc/selinux/config
    1. Set SELINUX=disabled
    2. Reboot the machine
  2. Do not run the Management Server with SUDO
    1. You have to be root otherwise the whole tmp directories will get screwed up.
  3. Export the Maven and Java environment variables
    1. export MAVEN_OPTS="-Xmx2048m -XX:MaxPermSize=1024m"
    2. export M2_HOME="/usr/bin/mvn"
    3. export JAVA_HOME="/usr/lib/jvm/java-1.7.0-openjdk-"
    4. export M2_HOME="/usr/share/apache-maven"


  1. devcloud-kvm
    1. https://cwiki.apache.org/confluence/display/CLOUDSTACK/devcloud-kvm
  2. libvirt
    1. https://libvirt.org/formatdomain.html
  3. Debian, KVM and libvirt
    1. https://nsrc.org/workshops/2014/sanog23-virtualization/raw-attachment/wiki/Agenda/ex-debian-kvm-libvirt.htm
  4. Virtualisation with KVM on a CentOS 6.4 Server
    1. https://www.howtoforge.com/virtualization-with-kvm-on-a-centos-6.4-server
  5. Configure nested virtualisation on CentOS 6
    1. http://wiki.centos.org/HowTos/NestedVirt
  6. Nested virtualisation with KVM Intel The cool Kernel stuff
    1. http://kashyapc.com/2012/01/14/nested-virtualization-with-kvm-intel/
    2. Enabling kvm_intel=1
  7. Installing KVM, libvir and virt-manager on CentOS 6.4
    1. https://n40lab.wordpress.com/2012/10/03/installing-kvm-with-yum-and-compiling-libvirt-and-virtmanager/
  8. Configure KVM to allow connection from virt-manager
    1. http://blog.ktz.me/?p=403



Not Published

0/1000 characters
Go Top