Eucalyptus

Eucalyptus 安裝手冊

參考文件

我的環境

pCloud.png

下載Eucalyptus主程式、相依程式與管理工具

[Download Page]

$ tree Eucalyptus2.0/
Eucalyptus2.0/
├── euca2ools-1.3.1-src-deps.tar.gz
├── euca2ools-1.3.1.tar.gz
├── eucalyptus-2.0.0-src-deps.tar.gz
└── eucalyptus-2.0.0-src-offline.tar.gz

pre-install

Ubuntu desktop 10.04

$ sudo apt-get install bzr gcc make apache2-threaded-dev ant openjdk-6-jdk libvirt-dev libcurl4-openssl-dev dhcp3-server vblade apache2 unzip curl vlan bridge-utils libvirt-bin kvm vtun tgt parted

Debian Gnu/Linux Squeeze

$ sudo apt-get install bzr gcc make apache2-threaded-dev ant openjdk-6-jdk libvirt-dev libcurl4-openssl-dev dhcp3-server vblade apache2 unzip curl vlan bridge-utils libvirt-bin kvm sudo vtun tgt parted

安裝步驟

解壓縮 eucalyptus 主程式與設定環境變數

$ tar zxf eucalyptus-2.0.0-src-offline.tar.gz
$ cd eucalyptus-2.0.0
$ export EUCALYPTUS_SRC=`pwd`
$ export EUCALYPTUS=/opt/eucalyptus

安裝相依套件

裝相依套件解壓縮完,放置 $EUCALYPTUS_SRC 下,與建立安裝目錄 $EUCALYPTUS/packages/

$ cd $EUCALYPTUS_SRC
$ tar zxf ../eucalyptus-2.0.0-src-deps.tar.gz
$ sudo mkdir -p $EUCALYPTUS/packages/

Axis2

$ sudo tar zxf $EUCALYPTUS_SRC/eucalyptus-src-deps/axis2-1.4.tgz -C $EUCALYPTUS/packages

之前忘了加 sudo 所以後面 build eucalyptus 時,java 會發現找不到 classpath

Axis2/C

設定 APACHE and APR header檔,環境變數

$ export APACHE_INCLUDES=/usr/include/apache2
$ export APR_INCLUDES=/usr/include/apr-1.0

在試一次看看,你設的變數是否有用

$ ls $APACHE_INCLUDES
$ ls $APR_INCLUDES

試了二個指令,其中一個沒出現 header 的檔案,那有可能是你的系統安裝目錄可能在不一樣的地方

設定 AXIS2C 與 編譯安裝

$ export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.6.0
$ cd $EUCALYPTUS_SRC/eucalyptus-src-deps/
$ tar zxf axis2c-src-1.6.0.tar.gz
$ cd axis2c-src-1.6.0
$ CFLAGS="-w" ./configure —prefix=${AXIS2C_HOME} —with-apache2=$APACHE_INCLUDES —with-apr=$APR_INCLUDES —enable-multi-thread=no
$ make
$ sudo make install

Rampart/C

設定 Rampart/C 與 編譯安裝

$ export AXIS2C_HOME=$EUCALYPTUS/packages/axis2c-1.6.0
$ export LD_LIBRARY_PATH=${AXIS2C_HOME}/lib:$LD_LIBRARY_PATH
$ cd $EUCALYPTUS_SRC/eucalyptus-src-deps/
$ tar zxf rampartc-src-1.3.0-0euca1.tar.gz
$ cd rampartc-src-1.3.0
$ ./configure —prefix=${AXIS2C_HOME} —enable-static=no —with-axis2=${AXIS2C_HOME}/include/axis2-1.6.0
$ make
$ sudo make install

edit $AXIS2C_HOME/axis2.xml and change <!phase name="Security"/> to <phase name="Security"/> in the 'inflow' and 'outflow' section

sudo vi $AXIS2C_HOME/axis2.xml

安裝 Eucalyptus 2.0

設定 JAVA 環境變數

$ export JAVA_HOME="/usr/lib/jvm/java-6-openjdk"
$ export JAVA="$JAVA_HOME/bin/java"

一樣,別忘了試看看,些變數是否設定正確

Eucalyptus主程式編譯安裝

$ cd $EUCALYPTUS_SRC
$ ./configure —with-axis2=$EUCALYPTUS/packages/axis2-1.4 —with-axis2c=$EUCALYPTUS/packages/axis2c-1.6.0 —enable-debug —prefix=$EUCALYPTUS
$ make
$ sudo make install

新增 Eucalyptus 使用者

$ sudo useradd eucalyptus

設定 Eucalyptus 組件 (Configure Eucalyptus components)

$ sudo $EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS —hypervisor kvm —instances /usr/local/eucalyptus —user eucalyptus —setup

網路環境設定 (Network Configuration)

edit $EUCALYPTUS/etc/eucalyptus/eucalyptus.conf file on the Cluster head-node:

$ sudo vi $EUCALYPTUS/etc/eucalyptus/eucalyptus.conf
VNET_BRIDGE="br0"
VNET_DHCPDAEMON="/usr/sbin/dhcpd3"
VNET_DHCPUSER="dhcpd"

bridge network setup

$ sudo vi /etc/network/interfaces
auto br0
iface br0 inet static
address 192.168.1.8
network 192.168.1.0
netmask 255.255.255.0
broadcast 192.168.1.255
gateway 192.168.1.1
bridge_ports eth0
bridge_fd 9
bridge_hello 2
bridge_maxage 12
bridge_stp off

Network restart

$ sudo /etc/init.d/networking restart

設定啟動腳本 (Configure your startup scripts)

Add eucalyptus-cloud on the Cloud head-node

$ sudo ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cloud /etc/init.d/eucalyptus-cloud

Add eucalyptus-cc on the Cluster head-node(s)

$ sudo ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-cc /etc/init.d/eucalyptus-cc

Bring daemon on booting process

$ sudo update-rc.d eucalyptus-cloud defaults

Eucalyptus Go

Eucalyptus 架構簡介

architecture-1.6.png
Walrus - a storage service included with Eucalyptus that is interface compatible with Amazon's S3.
Cloud controller (CLC) - 是一個 Java 程式,它提供 Web-service 和 Web interface 給外界,管理各種要求,負責更高階的排程和系統 accounting,也提供 S3-compatible bucket-based storage (Walrus) 和 EBS-style block-based storage
Cluster controller (CC) - C 撰寫,cluster-level 的排程和網路控制,在 Apache 內當成 Web services 來佈署
Node controller (NC) - C 撰寫,hypervisior 的 control,在 Apache 內當成 Web services 來佈署

Enable services on the front-end

$ sudo $EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS —setup
$ sudo $EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS —enable cloud —enable walrus —enable sc

Start each component on the appropriate host.

# start enabled front-end services
$ sudo $EUCALYPTUS/etc/init.d/eucalyptus-cloud start

# start the cluster controller
$ sudo $EUCALYPTUS/etc/init.d/eucalyptus-cc start

測試 Cloud Controller 是否正常

Connect to https://localhost:8443 with browser, 正常因該可以看到下面的畫面,如沒問題,那在來就是安裝 Node Controller的工作。
screenshot_2010-10-18-011455.png

Deploying Eucalyptus Node Controller

如成功完成上面所有的動作後,可以直接從些電腦利用 rsync 的方式復製到每台電腦,這樣可省下不少時間。

Deploying by rsync

利用 rsync 將 Eucalyptus 程式同步到其它機器

$ rsync -a $EUCALYPTUS/ root@{node ip address}:$EUCALYPTUS/

以下動作全部於Note Controller機器上操作

設定 Eucalyptus 環境變數

$ export EUCALYPTUS=/opt/eucalyptus

新增 eucalyptus 使用者帳號

$ sudo useradd eucalyptus

設定 Eucalyptus 組件 (Configure Eucalyptus components)

$ sudo $EUCALYPTUS/usr/sbin/euca_conf -d $EUCALYPTUS —hypervisor kvm —instances /usr/local/eucalyptus —user eucalyptus —setup

設定啟動腳本 (Configure your startup scripts)

Add eucalyptus-nc on the compute node(s)

$ sudo ln -sf $EUCALYPTUS/etc/init.d/eucalyptus-nc /etc/init.d/eucalyptus-nc

Running Eucalyptus

On each of the compute nodes run:

$ sudo $EUCALYPTUS/etc/init.d/eucalyptus-nc start

install euca2ools

安裝相依套件

$ sudo apt-get install python-boto python-m2crypto

安裝 euca2ools

$ tar zxf euca2ools-1.3.1.tar.gz; cd euca2ools-1.3.1
$ sudo make

設定PATH環境變數

$ export PATH=$PATH:$PWD/bin

Connect to NC with CLC

Registering eucalyptus components on the front end

$EUCALYPTUS/usr/sbin/euca_conf —register-walrus <front end IP address>
$EUCALYPTUS/usr/sbin/euca_conf —register-cluster <clustername> <front end IP address>
$EUCALYPTUS/usr/sbin/euca_conf —register-sc <clustername> <front end IP address>
$EUCALYPTUS/usr/sbin/euca_conf —register-nodes "<Node 0 IP address> <Node 1 IP address> … <Node N IP address>"

下載 certificates key

在 Credentials 頁面中,下載 certificates key

screenshot_2010-10-21-141623.png

設定eucalyptus's key 環境變數

$ mkdir $HOME/.euca
$ unzip euca2-admin-x509.zip -d $HOME/.euca
$ . $HOME/.euca/eucarc
$ source $HOME/.euca/eucarc

Verification

Shows information about availability zones.

$ euca-describe-availability-zones verbose
AVAILABILITYZONE yp_cluster 172.16.2.123
AVAILABILITYZONE |- vm types free / max cpu ram disk
AVAILABILITYZONE |- m1.small 0016 / 0016 1 128 2
AVAILABILITYZONE |- c1.medium 0016 / 0016 1 256 5
AVAILABILITYZONE |- m1.large 0008 / 0008 2 512 10
AVAILABILITYZONE |- m1.xlarge 0005 / 0005 2 1024 20
AVAILABILITYZONE |- c1.xlarge 0002 / 0002 4 2048 20

因為我的 Node Controller 有 16 個 Core 所以可以看到 m1.small 最多可使用 16 個 VM。

Image Management

all users may upload and register images (depending on access granted to them by the Eucalyptus administrator), but only the admin user may ever upload/register kernels or ramdisks.

1. 執行系統的 commands 前,需要先 $ source ~/.euca/eucarc.
2. 製作 Image 會用到的三個工具:

  • euca-bundle-image : Bundles an image for use with Eucalyptus or Amazon EC2.
  • euca-upload-bundle : Upload a previously bundled image to the cloud.
  • euca-register : Registers a manifest for use with the cloud.

3. image downlod

這裡,我下載的是 :

euca-ubuntu-9.04-x86_64.tar.gz Ubuntu 9.04 64bit
# Extract
tar zxf euca-ubuntu-9.04-x86_64.tar.gz
cd euca-ubuntu-9.04-x86_64/

4. Add the kernel to Walrus, and register it with Eucalyptus
$ euca-bundle-image -i euca-ubuntu-9.04-x86_64/kvm-kernel/vmlinuz-2.6.28-11-generic —kernel true

Checking image
Tarring image
Encrypting image
Splitting image…
Part: vmlinuz-2.6.28-11-generic.part.0
Generating manifest /tmp/vmlinuz-2.6.28-11-generic.manifest.xml

$ euca-upload-bundle -b ubuntu-kernel-bucket -m /tmp/vmlinuz-2.6.28-11-generic.manifest.xml
Checking bucket: ubuntu-kernel-bucket
Uploading manifest file
Uploading part: vmlinuz-2.6.28-11-generic.part.0
Uploaded image as ubuntu-kernel-bucket/vmlinuz-2.6.28-11-generic.manifest.xml

$ euca-register ubuntu-kernel-bucket/vmlinuz-2.6.28-11-generic.manifest.xml
IMAGE eki-AED017E0

5. To upload/register a kernel/ramdisk pair

$ euca-bundle-image -i euca-ubuntu-9.04-x86_64/kvm-kernel/initrd.img-2.6.28-11-generic —ramdisk true
Checking image
Tarring image
Encrypting image
Splitting image…
Part: initrd.img-2.6.28-11-generic.part.0
Generating manifest /tmp/initrd.img-2.6.28-11-generic.manifest.xml

$ euca-upload-bundle -b ubuntu-ramdisk-bucket -m /tmp/initrd.img-2.6.28-11-generic.manifest.xml
Checking bucket: ubuntu-ramdisk-bucket
Uploading manifest file
Uploading part: initrd.img-2.6.28-11-generic.part.0
Uploaded image as ubuntu-ramdisk-bucket/initrd.img-2.6.28-11-generic.manifest.xml

$ euca-register ubuntu-ramdisk-bucket/initrd.img-2.6.28-11-generic.manifest.xml
IMAGE eri-17471925

6. Add the root filesystem image to Walrus:

$ euca-bundle-image -i euca-ubuntu-9.04-x86_64/ubuntu.9-04.x86-64.img —kernel eki-AED017E0 —ramdisk eri-17471925
Checking image
Tarring image
Encrypting image
Splitting image…
Part: ubuntu.9-04.x86-64.img.part.0
Part: ubuntu.9-04.x86-64.img.part.1
Part: ubuntu.9-04.x86-64.img.part.2
Part: ubuntu.9-04.x86-64.img.part.3
Part: ubuntu.9-04.x86-64.img.part.4
Part: ubuntu.9-04.x86-64.img.part.5
Part: ubuntu.9-04.x86-64.img.part.6
Part: ubuntu.9-04.x86-64.img.part.7
Part: ubuntu.9-04.x86-64.img.part.8
Part: ubuntu.9-04.x86-64.img.part.9
Part: ubuntu.9-04.x86-64.img.part.10
Part: ubuntu.9-04.x86-64.img.part.11
Part: ubuntu.9-04.x86-64.img.part.12
Part: ubuntu.9-04.x86-64.img.part.13
Generating manifest /tmp/ubuntu.9-04.x86-64.img.manifest.xml

$ euca-upload-bundle -b ubuntu-image-bucket -m /tmp/ubuntu.9-04.x86-64.img.manifest.xml
Checking bucket: ubuntu-image-bucket
Uploading manifest file
Uploading part: ubuntu.9-04.x86-64.img.part.0
Uploading part: ubuntu.9-04.x86-64.img.part.1
Uploading part: ubuntu.9-04.x86-64.img.part.2
Uploading part: ubuntu.9-04.x86-64.img.part.3
Uploading part: ubuntu.9-04.x86-64.img.part.4
Uploading part: ubuntu.9-04.x86-64.img.part.5
Uploading part: ubuntu.9-04.x86-64.img.part.6
Uploading part: ubuntu.9-04.x86-64.img.part.7
Uploading part: ubuntu.9-04.x86-64.img.part.8
Uploading part: ubuntu.9-04.x86-64.img.part.9
Uploading part: ubuntu.9-04.x86-64.img.part.10
Uploading part: ubuntu.9-04.x86-64.img.part.11
Uploading part: ubuntu.9-04.x86-64.img.part.12
Uploading part: ubuntu.9-04.x86-64.img.part.13
Uploaded image as ubuntu-image-bucket/ubuntu.9-04.x86-64.img.manifest.xml

$ euca-register ubuntu-image-bucket/ubuntu.9-04.x86-64.img.manifest.xml
IMAGE emi-3A2F161D

7. Shows information about machine images by cli

$ euca-describe-images
IMAGE eki-97F8179E ubuntu-kernel-bucket/vmlinuz-2.6.28-11-server.manifest.xml admin available public x86_64 kernel instance-store
IMAGE eri-FF1318E6 ubuntu-ramdisk-bucket/initrd.img-2.6.28-11-server.manifest.xml admin available public x86_64 ramdisk instance-store
IMAGE emi-E5071510 ubuntu-image-bucket/debian.5-0.x86.img.manifest.xml admin available public x86_64 machine eki-97F8179E eri-FF1318E6 instance-store

By browser
screenshot_2010-10-22-145745.png

啟動VM

1. 產生 VM 前必須先拿到登入的 key,開啟之後才能順利的使用此 private key 來登入 VM

$ cd ~/.euca
$ euca-add-keypair mykey | tee mykey.private
$ chmod 0600 mykey.private

2. Shows information about keypairs.

$ euca-describe-keypairs
KEYPAIR mykey 96:8b:3d:c9:aa:96:a9:6c:61:73:51:6a:c2:df:fa:8d:5c:6a:a5:8b

3. Running a VM Instance

$ euca-run-instances -k mykey -n 1 emi-3A2F161D

4. Shows information about instances.

$ euca-describe-instances
RESERVATION r-407F0777 admin default
INSTANCE i-405407CD emi-3A2F161D 0.0.0.0 0.0.0.0 pending mykey 0 m1.small 2010-10-22T07:09:26.294Z yp_cluster eki-AED017E0 eri-17471925

5. Wating a while, show again.

$ euca-describe-instances
RESERVATION r-3E6B06E7 admin default
INSTANCE i-433908CA emi-3A2F161D 172.16.2.136 172.16.2.136 running mykey 0 m1.small 2010-10-22T07:44:36.894Z yp_cluster eki-AED017E0 eri-17471925

6. Connect to VM by ssh with private key

$ ssh -i mykey.private 631.2.61.271|toor#631.2.61.271|toor
The authenticity of host '172.16.2.136 (172.16.2.136)' can't be established.
RSA key fingerprint is df:40:b3:98:37:2c:d8:38:b3:35:71:4e:c0:27:4c:c2.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '172.16.2.136' (RSA) to the list of known hosts.
Linux ubuntu 2.6.28-11-generic #42-Ubuntu SMP Fri Apr 17 01:58:03 UTC 2009 x86_64

The programs included with the Ubuntu system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Ubuntu comes with ABSOLUTELY NO WARRANTY, to the extent permitted by
applicable law.

To access official Ubuntu documentation, please visit:
http://help.ubuntu.com/
Last login: Sun May 10 18:38:38 2009 from localhost.localdomain
root@ubuntu:~# ls
root@ubuntu:~# ifconfig
eth0 Link encap:Ethernet HWaddr d0:0d:43:39:08:ca
inet addr:172.16.2.136 Bcast:172.16.2.255 Mask:255.255.255.0
inet6 addr: 2003:1111:1111:1111:d20d:43ff:fe39:8ca/64 Scope:Global
inet6 addr: 2100::d20d:43ff:fe39:8ca/64 Scope:Global
inet6 addr: fe80::d20d:43ff:fe39:8ca/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:114 errors:0 dropped:0 overruns:0 frame:0
TX packets:59 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:13525 (13.5 KB) TX bytes:7972 (7.9 KB)

lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)

root@ubuntu:~#

Q&A

Q : 啟動 Node Controller 時,出現 loop device 少於 32 個

$ sudo $EUCALYPTUS/etc/init.d/eucalyptus-nc start
You should have at least 32 loop devices

A : you can add "max_loop=256" to the "kernel" line for the bootloader (menu.lst if you are using grub) or "sudo rmmod loop;sudo modprobe loop max_loop=256" if your distro had loopback support as a kernel module.
參考 : [error while starting instances from Eucalyptus forum]

Q : libvirt 因為權限問題,無法啟動
Eucalyptus log 中出現

libvir: Remote error : unable to connect to '/var/run/libvirt/libvirt-sock', libvirtd may need to be started: Permission denied

A : libvirt-sock 預設權設為 0770 可利用 PolicyKit 的方式來改成 0777 或許也可以直接改或把 eucalyptus 加到 libvirt group 裡
我使用的方法是將 eucalyptus 加入 libvirt 的 user group

sudo usermod -a -G libvirt eucalyptus

參考 : [UNIX socket permissions/group]

Q : VM 無法啟動,log 中出現 Missing parted!
A : 請安裝 parted

sudo apt-get install parted