Ny compute-node for RH Openstack 4 (Havana) på RHEL6

1.1   Nettverk

Konfigurerte API-nett og storage-nett:

# cat ifcfg-em2
DEVICE="em2"
BOOTPROTO="static"
IPADDR="129.240.149.231"
NETMASK="255.255.255.192"
IPV6INIT="yes"
HWADDR="B8:AC:6F:97:1A:6C"
NM_CONTROLLED="yes"
ONBOOT="yes"
TYPE="Ethernet"
UUID="98b9845b-4231-4b9c-a42f-0324dde83c31"

# cat ifcfg-em3
DEVICE="em3"
BOOTPROTO="static"
IPADDR="129.240.149.104"
NETMASK="255.255.255.192"
IPV6INIT="yes"
HWADDR="B8:AC:6F:97:1A:6E"
NM_CONTROLLED="yes"
ONBOOT="yes"
TYPE="Ethernet"
UUID="6d8b3724-41ac-490b-b7de-09ecd5e3f139"

1.2   Installasjon av software requirements

Installerte yum priorities-plugin:

yum install -y yum-plugin-priorities

Legger til openstack-repo rhel-x86_64-server-6-ost-4 og RHSS-repo rhel-x86_64-server-rhsclient-6:

rhn-channel -a -c rhel-x86_64-server-6-ost-4 \
    -c rhel-x86_64-server-rhsclient-6

Legger inn i /etc/yum/pluginconf.d/rhnplugin.conf:

[rhel-x86_64-server-6-ost-4]
priority = 1

Oppdaterer:

yum -y update

Rebootet:

reboot

1.3   Installere pakker

Følgende installeres:

yum -y install openstack-nova-common \
   openstack-nova-compute \
   openstack-ceilometer-compute \
   openstack-ceilometer-common \
   openstack-utils \
   openstack-neutron \
   openvswitch \
   openstack-neutron-openvswitch \
   python-cinderclient \
   glusterfs-api \
   glusterfs-libs \
   glusterfs-fuse \
   glusterfs \
   openstack-nova-console

1.4   Konfigurere Open vSwitch

1. Starte tjenesten og sørge for at den kjører ved boot:

   service openvswitch start
   chkconfig openvswitch on
   

2. Lager intern bridge:

   ovs-vsctl add-br br-int
   

3. Lager /etc/sysconfig/network-scripts/ifcfg-br-int med innhold:

   DEVICE=br-int
   DEVICETYPE=ovs
   TYPE=OVSBridge
   ONBOOT=yes
   BOOTPROTO=none
   

4. Starte agenten og sørge for at den kjører ved boot:

   service neutron-openvswitch-agent start
   chkconfig neutron-openvswitch-agent on
   

5. Sørge for at ovs-cleanup kjører ved boot:

   chkconfig neutron-ovs-cleanup on
   

1.5   Konfigurere autentisering

1. Sette auth_strategy:

   openstack-config --set /etc/nova/nova.conf \
      DEFAULT auth_strategy keystone
   

2. Sette authentication host:

   openstack-config --set /etc/nova/api-paste.ini \
      filter:authtoken auth_host rhos-controller-api.uio.no
   

3. Sette admin_tenant_name:

   openstack-config --set /etc/nova/api-paste.ini \
      filter:authtoken admin_tenant_name services
   

4. Sette admin_user:

   openstack-config --set /etc/nova/api-paste.ini \
      filter:authtoken admin_user nova
   

5. Sette passord:

   openstack-config --set /etc/nova/api-paste.ini \
      filter:authtoken admin_password $NOVA_PASS
   

1.6   Konfigurere database

Sette database connection:

openstack-config --set /etc/nova/nova.conf \
   DEFAULT sql_connection \
   mysql://nova:${NOVA_DBPASS}@rhos-controller-api.uio.no/nova

1.7   Konfigurere message broker (QPID)

Sette rpc_backend:

openstack-config --set /etc/nova/nova.conf \
   DEFAULT rpc_backend nova.openstack.common.rpc.impl_qpid

Sette hostname:

openstack-config --set /etc/nova/nova.conf \
   DEFAULT qpid_hostname rhos-controller-api.uio.no

1.8   Konfigurere nettverk

1. Sette network api til "neutron":

   openstack-config --set /etc/nova/nova.conf \
      DEFAULT network_api_class nova.network.neutronv2.api.API
   

2. Sette URL for neutron API:

   openstack-config --set /etc/nova/nova.conf \
      DEFAULT neutron_url http://rhos-network-api.uio.no:9696/
   

3. Sette neutron_admin_tenant_name:

   openstack-config --set /etc/nova/nova.conf \
      DEFAULT neutron_admin_tenant_name services
   

4. Sette neutron_admin_username:

   openstack-config --set /etc/nova/nova.conf \
      DEFAULT neutron_admin_username neutron
   

5. Sette neutron_admin_password:

   openstack-config --set /etc/nova/nova.conf \
      DEFAULT neutron_admin_password $NEUTRON_PASS
   

6. Sette neutron_admin_auth_url:

   openstack-config --set /etc/nova/nova.conf \
     DEFAULT neutron_admin_auth_url http://rhos-controller-api.uio.no:35357/v2.0
   

7. Sette security_group_api:

   openstack-config --set /etc/nova/nova.conf \
     DEFAULT security_group_api neutron
   

8. Sette firewall_driver:

   openstack-config --set /etc/nova/nova.conf \
     DEFAULT firewall_driver nova.virt.firewall.NoopFirewallDriver
   

1.9   Konfigurere QPID til neutron på compute-node

1. Sette RPC backend til QPID:

   openstack-config --set /etc/neutron/neutron.conf \
      DEFAULT rpc_backend neutron.openstack.common.rpc.impl_qpid
   

2. Sette QPID hostname:

   openstack-config --set /etc/neutron/neutron.conf \
      DEFAULT qpid_hostname rhos-controller-api.uio.no
   

1.10   Konfigurere Virtual Interface Plugging

Konfigurere til å bruke nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver, siden vi bruke Open vSwitch:

openstack-config --set /etc/nova/nova.conf \
   DEFAULT libvirt_vif_driver \
   nova.virt.libvirt.vif.LibvirtHybridOVSBridgeDriver

1.11   Konfigurere firewall

Fikse iptables:

iptables -A INPUT -p tcp -m multiport --dports 5900:5999 -j ACCEPT
iptables-save > /etc/sysconfig/iptables

1.12   Starte tjenestene

Starte og sørge for at kjører ved boot:

service messagebus start
chkconfig messagebus on
service libvirtd start
chkconfig libvirtd on
service openstack-nova-compute start
chkconfig openstack-nova-compute on

1.13   Konfigurere Glance

Sette glance_host:

openstack-config --set /etc/nova/nova.conf DEFAULT \
   glance_host rhos-controller-api.uio.no

1.14   Konfigurere plugin (Open vSwitch)

1. Lage symlink:

   ln -s /etc/neutron/plugins/openvswitch/ovs_neutron_plugin.ini \
       /etc/neutron/plugin.ini
   

2. Sette på tunneling, sette type GRE og tunnel id range:

   openstack-config --set /etc/neutron/plugin.ini \
     OVS enable_tunneling True
   openstack-config --set /etc/neutron/plugin.ini \
     OVS tenant_network_type gre
   openstack-config --set /etc/neutron/plugin.ini \
     OVS tunnel_id_ranges "1:1000"
   

3. Sette hvilke bridger som skal brukes til tunnellering:

   openstack-config --set /etc/neutron/plugin.ini \
     OVS integration_bridge br-int
   openstack-config --set /etc/neutron/plugin.ini \
     OVS tunnel_bridge br-tun
   

4. Lokal IP/nett som skal brukes for GRE ut (em1 på hver host):

   openstack-config --set /etc/neutron/plugin.ini \
     OVS local_ip $(ip addr list em1 | awk '/inet / {print $2}' | cut -d/ -f1)
   

5. Sette core_plugin til OVS:

   openstack-config --set /etc/neutron/neutron.conf \
     DEFAULT core_plugin \
     neutron.plugins.openvswitch.ovs_neutron_plugin.OVSNeutronPluginV2
   

6. Sette firewall driver:

   openstack-config --set /etc/neutron/plugin.ini \
      securitygroup firewall_driver \
      neutron.agent.linux.iptables_firewall.OVSHybridIptablesFirewallDriver
   

1.15   Konfigurere lagring (gluster)

1. Editere /etc/nova/nova.conf, sette:

   openstack-config --set /etc/nova/nova.conf \
      DEFAULT instances_path /gluster/nova/instances
   

2. Lage mountpoint:

   mkdir -p /gluster/nova/instances
   

3. Montere:

   mount -t glusterfs rhos-storage01-data:/nova-vol /gluster/nova/instances
   

4. Endre eierskap (BUG i dok.: må gjøres etter montering):

   chown -R nova:nova /gluster/nova/
   

5. Legge til i fstab:

   rhos-storage01-data:/nova-vol /gluster/nova/instances glusterfs defaults,_netdev 0 0
   

6. Restarte nova-tjenesten:

   service openstack-nova-compute restart
   

1.16   Til slutt

Reboote:

reboot

2.1   Firewall

Fikse iptables:

iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 6080 -j ACCEPT
iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 6081 -j ACCEPT
iptables-save > /etc/sysconfig/iptables

2.2   Konfigurasjon

Sette verdier i /etc/nova/nova.conf:

openstack-config --set /etc/nova/nova.conf \
   DEFAULT vncserver_listen 0.0.0.0
openstack-config --set /etc/nova/nova.conf \
   DEFAULT vncserver_proxyclient_address \
   $(ip addr list em1 | awk '/inet / {print $2}' | cut -d/ -f1)
openstack-config --set /etc/nova/nova.conf \
   DEFAULT novncproxy_base_url http://rhos-controller.uio.no:6080/vnc_auto.html

Editere /etc/libvirt/qemu.conf:

vnc_listen = "0.0.0.0"

Deretter restart tjenester:

service libvirtd restart
service openstack-nova-compute restart

3.1   Firewall

Fikse iptables:

iptables -A INPUT -m state --state NEW -m tcp -p tcp --dport 16509 -j ACCEPT
iptables -A INPUT -p tcp -m multiport --dports 49152:49261 -j ACCEPT
iptables-save > /etc/sysconfig/iptables

3.2   Konfigurere libvirt og qemu

Editere /etc/libvirt/libvirtd.conf:

listen_tls = 0
listen_tcp = 1
auth_tcp = "none"

Editere /etc/sysconfig/libvirtd:

LIBVIRTD_ARGS="--listen"

Restarte libvirtd:

service libvirtd restart

3.3   Konfigurere SELinux

Gjøres på rhos-compute01:

setsebool -P virt_use_fusefs on

3.4   SSH for nova

nova før:

# getent passwd nova
nova:x:162:162:OpenStack Nova Daemons:/var/lib/nova:/sbin/nologin

Sette nytt shell:

usermod -s /bin/bash nova

Etterpå:

# getent passwd nova
nova:x:162:162:OpenStack Nova Daemons:/var/lib/nova:/bin/bash

Lage ssh-nøkkel for nova:

# su - nova
$ ssh-keygen

Offentlig nøkkel for alle compute-noder legges inn i authorized_keys på alle compute-noder. FIXME: Bruke samme nøkkel overalt?

SELinux må endres for å tillate sshd å lese på nova-brukeren sitt hjemmeområde:

# cat /var/log/audit/audit.log | audit2allow -m foo

module foo 1.0;

require {
      type nova_var_lib_t;
      type sshd_t;
      class dir { search getattr };
      class file { read getattr open };
}

#============= sshd_t ==============
allow sshd_t nova_var_lib_t:dir { search getattr };
allow sshd_t nova_var_lib_t:file { read getattr open };

Laget en modul:

# cat /var/log/audit/audit.log | audit2allow -M nova_sshd
******************** IMPORTANT ***********************
To make this policy package active, execute:

semodule -i nova_sshd.pp

Lastet den inn:

semodule -i nova_sshd.pp