Innhold
1 Initielt oppsett
Involverte maskiner:
- rhos-storage01.uio.no
- rhos-storage02.uio.no
- rhos-compute01.uio.no (Openstack compute)
Involverte nett:
| 129.240.148.192/26 | iaas-mgmt, vlan864 |
| 129.240.149.0/26 | iaas-admin, vlan846 |
| 129.240.149.64/26 | iaas-storage, vlan847 |
1.1 Installasjon
Notat
Dokumentasjon fra Red Hat: https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.1/html-single/Installation_Guide/index.html
Maskinene ble installert manuelt. Eneste custom-endring som ble gjort under installasjon var å redusere størrelsen på /home til 1024 MB.
Etter installasjon ble maskinene registrert i Red Hat Network (RHN):
rhn_register
Deretter ble følgende kanaler lagt til:
rhn-channel --add --channel=rhel-x86_64-server-6-rhs-2.1 rhn-channel --add --channel=rhel-x86_64-server-sfs-6.4.z
Da er følgende kanaler tilgjengelig:
# yum repolist [...] repo id repo name status rhel-x86_64-server-6-rhs-2.1 Red Hat Storage Server 2.1 (RHEL 6.4.z for x86_64) 192 rhel-x86_64-server-6.4.z RHEL EUS Server (v. 6.4.z for 64-bit x86_64) 11,184 rhel-x86_64-server-sfs-6.4.z RHEL EUS Server Scalable File System (v. 6.4.z for x86_64) 32 repolist: 11,408
Kjørte så inn oppdateringer:
yum -y update
La inn nøkler for jump-maskinene våre inn i ~/.ssh/auhorized_keys.
2 Konfigurasjon
Notat
Dokumentasjon fra Red Hat: https://access.redhat.com/site/documentation/en-US/Red_Hat_Storage/2.1/html-single/Administration_Guide/index.html
2.1 Legger til i Trusted Storage Pool
Kjører på rhos-storage01:
# gluster peer probe rhos-storage02-data.uio.no peer probe: success.
Sjekker status:
# gluster peer status Number of Peers: 1 Hostname: rhos-storage02-data.uio.no Uuid: f6c8dc1a-14a4-4bdf-bcd0-4ec98a608504 State: Peer in Cluster (Connected)
2.2 Formatterer bricks
Notat
Red Hat dok: Formatting and Mounting Bricks
Hver av nodene har et device /dev/sdb som skal bli en brick:
[root@rhos-storage01 ~]# lsblk /dev/sdb NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdb 8:16 0 558.4G 0 disk [root@rhos-storage02 ~]# lsblk /dev/sdb NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sdb 8:16 0 418.6G 0 disk
Lager volumgruppe og logisk volum på 200GB på begge noder:
# pvcreate /dev/sdb Physical volume "/dev/sdb" successfully created # vgcreate -s 512m vg_bricks /dev/sdb Volume group "vg_bricks" successfully created # lvcreate -L200g -n exp1 vg_bricks Logical volume "exp1" created
Formatterer med XFS:
# mkfs.xfs -i size=512 /dev/vg_bricks/exp1
meta-data=/dev/vg_bricks/exp1 isize=512 agcount=4, agsize=13107200 blks
= sectsz=512 attr=2, projid32bit=0
data = bsize=4096 blocks=52428800, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0
log =internal log bsize=4096 blocks=25600, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Finner UUID til device:
[root@rhos-storage01 ~]# blkid /dev/vg_bricks/exp1 /dev/vg_bricks/exp1: UUID="c7579092-f578-4494-8733-c31f5f6d4667" TYPE="xfs" [root@rhos-storage02 ~]# blkid /dev/vg_bricks/exp1 /dev/vg_bricks/exp1: UUID="d6bf52e1-f20b-44fc-a07d-3014ef48c90f" TYPE="xfs"
Lager entry i /etc/fstab med UUID (eksempel fra rhos-storage01):
UUID=c7579092-f578-4494-8733-c31f5f6d4667 /exp1 xfs default 1 2
Lager mountpoint:
mkdir /exp1
Monterer:
mount /exp1
Lager et katalog under /exp1:
mkdir /exp1/bricksrv
Det er denne som vil bli brukt som en brick. For forklaring, se Red Hat dok.
2.3 Lage gluster-volum
Notat
Velger å bruke Distributed Replicated Volume. Se Red Hat dokumentasjon: Creating Distributed Replicated Volumes
Lager volum:
# gluster volume create volume01 replica 2 transport tcp \
rhos-storage01-data:/exp1/bricksrv \
rhos-storage02-data:/exp1/bricksrv
volume create: volume01: success: please start the volume to access data
Starter:
# gluster volume start volume01 volume start: volume01: success
Sjekker status:
# gluster volume status volume01 Status of volume: volume01 Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick rhos-storage01-data:/exp1/bricksrv 49152 Y 1323 Brick rhos-storage02-data:/exp1/bricksrv 49152 Y 701 NFS Server on localhost 2049 Y 1335 Self-heal Daemon on localhost N/A Y 1342 NFS Server on rhos-storage02-data.uio.no 2049 Y 715 Self-heal Daemon on rhos-storage02-data.uio.no N/A Y 720 Task Status of Volume volume01 ------------------------------------------------------------------------------ There are no active volume tasks
2.4 Sette ACL
La rhos-compute01-data.uio.no og tux.uio.no få tilgang:
# gluster volume set volume01 auth.allow 129.240.149.102,129.240.6.13 volume set: success
Sjekke:
# gluster volume info volume01 Volume Name: volume01 Type: Replicate Volume ID: 74925a74-4f22-45ce-9f38-16369e1e0259 Status: Started Number of Bricks: 1 x 2 = 2 Transport-type: tcp Bricks: Brick1: rhos-storage01-data:/exp1/bricksrv Brick2: rhos-storage02-data:/exp1/bricksrv Options Reconfigured: auth.reject: none auth.allow: 129.240.149.102,129.240.6.13
2.5 Oppsett av klient
Notat
Dette gjøres på rhos-compute01 og rhos-controller.
Installere riktig RHN-kanal:
rhn-channel --add --channel=rhel-x86_64-server-rhsclient-6
Oppgradere:
yum -y update
Installere pakker:
yum -y install glusterfs glusterfs-fuse
Teste montering:
# mkdir /glusterfs # mount -t glusterfs rhos-storage01-data:/volume01 /glusterfs # mount | grep gluster rhos-storage01-data:/volume01 on /glusterfs type fuse.glusterfs (rw,default_permissions,allow_other,max_read=131072) # df -h /glusterfs Filesystem Size Used Avail Use% Mounted on rhos-storage01-data:/volume01 200G 33M 200G 1% /glusterfs
Avmontere og fjerne mountpoint:
umount /glusterfs rmdir /glusterfs
2.6 Fjerne volum
Test-volumet fjernes.
Status:
# gluster volume status volume01 Status of volume: volume01 Gluster process Port Online Pid ------------------------------------------------------------------------------ Brick rhos-storage01-data:/exp1/bricksrv 49152 Y 1323 Brick rhos-storage02-data:/exp1/bricksrv 49152 Y 701 NFS Server on localhost 2049 Y 1335 Self-heal Daemon on localhost N/A Y 1342 NFS Server on rhos-storage02-data.uio.no 2049 Y 715 Self-heal Daemon on rhos-storage02-data.uio.no N/A Y 720 Task Status of Volume volume01 ------------------------------------------------------------------------------ There are no active volume tasks
Stoppe:
# gluster volume stop volume01 Stopping volume will make its data inaccessible. Do you want to continue? (y/n) y volume stop: volume01: success
Fjerne:
# gluster volume delete volume01 Deleting volume will erase all information about the volume. Do you want to continue? (y/n) y volume delete: volume01: success