가상 환경 서버 생성
노트북 2ea를 이용한 VM Node 6ea를 생성하는 예시입니다. HostOS는 Ubuntu 22.04(Live Image)이고, VM 으로 구성되는 GuestOS는 Ubuntu22.04(cloud Image)입니다. External과 Internal을 구분하기 위해 2개의 Lan을 Bridge화 하여 각각 libvirt 의 br0, br1의 브릿지로 구성하였습니다.
노트북의 성능은 각각 8코어 32GB 메모리에 512GB 스토리지 그리고 1기는 1TB의 스토리지가 추가로 구성되어 있습니다.
각 노트북에 Ubuntu 22.04를 설치 한 후, libvirt를 설치하고 Bridge 설정을 합니다.
### libvirt 설치
apt install qemu-kvm libvirt-daemon-system virtinst
클라우드 이미지 경로와 VM 서버의 기본 경로를 생성합니다.
### 경로 생성
mkdir -p /VM/playcecloud /VM/image
NIC Bridge 설정
### NIC Name 확인
ip a
## 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
## link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
## inet 127.0.0.1/8 scope host lo
## valid_lft forever preferred_lft forever
## inet6 ::1/128 scope host
## valid_lft forever preferred_lft forever
## 2: wlp1s0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
## link/ether b0:7d:64:f2:02:97 brd ff:ff:ff:ff:ff:ff
## 3: enx00e04c68318f: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond-svc state UP group default qlen 1000
## link/ether ea:80:d8:a3:bd:c1 brd ff:ff:ff:ff:ff:ff permaddr 00:e0:4c:68:31:8f
## 4: enx88366cf74ee8: <BROADCAST,MULTICAST,SLAVE,UP,LOWER_UP> mtu 1500 qdisc fq_codel master bond-dat state UP group default qlen 1000
## link/ether e2:f5:21:a9:e3:3d brd ff:ff:ff:ff:ff:ff permaddr 88:36:6c:f7:4e:e8
# NIC은 구성에 따라 다름 ( 여기서는 enx00e04c68318f, enx88366cf74ee8 사용 )
### Netplan 수정
cd /etc/netplan
tee > 02-network-manager-wired.yaml << EOF
# NIC Solt 0 Port 0 == enx00e04c68318f == Ext
# NIC Solt 0 Port 1 ==
# NIC Solt 1 Port 0 == enx88366cf74ee8 == Int
# /etc/netplan/99.nic_bind_config.yaml
# This is the network config written by 'subiquity'
network:
renderer: networkd
version: 2
ethernets:
enx88366cf74ee8: {}
bonds:
bond-dat:
interfaces: [enx88366cf74ee8]
parameters:
mode: balance-rr
mii-monitor-interval: 1
vlans:
bond-svc.300:
id: 300
link: bond-dat
bridges:
br2:
interfaces: [bond-svc.300]
addresses: [10.3.0.100/16]
br3:
interfaces: [bond-dat]
addresses: [10.0.0.100/24]
EOF
tee > 03-network-manager-wired.yaml << EOF
# NIC Solt 0 Port 0 == enx00e04c68318f == Ext
# NIC Solt 0 Port 1 ==
# NIC Solt 1 Port 0 == enx88366cf74ee8 == Int
# /etc/netplan/99.nic_bind_config.yaml
# This is the network config written by 'subiquity'
network:
renderer: networkd
version: 2
ethernets:
enx00e04c68318f: {}
bonds:
bond-svc:
interfaces: [enx00e04c68318f]
parameters:
mode: balance-rr
mii-monitor-interval: 1
vlans:
bond-svc.200:
id: 200
link: bond-svc
bridges:
br0:
interfaces: [bond-svc]
addresses: [10.1.0.100/16]
routes:
- to: 0.0.0.0/0
via: 10.1.1.254
nameservers:
addresses: [8.8.8.8]
br1:
interfaces: [bond-svc.200]
addresses: [10.2.0.100/16]
EOF
# 간혹 띄어쓰기가 잘 못 들어가는 경우가 있음 언제나 확인 후 적용
# VLan 테스트를 위해 VLan 추가 되어 있음
### Netplan 적용
netplan generate
netplan apply
### IP Address 확인
ip -4 -o a
## 1: lo inet 127.0.0.1/8 scope host lo\ valid_lft forever preferred_lft forever
## 6: br0 inet 10.1.0.100/16 brd 10.1.255.255 scope global br0\ valid_lft forever preferred_lft forever
## 8: br1 inet 10.2.0.100/16 brd 10.2.255.255 scope global br1\ valid_lft forever preferred_lft forever
## 11: br3 inet 10.0.0.100/24 brd 10.0.0.255 scope global br3\ valid_lft forever preferred_lft forever
## 12: br2 inet 10.3.0.100/16 brd 10.3.255.255 scope global br2\ valid_lft forever preferred_lft forever
### 통신 확인
ip r
## default via 10.1.1.254 dev br0 proto static
## 10.0.0.0/24 dev br3 proto kernel scope link src 10.0.0.100
## 10.1.0.0/16 dev br0 proto kernel scope link src 10.1.0.100
## 10.2.0.0/16 dev br1 proto kernel scope link src 10.2.0.100
## 10.3.0.0/16 dev br2 proto kernel scope link src 10.3.0.100
ping 8.8.8.8 -c 10
ping 10.1.1.254 -c 10
ping 10.1.0.100 -c 1
ping 10.1.0.200 -c 10
ping 10.0.0.100 -c 1
ping 10.0.0.200 -c 10
Virsh Network 설정
### Virsh Network 를 Bridge 로 설정
# libvrit 기본 NIC 삭제( NAT )
virsh net-destroy default
virsh net-undefine default
virsh net-list
## Name State Autostart Persistent
## -------------------------------------------
# bridge NIC 생성
cd /VM/script
tee > virbr0.xml << EOF
<network>
<name>br0</name>
<forward mode='bridge'/>
<bridge name='br0'/>
</network>
EOF
tee > virbr3.xml << EOF
<network>
<name>br3</name>
<forward mode='bridge'/>
<bridge name='br3'/>
</network>
EOF
# virsh network 생성
virsh net-define virbr0.xml
virsh net-autostart br0
virsh net-start br0
virsh net-define virbr3.xml
virsh net-autostart br3
virsh net-start br3
# virsh network 확인
virsh net-list
## Name State Autostart Persistent
## -------------------------------------------
## br0 active yes yes
## br3 active yes yes
계획대로 1기의 랩탑엔 2개의 작은 VM, 1개의 큰 VM을 구성합니다. 선행으로 deploy, control-plane01, worker01을 만들겠습니다.
클라우드 이미지를 다운로드 한후, 이미지를 이용하여 VM 을 생성합니다. 이때 NIC의 지정과 디스크 크기를 설정합니다.
deploy 01
# 이미지 보관 장소 이동
cd /VM/image
# 이미지 획득 경로 http://ubuntu.com
# server 이미지
wget http://cloud-images.ubuntu.com/jammy/current/jammy-server-cloudimg-amd64.img
# image info
qemu-img info jammy-server-cloudimg-amd64.img
## image: jammy-server-cloudimg-amd64.img
## file format: qcow2
## virtual size: 2.2 GiB (2361393152 bytes)
## disk size: 566 MiB
## cluster_size: 65536
## Format specific information:
## compat: 0.10
## refcount bits: 16
# 이미지 생성
qemu-img create -f qcow2 -F qcow2 -b /VM/image/jammy-server-cloudimg-amd64.img /VM/playcecloud/TestVMdeploy01.qcow2
## Formatting '/VM/playcecloud/TestVMdeploy01.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2361393152 backing_file=/VM/image/jammy-server-cloudimg-amd64.img backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
# 생성 확인
qemu-img info /VM/playcecloud/TestVMdeploy01.qcow2
## image: /VM/playcecloud/TestVMdeploy01.qcow2
## file format: qcow2
## virtual size: 2.2 GiB (2361393152 bytes)
## disk size: 196 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# 가상 볼륨 크기 변경
qemu-img resize /VM/playcecloud/TestVMdeploy01.qcow2 100G
## Image resized.
# 볼륨 변경 확인
qemu-img info /VM/playcecloud/TestVMdeploy01.qcow2
## image: /VM/playcecloud/TestVMdeploy01.qcow2
## file format: qcow2
## virtual size: 100 GiB (107374182400 bytes)
## disk size: 200 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# hostname 설정 파일 생성
mkdir /VM/script
cat > /VM/script/meta-data << EOF
local-hostname: TestVMdeploy01
EOF
# key file 지정 ( 계정이 cloud 일때 )
export account="cloud"
export PUB_KEY=$(cat /home/${account}/.ssh/authorized_keys)
# user 설정 파일 생성
cat > /VM/script/user-data << EOF
#cloud-config
password: password
users:
- default
- name: ${account}
ssh-authorized-keys:
- '$PUB_KEY'
sudo: ['ALL=(ALL) NOPASSWD:ALL']
groups: sudo
shell: /bin/bash
runcmd:
- cat /home/kwlee/.ssh/authorized_keys > /root/.ssh/authorized_keys
EOF
# OS 이미지 생성
genisoimage -output /VM/playcecloud/TestVMdeploy01.iso -volid cidata -joliet -rock /VM/script/user-data /VM/script/meta-data
# 기준 서버 생성
virt-install --connect qemu:///system --virt-type kvm --name deploy --ram 6124 --vcpus=2 --os-type linux --os-variant ubuntu22.04 --disk path=/VM/playcecloud/TestVMdeploy01.qcow2,format=qcow2 --disk /VM/playcecloud/TestVMdeploy01.iso,device=cdrom --import --network bridge=br0,model=virtio --network bridge=br3 --noautoconsole
Control-Plane 01
# 이미지 보관 장소 이동
cd /VM/image
# 이미지 생성
qemu-img create -f qcow2 -F qcow2 -b /VM/image/jammy-server-cloudimg-amd64.img /VM/playcecloud/TestVMctr01.qcow2
## Formatting '/VM/playcecloud/TestVMctr01.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2361393152 backing_file=/VM/image/jammy-server-cloudimg-amd64.img backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
# 생성 확인
qemu-img info /VM/playcecloud/TestVMctr01.qcow2
## image: /VM/playcecloud/TestVMctr01.qcow2
## file format: qcow2
## virtual size: 2.2 GiB (2361393152 bytes)
## disk size: 196 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# 가상 볼륨 크기 변경
qemu-img resize /VM/playcecloud/TestVMctr01.qcow2 100G
## Image resized.
# 볼륨 변경 확인
qemu-img info /VM/playcecloud/TestVMctr01.qcow2
## image: /VM/playcecloud/TestVMctr01.qcow2
## file format: qcow2
## virtual size: 100 GiB (107374182400 bytes)
## disk size: 200 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# hostname 설정 파일 생성
cat > /VM/script/meta-data << EOF
local-hostname: TestVMctr01
EOF
# key file 지정 ( 계정이 cloud 일때 )
export account="cloud"
export PUB_KEY=$(cat /home/${account}/.ssh/authorized_keys)
# user 설정 파일 생성
cat > /VM/script/user-data << EOF
#cloud-config
password: password
users:
- default
- name: ${account}
ssh-authorized-keys:
- '$PUB_KEY'
sudo: ['ALL=(ALL) NOPASSWD:ALL']
groups: sudo
shell: /bin/bash
runcmd:
- cat /home/kwlee/.ssh/authorized_keys > /root/.ssh/authorized_keys
EOF
# OS 이미지 생성
genisoimage -output /VM/playcecloud/TestVMctr01.iso -volid cidata -joliet -rock /VM/script/user-data /VM/script/meta-data
# 기준 서버 생성
virt-install --connect qemu:///system --virt-type kvm --name controlplane01 --ram 6124 --vcpus=2 --os-type linux --os-variant ubuntu22.04 --disk path=/VM/playcecloud/TestVMctr01.qcow2,format=qcow2 --disk /VM/playcecloud/TestVMctr01.iso,device=cdrom --import --network bridge=br0,model=virtio --network bridge=br3 --noautoconsole
Worker 01
# 이미지 보관 장소 이동
cd /VM/image
# 이미지 생성
qemu-img create -f qcow2 -F qcow2 -b /VM/image/jammy-server-cloudimg-amd64.img /VM/playcecloud/TestVMwrk01.qcow2
## Formatting '/VM/playcecloud/TestVMwrk01.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2361393152 backing_file=/VM/image/jammy-server-cloudimg-amd64.img backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
# 생성 확인
qemu-img info /VM/playcecloud/TestVMwrk01.qcow2
## image: /VM/playcecloud/TestVMwrk01.qcow2
## file format: qcow2
## virtual size: 2.2 GiB (2361393152 bytes)
## disk size: 196 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# 가상 볼륨 크기 변경
qemu-img resize /VM/playcecloud/TestVMwrk01.qcow2 100G
## Image resized.
# 볼륨 변경 확인
qemu-img info /VM/playcecloud/TestVMwrk01.qcow2
## image: /VM/playcecloud/TestVMwrk01.qcow2
## file format: qcow2
## virtual size: 100 GiB (107374182400 bytes)
## disk size: 200 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# hostname 설정 파일 생성
cat > /VM/script/meta-data << EOF
local-hostname: TestVMwrk01
EOF
# key file 지정 ( 계정이 cloud 일때 )
export account="cloud"
export PUB_KEY=$(cat /home/${account}/.ssh/authorized_keys)
# user 설정 파일 생성
cat > /VM/script/user-data << EOF
#cloud-config
password: password
users:
- default
- name: ${account}
ssh-authorized-keys:
- '$PUB_KEY'
sudo: ['ALL=(ALL) NOPASSWD:ALL']
groups: sudo
shell: /bin/bash
runcmd:
- cat /home/kwlee/.ssh/authorized_keys > /root/.ssh/authorized_keys
EOF
# OS 이미지 생성
genisoimage -output /VM/playcecloud/TestVMwrk01.iso -volid cidata -joliet -rock /VM/script/user-data /VM/script/meta-data
# 기준 서버 생성
virt-install --connect qemu:///system --virt-type kvm --name worker01 --ram 16384 --vcpus=4 --os-type linux --os-variant ubuntu22.04 --disk path=/VM/playcecloud/TestVMwrk01.qcow2,format=qcow2 --disk /VM/playcecloud/TestVMwrk01.iso,device=cdrom --import --network bridge=br0,model=virtio --network bridge=br3 --noautoconsole
구성 확인
# 가상 서버 구성 확인
virsh list
## Id Name State
## --------------------------------
## 4 controlplane01 running
## 5 deploy running
## 6 worker01 running
# 서버 접속 확인
virsh console ${Virsh_NAME} # Password 사용시
ssh ${account}@${IPADDRESS} -i ~/.ssh/id_rsa # ssh-key 사용시
###############################################################################################
# 만약 서버 IP Address를 찾을 수 없거나, 패스워드를 모를경우
# 해당 서버 정지
virsh shutdown ${SERVER_NAME}
## Domain ${SERVER_NAME} is being shutdown
# root 패스워드 설정
apt install guestfs-tools
export PASS=`echo password`
virt-customize -a /VM/playcecloud/${SERVER_QCOW_NAME}.qcow2 --root-password password:$PASS --uninstall cloud-init
## [ 0.0] Examining the guest ...
## [ 13.7] Setting a random seed
## [ 13.9] Uninstalling packages: cloud-init
## [ 16.8] Setting passwords
## [ 23.6] Finishing off
# 기준 서버 실행
virsh start ${SERVER_NAME}
## Domain ${SERVER_NAME} started
2번째 랩탑입니다.
Control-Plane 02
# 이미지 보관 장소 이동
cd /VM/image
# 이미지 생성
qemu-img create -f qcow2 -F qcow2 -b /VM/image/jammy-server-cloudimg-amd64.img /VM/playcecloud/TestVMctr02.qcow2
## Formatting '/VM/playcecloud/TestVMctr02.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2361393152 backing_file=/VM/image/jammy-server-cloudimg-amd64.img backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
# 생성 확인
qemu-img info /VM/playcecloud/TestVMctr02.qcow2
## image: /VM/playcecloud/TestVMctr02.qcow2
## file format: qcow2
## virtual size: 2.2 GiB (2361393152 bytes)
## disk size: 196 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# 가상 볼륨 크기 변경
qemu-img resize /VM/playcecloud/TestVMctr02.qcow2 100G
## Image resized.
# 볼륨 변경 확인
qemu-img info /VM/playcecloud/TestVMctr02.qcow2
## image: /VM/playcecloud/TestVMctr02.qcow2
## file format: qcow2
## virtual size: 100 GiB (107374182400 bytes)
## disk size: 200 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# hostname 설정 파일 생성
mkdir /VM/script
cat > /VM/script/meta-data << EOF
local-hostname: TestVMctr02
EOF
# key file 지정 ( 계정이 cloud 일때 )
export account="cloud"
export PUB_KEY=$(cat /home/${account}/.ssh/authorized_keys)
# user 설정 파일 생성
cat > /VM/script/user-data << EOF
#cloud-config
password: password
users:
- default
- name: ${account}
ssh-authorized-keys:
- '$PUB_KEY'
sudo: ['ALL=(ALL) NOPASSWD:ALL']
groups: sudo
shell: /bin/bash
runcmd:
- cat /home/kwlee/.ssh/authorized_keys > /root/.ssh/authorized_keys
EOF
# OS 이미지 생성
genisoimage -output /VM/playcecloud/TestVMctr02.iso -volid cidata -joliet -rock /VM/script/user-data /VM/script/meta-data
# 기준 서버 생성
virt-install --connect qemu:///system --virt-type kvm --name controlplane02 --ram 6124 --vcpus=2 --os-type linux --os-variant ubuntu22.04 --disk path=/VM/playcecloud/TestVMctr02.qcow2,format=qcow2 --disk /VM/playcecloud/TestVMctr02.iso,device=cdrom --import --network bridge=br0,model=virtio --network bridge=br3 --noautoconsole
Control-Plane 03
# 이미지 보관 장소 이동
cd /VM/image
# 이미지 생성
qemu-img create -f qcow2 -F qcow2 -b /VM/image/jammy-server-cloudimg-amd64.img /VM/playcecloud/TestVMctr03.qcow2
## Formatting '/VM/playcecloud/TestVMctr03.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2361393152 backing_file=/VM/image/jammy-server-cloudimg-amd64.img backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
# 생성 확인
qemu-img info /VM/playcecloud/TestVMctr03.qcow2
## image: /VM/playcecloud/TestVMctr03.qcow2
## file format: qcow2
## virtual size: 2.2 GiB (2361393152 bytes)
## disk size: 196 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# 가상 볼륨 크기 변경
qemu-img resize /VM/playcecloud/TestVMctr03.qcow2 100G
## Image resized.
# 볼륨 변경 확인
qemu-img info /VM/playcecloud/TestVMctr03.qcow2
## image: /VM/playcecloud/TestVMctr03.qcow2
## file format: qcow2
## virtual size: 100 GiB (107374182400 bytes)
## disk size: 200 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# hostname 설정 파일 생성
mkdir /VM/script
cat > /VM/script/meta-data << EOF
local-hostname: TestVMctr03
EOF
# key file 지정 ( 계정이 cloud 일때 )
export account="cloud"
export PUB_KEY=$(cat /home/${account}/.ssh/authorized_keys)
# user 설정 파일 생성
cat > /VM/script/user-data << EOF
#cloud-config
password: password
users:
- default
- name: ${account}
ssh-authorized-keys:
- '$PUB_KEY'
sudo: ['ALL=(ALL) NOPASSWD:ALL']
groups: sudo
shell: /bin/bash
runcmd:
- cat /home/kwlee/.ssh/authorized_keys > /root/.ssh/authorized_keys
EOF
# OS 이미지 생성
genisoimage -output /VM/playcecloud/TestVMctr03.iso -volid cidata -joliet -rock /VM/script/user-data /VM/script/meta-data
# 기준 서버 생성
virt-install --connect qemu:///system --virt-type kvm --name controlplane03 --ram 6124 --vcpus=2 --os-type linux --os-variant ubuntu22.04 --disk path=/VM/playcecloud/TestVMctr03.qcow2,format=qcow2 --disk /VM/playcecloud/TestVMctr03.iso,device=cdrom --import --network bridge=br0,model=virtio --network bridge=br3 --noautoconsole
Worker 02
# 이미지 보관 장소 이동
cd /VM/image
# 이미지 생성
qemu-img create -f qcow2 -F qcow2 -b /VM/image/jammy-server-cloudimg-amd64.img /VM/playcecloud/TestVMwrk02.qcow2
## Formatting '/VM/playcecloud/TestVMwrk02.qcow2', fmt=qcow2 cluster_size=65536 extended_l2=off compression_type=zlib size=2361393152 backing_file=/VM/image/jammy-server-cloudimg-amd64.img backing_fmt=qcow2 lazy_refcounts=off refcount_bits=16
# 생성 확인
qemu-img info /VM/playcecloud/TestVMwrk02.qcow2
## image: /VM/playcecloud/TestVMwrk02.qcow2
## file format: qcow2
## virtual size: 2.2 GiB (2361393152 bytes)
## disk size: 196 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# 가상 볼륨 크기 변경
qemu-img resize /VM/playcecloud/TestVMwrk02.qcow2 100G
## Image resized.
# 볼륨 변경 확인
qemu-img info /VM/playcecloud/TestVMwrk02.qcow2
## image: /VM/playcecloud/TestVMwrk02.qcow2
## file format: qcow2
## virtual size: 100 GiB (107374182400 bytes)
## disk size: 200 KiB
## cluster_size: 65536
## backing file: /VM/image/jammy-server-cloudimg-amd64.img
## backing file format: qcow2
## Format specific information:
## compat: 1.1
## compression type: zlib
## lazy refcounts: false
## refcount bits: 16
## corrupt: false
## extended l2: false
# hostname 설정 파일 생성
cat > /VM/script/meta-data << EOF
local-hostname: TestVMwrk02
EOF
# key file 지정 ( 계정이 cloud 일때 )
export account="cloud"
export PUB_KEY=$(cat /home/${account}/.ssh/authorized_keys)
# user 설정 파일 생성
cat > /VM/script/user-data << EOF
#cloud-config
password: password
users:
- default
- name: ${account}
ssh-authorized-keys:
- '$PUB_KEY'
sudo: ['ALL=(ALL) NOPASSWD:ALL']
groups: sudo
shell: /bin/bash
runcmd:
- cat /home/kwlee/.ssh/authorized_keys > /root/.ssh/authorized_keys
EOF
# OS 이미지 생성
genisoimage -output /VM/playcecloud/TestVMwrk02.iso -volid cidata -joliet -rock /VM/script/user-data /VM/script/meta-data
# 기준 서버 생성
virt-install --connect qemu:///system --virt-type kvm --name worker02 --ram 16384 --vcpus=4 --os-type linux --os-variant ubuntu22.04 --disk path=/VM/playcecloud/TestVMwrk02.qcow2,format=qcow2 --disk /VM/playcecloud/TestVMwrk02.iso,device=cdrom --import --network bridge=br0,model=virtio --network bridge=br3 --noautoconsole
구성 확인
# 가상 서버 구성 확인
virsh list
## Id Name State
## --------------------------------
## 1 controlplane02 running
## 2 controlplane03 running
## 3 worker02 running
# 서버 접속 확인
virsh console ${Virsh_NAME} # Password 사용시
ssh ${account}@${IPADDRESS} -i ~/.ssh/id_rsa # ssh-key 사용시
###############################################################################################
# 만약 서버 IP Address를 찾을 수 없거나, 패스워드를 모를경우
# 해당 서버 정지
virsh shutdown ${SERVER_NAME}
## Domain ${SERVER_NAME} is being shutdown
# root 패스워드 설정
apt install guestfs-tools
export PASS=`echo password`
virt-customize -a /VM/playcecloud/${SERVER_QCOW_NAME}.qcow2 --root-password password:$PASS --uninstall cloud-init
## [ 0.0] Examining the guest ...
## [ 13.7] Setting a random seed
## [ 13.9] Uninstalling packages: cloud-init
## [ 16.8] Setting passwords
## [ 23.6] Finishing off
# 기준 서버 실행
virsh start ${SERVER_NAME}
## Domain ${SERVER_NAME} started
VM IP Address 설정
# deploy 서버
tee > /etc/netplan/50-cloud-init.yaml << EOF
network:
version: 2
renderer: networkd
ethernets:
enp1s0:
dhcp4: false
dhcp6: false
addresses:
- 10.1.0.11/16
routes:
- to: 0.0.0.0/0
via: 10.1.1.254
nameservers:
addresses:
- 8.8.8.8
enp2s0:
dhcp4: false
dhcp6: false
addresses:
- 10.0.0.11/24
EOF
netplan apply
# control-plane01 서버
tee > /etc/netplan/50-cloud-init.yaml << EOF
network:
version: 2
renderer: networkd
ethernets:
enp1s0:
dhcp4: false
dhcp6: false
addresses:
- 10.1.0.21/16
routes:
- to: 0.0.0.0/0
via: 10.1.1.254
nameservers:
addresses:
- 8.8.8.8
enp2s0:
dhcp4: false
dhcp6: false
addresses:
- 10.0.0.21/24
EOF
netplan apply
# control-plane02 서버
tee > /etc/netplan/50-cloud-init.yaml << EOF
network:
version: 2
renderer: networkd
ethernets:
enp1s0:
dhcp4: false
dhcp6: false
addresses:
- 10.1.0.22/16
routes:
- to: 0.0.0.0/0
via: 10.1.1.254
nameservers:
addresses:
- 8.8.8.8
enp2s0:
dhcp4: false
dhcp6: false
addresses:
- 10.0.0.22/24
EOF
netplan apply
# control-plane03 서버
tee > /etc/netplan/50-cloud-init.yaml << EOF
network:
version: 2
renderer: networkd
ethernets:
enp1s0:
dhcp4: false
dhcp6: false
addresses:
- 10.1.0.23/16
routes:
- to: 0.0.0.0/0
via: 10.1.1.254
nameservers:
addresses:
- 8.8.8.8
enp2s0:
dhcp4: false
dhcp6: false
addresses:
- 10.0.0.23/24
EOF
netplan apply
# worker01 서버
tee > /etc/netplan/50-cloud-init.yaml << EOF
network:
version: 2
renderer: networkd
ethernets:
enp1s0:
dhcp4: false
dhcp6: false
addresses:
- 10.1.0.31/16
routes:
- to: 0.0.0.0/0
via: 10.1.1.254
nameservers:
addresses:
- 8.8.8.8
enp2s0:
dhcp4: false
dhcp6: false
addresses:
- 10.0.0.31/24
EOF
netplan apply
# worker02 서버
tee > /etc/netplan/50-cloud-init.yaml << EOF
network:
version: 2
renderer: networkd
ethernets:
enp1s0:
dhcp4: false
dhcp6: false
addresses:
- 10.1.0.32/16
routes:
- to: 0.0.0.0/0
via: 10.1.1.254
nameservers:
addresses:
- 8.8.8.8
enp2s0:
dhcp4: false
dhcp6: false
addresses:
- 10.0.0.32/24
EOF
netplan apply
계정 설정
## 만약에 계정이 정상적으로 생성이 되어 있지 않다면 VM 별로 적용 합니다.
# 계정 생성
export account="cloud"
groupadd -g 1001 ${account}
useradd -g 1001 -m -s /bin/bash ${account}
# sudo 설정
tee >> /etc/sudoers << EOF
${account} ALL=(ALL) NOPASSWD:ALL
EOF
# 계정 접속 Key 생성 ( 사용할 id_rsa.pub 키 내용 복사 == ${sshkey} )
su - ${account}
mkdir .ssh
chmod 700 .ssh
tee > ~/.ssh/authorized_keys << EOF
${sshkey}
EOF
chmod 600 ~/.ssh/authorized_keys
deploy용 스토리지 1TB를 NFS 로 구성하여 연결합니다. 랩탑 어디에서 작업하셔도 무관합니다.
# nfs에 사용할 disk 확인
df -h
## Filesystem Size Used Avail Use% Mounted on
## tmpfs 3.1G 2.4M 3.1G 1% /run
## /dev/nvme0n1p2 457G 15G 420G 4% /
## tmpfs 16G 0 16G 0% /dev/shm
## tmpfs 5.0M 4.0K 5.0M 1% /run/lock
## tmpfs 16G 0 16G 0% /run/qemu
## /dev/nvme0n1p1 511M 6.1M 505M 2% /boot/efi
## tmpfs 3.1G 76K 3.1G 1% /run/user/128
## tmpfs 3.1G 68K 3.1G 1% /run/user/1000
## /dev/sda1 880G 28K 835G 1% /mnt
# nfs-utils 설치
apt install nfs-kernel-server
chown -R nobody:nogroup /mnt
tee >> /etc/exports << EOF
/mnt 10.0.0.0/24(rw,sync,no_subtree_check)
EOF
exportfs -a
systemctl restart nfs-kernel-server
# NFS 서버 확인
showmount -e localhost
## Export list for localhost:
## /mnt 10.0.0.0/24
deploy서버에 nfs 연동 하기
# deploy 서버에서 작업합니다.
# 경로 생성
mkdir -p /playcecloud/srcdata
# nfs utils 설치
apt install nfs-common
# mount
mount -t nfs 10.0.0.200:/mnt /playcecloud
# fstab 등록
echo "10.0.0.200:/mnt /playcecloud nfs defaults 0 0" >> /etc/fstab
mount -a
# mount 확인
df -h
## Filesystem Size Used Avail Use% Mounted on
## tmpfs 591M 1.1M 590M 1% /run
## /dev/vda1 97G 1.5G 96G 2% /
## tmpfs 2.9G 0 2.9G 0% /dev/shm
## tmpfs 5.0M 0 5.0M 0% /run/lock
## /dev/vda15 105M 6.1M 99M 6% /boot/efi
## tmpfs 591M 4.0K 591M 1% /run/user/1001
## 10.0.0.200:/mnt 880G 0 835G 0% /playcecloud
자 이렇게 실습 준비가 끝났네요!!
이제 Playce Kube를 설치 할 수 있는 환경이 구성되었습니다.!
'kubernetes' 카테고리의 다른 글
| Playce Kube - CNI(Calico) (0) | 2023.06.26 |
|---|---|
| Playce Kube - 설치 확인 (0) | 2023.05.13 |
| Playce Kube - System Design (0) | 2023.01.14 |
| Playce Kube - 설치 (0) | 2023.01.14 |
| Playce Kube - 구조 (0) | 2022.08.29 |