====== 34 UnboundでHA構成 ====== この冗長化構成であれば、LVSが1台落ちても、DNSが1台落ちても、HVが1台落ちてもDNS/NTPサーバは稼働し続けられます。 {{drawio>06_virtualization:05_container:diagram1.png}} ===== IP構成 ===== ^ HV ^ VM ^ Gloval IP ^ Local IP ^ Gloval VIP ^ Local VIP ^ | HV01 | lvs01 | ens5: NIC only | ens4: 192.168.0.21 | 100.10.10.15 | 192.168.0.15 | | HV02 | lvs02 | ens5: NIC only | ens4: 192.168.0.22 | ::: | ::: | | HV01 | dns01 | ens5: 100.10.10.11 | ens4: 192.168.0.11 | | | | HV02 | dns02 | ens5: 100.10.10.12 | ens4: 192.168.0.12 | | | ===== 全VM共通 ===== ==== 1.事前準備 ==== sudo apt update sudo apt upgrade sudo timedatectl set-timezone Asia/Tokyo sudo localectl set-locale LANG=ja_JP.UTF-8 cat /etc/systemd/timesyncd.conf systemctl status systemd-timesyncd timedatectl timesync-status apt -y install tuned && tuned-adm profile latency-performance cat << __EOM__ >> /etc/hosts.allow sshd: 192.168.0. :allow sshd: all :deny __EOM__ sudo apt install resolvconf cat << __EOM__ >> /etc/resolvconf/resolv.conf.d/head nameserver 1.1.1.1 __EOM__ resolvconf -u ===== DNS VM ===== ==== 1.NTP ==== mkdir /app cd /app git clone https://github.com/cturra/docker-ntp.git mv docker-ntp ntp docker-compose.ymlに、network_mode追加 version: '3.9' services: ntp: build: . image: cturra/ntp:latest container_name: ntp restart: always ports: - 123:123/udp environment: - NTP_SERVERS=ntp.nict.jp,ntp.jst.mfeed.ad.jp,time.google.com - LOG_LEVEL=0 - TZ=Asia/Tokyo network_mode: "host" ==== 2.NTP起動 ==== docker-compose up -d === NTP動作確認 === サーバ # docker-compose exec ntp chronyc tracking Reference ID : D8EF230C (time4.google.com) Stratum : 2 Ref time (UTC) : Fri Dec 20 23:37:55 2024 System time : 0.028564310 seconds slow of NTP time Last offset : -0.000624378 seconds RMS offset : 0.000624378 seconds Frequency : 10.253 ppm slow Residual freq : -0.002 ppm Skew : 13.627 ppm Root delay : 0.034119464 seconds Root dispersion : 0.000823297 seconds Update interval : 64.3 seconds Leap status : Normal クライアント # ntpdate 10.60.1.251 21 Dec 08:35:17 ntpdate[1598029]: adjust time server 10.60.1.251 offset +0.021246 sec ==== 3.unbound ==== arptables install apt install arptables mkdir -p /app/unbound cd /app/unbound cat << __EOM__ > docker-compose.yml services: unbound: image: mvance/unbound:latest container_name: unbound restart: always ports: - '53:53' - '53:53/udp' #volumes: # - './unbound:/opt/unbound/etc/unbound/' network_mode: "host" __EOM__ ==== 3.unbound起動 ==== docker-compose up -d ==== 3.unbound修正 ==== unboundフォルダをコピーしてくる docker cp unbound:/opt/unbound/etc/unbound . cat << __EOM__ > docker-compose.yml services: unbound: image: mvance/unbound:latest container_name: unbound restart: always ports: - '53:53' - '53:53/udp' volumes: - './unbound:/opt/unbound/etc/unbound/' network_mode: "host" __EOM__ docker-compose up -d ==== 4.unbound修正② ==== === root.hints === cd /app/unbound/unbound curl -L -o root.hints https://www.internic.net/domain/named.cache === unbound_server.pem/unbound_control.pem === docker-compose exec unbound bash unbound-control-setup ==== 5.Loopback ==== VIPから来る通信を受け取るため、Loopback設定 cat << __EOM__ > /etc/rc.local #!/bin/bash iptables -t nat -A PREROUTING -d 192.168.0.15/32 -j REDIRECT iptables -t nat -A PREROUTING -d 100.10.10.15/32 -j REDIRECT __EOM__ chmod 700 /etc/rc.local ==== 6.resolvconf ==== cat << __EOM__ > /etc/resolvconf/resolv.conf.d/head nameserver 127.0.0.1 nameserver 1.1.1.1 __EOM__ resolvconf -u systemd-resolvedでポート 53 を使用しないように設定 sed -i 's/^.DNSStubListener=yes$/DNSStubListener=no/g' /etc/systemd/resolved.conf grep ^DNSStubListener /etc/systemd/resolved.conf DNSStubListener=no sudo systemctl restart systemd-resolved sudo systemctl enable systemd-resolved ===== LVS VM ===== ==== 1.KeepAliveインストール ==== apt install ipvsadm keepalived ==== 2.KeepAlive設定 ==== 今回KeepAliveのVRRPで、arpの問題が出て実際にはVIPを持っていない方のVMへ通信が飛ぶことで、dig などがタイムアウトする問題が発生した。 このarp問題を解決する方法を記述 [[https://qiita.com/uturned0/items/860c99783550d9be323e|VRRPの動き]] ==== 3.track_script ==== track_script用意（冗長構成を切り替える際にトリガー） これを用意する事によって、ip l set ens5 down すると、VRRPが切り替わる。 cat << __EOM__ > /etc/keepalived/chk_gateway #!/bin/bash if ping -c 1 100.10.10.254 -I ens5 > /dev/null 2>&1; then exit 0 else exit 1 fi __EOM__ chmod +x /etc/keepalived/chk_gateway ==== 4.notify用意 ==== cat << __EOM__ > /etc/keepalived/keepalived_notify.sh #!/bin/bash INSTANCE=$1 INTERFACE=$2 STATE=$3 case $STATE in "MASTER"|"START") echo "`date`: Switching to MASTER, bringing up ens5 " >> /var/log/keepalived_notify.log arptables -F OUTPUT ;; "BACKUP" | "FAULT" | "STOP") echo "`date`: Switching to BACKUP/FAULT, bringing down ens5 " >> /var/log/keepalived_notify.log if [ "$(arptables -L OUTPUT -nv | grep ens5)" == "" ]; then arptables -A OUTPUT -o ens5 -j DROP fi ;; *) echo "`date`: Unknown state: $STATE" >> /var/log/keepalived_notify.log ;; esac exit 0 __EOM__ chmod +x /etc/keepalived/keepalived_notify.sh ==== 5.keepalived.conf ==== cat << __EOM__ > /etc/keepalived/keepalived.conf vrrp_instance VI { state MASTER interface ens4 virtual_router_id 1 priority 101 advert_int 1 authentication { auth_type PASS auth_pass XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX } virtual_ipaddress { 192.168.0.15/24 dev ens4 100.10.10.15/24 dev ens5 } track_interface { ens4 ens5 } track_script { script "/etc/keepalived/chk_gateway" } notify "/etc/keepalived/keepalived_notify.sh" } virtual_server 192.168.0.15 53 { delay_loop 6 lb_algo sh lb_kind DR protocol UDP real_server 192.168.0.11 53 { TCP_CHECK { connect_port 53 connect_timeout 10 } } real_server 192.168.0.12 53 { TCP_CHECK { connect_port 53 connect_timeout 10 } } } virtual_server 192.168.0.15 53 { delay_loop 6 lb_algo sh lb_kind DR protocol TCP real_server 192.168.0.11 53 { TCP_CHECK { connect_port 53 connect_timeout 10 } } real_server 192.168.0.11 53 { TCP_CHECK { connect_port 53 connect_timeout 10 } } } virtual_server 100.10.10.15 53 { delay_loop 6 lb_algo sh lb_kind DR protocol UDP real_server 100.10.10.11 53 { TCP_CHECK { connect_port 53 connect_timeout 10 } } real_server 100.10.10.11 53 { TCP_CHECK { connect_port 53 connect_timeout 10 } } } virtual_server 100.10.10.15 53 { delay_loop 6 lb_algo sh lb_kind DR protocol TCP real_server 100.10.10.11 53 { TCP_CHECK { connect_port 53 connect_timeout 10 } } real_server 100.10.10.11 53 { TCP_CHECK { connect_port 53 connect_timeout 10 } } } virtual_server 192.168.0.15 123 { delay_loop 6 lb_algo rr lb_kind DR protocol UDP real_server 192.168.0.11 123 { UDP_CHECK { connect_port 123 connect_timeout 10 } } real_server 192.168.0.12 123 { UDP_CHECK { connect_port 123 connect_timeout 10 } } } __EOM__ ※BACKUP側は、stateと、priorityが違うだけです。 vrrp_instance VI { state BACKUP interface ens4 virtual_router_id 1 priority 100 advert_int 1 ==== 6.KeepAlive起動 ==== systemctl restart keepalived.service ==== 7.KeepAlive確認 ==== MASTER側にVIPが付いている事確認 # ip -4 a 1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 inet 127.0.0.1/8 scope host lo valid_lft forever preferred_lft forever 2: ens4: mtu 1500 qdisc fq_codel state UP group default qlen 1000 altname enp0s4 inet 192.168.0.11/16 metric 100 brd 192.168.0.255 scope global dynamic ens4 valid_lft 83689sec preferred_lft 83689sec inet 192.168.0.15/16 scope global secondary ens4 valid_lft forever preferred_lft forever 3: ens5: mtu 1500 qdisc fq_codel state UP group default qlen 1000 altname enp0s5 inet 100.10.10.15/24 scope global ens5 valid_lft forever preferred_lft forever # ipvsadm -Ln IP Virtual Server version 1.2.1 (size=4096) Prot LocalAddress:Port Scheduler Flags -> RemoteAddress:Port Forward Weight ActiveConn InActConn TCP 100.10.10.15:53 sh -> 100.10.10.11:53 Route 1 0 0 -> 100.10.10.12:53 Route 1 0 0 TCP 192.168.0.15:53 sh -> 192.168.0.11:53 Route 1 0 0 -> 192.168.0.12:53 Route 1 0 0 UDP 100.10.10.15:53 sh -> 100.10.10.11:53 Route 1 0 0 -> 100.10.10.12:53 Route 1 0 0 UDP 192.168.0.15:53 sh -> 192.168.0.11:53 Route 1 0 0 -> 192.168.0.12:53 Route 1 0 0 UDP 192.168.0.15:123 rr -> 192.168.0.11:123 Route 1 0 0 -> 192.168.0.12:123 Route 1 0 0 ===== エラー ===== ==== network_mode is incompatible with port_bindings ==== ERROR: for unbound "host" network_mode is incompatible with port_bindings ERROR: for unbound "host" network_mode is incompatible with port_bindings . . . docker.errors.InvalidArgument: "host" network_mode is incompatible with port_bindings === 対応 === バージョンによってはこんなエラーになる。 この場合は、network_mode "host" を指定する時は、portをコメントアウト {{tag>DNS unbound docker}}