抱歉，您的浏览器无法访问本站
本页面需要浏览器支持（启用）JavaScript
了解详情 >

工具

chat

badge

友链

keepalived双主模型的实现

中间件应用高可用

发布于：2018年2月7日

次浏览

[toc]

一、简介

keepalived 简介:是服务器高可用的一个重要软件;
它的核心组件有vrrp ，stack， checker ，ipvs，warpper， watch dog ;

它是vrrp协议的实现，原生设计目的为高可用ipvs服务；
keepalived能够通过配置文件中定义生成ipvs规则;

并能够对RS的健康状态进行检测；vrrp_script,vrrp_track;

  • 双主模型的实现

简介:
双方模型(主/备，备/主)的这里的意思是，一个keepalived配置中;
一个虚拟IP地址为主，另一个为备。而在另一个keepalived的配置中;
与其它主机则恰恰相反，一个虚拟IP地址为备，另一个为主;

以下为此次双主模型实现的拓扑

二、HA Cluster配置的前提；

(1)要点:各节点之间的时间秘需要同步:

ntpdate 172.16.0.1 (注此:处可自己在网上找个时间同步服务器)

(2)确保iptables及selinux不会阻碍:

#iptables -F && setenforce 0

###(3)各节点之间可通过主机名互相通信(对keepalived并非必须):

# vim /etc/hosts 

172.16.250.140 kpl1

172.16.250.158 kpl2 两台keepalived主机都要修改

###(4)各节点之间root用户可以基于密钥认证的ssh通信

# ssh-copy-id -i 172.16.250.140

# ssh-copy-id -i 172.16.250.158

三、dr集群配置

此处使用dr集群类型

首先将dr模型配置好 

# vim setkp.sh
# !/bin/bash 
vip=172.16.26.126
vip2=172.16.26.127
mask=255.255.255.255
interface='lo:0'
interface2='lo:1'
eth='eno16777736:0' 
case $1 in 
start) 
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $interface $vip netmask $mask broadcast $vip up 
ifconfig $interface2 $vip2 netmask $mask broadcast $vip2 up 
route add -host $vip dev $interface 
route add -host $vip2 dev $interface2
;; 
dstart) 
ifconfig $eth $vip/32 netmask $mask broadcast $vip up
;;
dstop)
ifconfig $eth down
;;
stop)
ifconfig $interface down 
ifconfig $interface2 down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
;; 
status) 
ifconfig 
cat /proc/sys/net/ipv4/conf/all/arp_ignore
cat /proc/sys/net/ipv4/conf/lo/arp_ignore
cat /proc/sys/net/ipv4/conf/all/arp_announce
cat /proc/sys/net/ipv4/conf/lo/arp_announce
;; 
*)
echo "Usage: $(basename $0) {dstart|dstop|start|stop}"
exit 1 
esac

RS1、RS2主机中都执行执行此脚本 

# sh setkp.sh start 
# sh setkp.sh status 可用来查看当前网络的配置状态 
#RS1、RS2主机中安装httpd并启动 
# yum -y install httpd && systemctl start httpd  
RS1# echo "<h1>RS1</h1>" > /usr/share/nginx/html/index.html 
RS2 # echo "<h1>RS2</h1>" > /usr/share/nginx/html/index.html

四、keepalived配置

keepalived,keepalived2 中安装nginx
(httpd和nginx都可以,此处用来做为keepalived的本身的提供sorry server)
当dr集群的两台节点都停止时，会由keepalived本身来提供一个页面

# yum -y install nginx keepalived ipvsadm && systemctl start nginx 
# echo "<h1>sorry server keepalived 1 </h1>" > /usr/share/nginx/html/index.html 
# echo "<h1>sorry server keepalived 2 </h2>" > /usr/share/nginx/html/index.html 
```bash

在一台keepalived主机上进行测试

```bash
# curl 172.16.251.232
<h1>RS1</h2>
# curl 172.16.250.159
<h1>RS2</h1>

keepalived,keepalived2 中在 /etc/keepalived/目录下编辑一个脚本来用来在主备变化
或服务down掉时发邮件给系统用户 

# vim kmail.sh 
#!/bin/bash
contact='root@localhost'
notify(){
mailsubject="$(hostname) to be $1:vip floating"
mailbody="$(date +'%F %T'):vrrp transition, $(hostname) change to be $1"
echo $mailbody | mail -s "$mailsubject" $contact
}
case $1 in 
master )
notify master
;;
backup )
notify backup 
;;
fault )
notify fault 
;;
*)
echo "Usage:$(basename $0) {master |backup|fault}"
;;
esac
#chmod +x kmail.sh

注:keepalived的两台主机的配置基本相同，当配置模型为主/备模型的时候，主备之间需要修改三个指令

分别为 router_id kpl1 在keepalived2上时需要修改为 router_id kpl2
state MASTER 在keepalived2上时需要修改为 state BACKUP
priority 100 在keepalived2上时需要修改为 priority 90 (此处的值比主服务器的小便可以)
以下主/备的配置示例


###先把备的服务器启动

# tcpdump -i eno33554984 -nn host 224.0.61.61

可使用tcpdump 抓取组播地址 来查看其通过组播方式传递的心跳信息
以下的配置文件便是keepavlied双主模型的实现
对防火墙打标记
keepalived 1

# iptables -t mangle -A PREROUTING -d 172.16.26.126 -p tcp --dport 80 -j MARK --set-mark 3 
# iptables -t mangle -A PREROUTING -d 172.16.26.127 -p tcp --dport 80 -j MARK --set-mark 3
# vim /etc/keepalived/keepalived.conf
! Configuration File for keepalived 
global_defs {
  notification_email {
root@localhost
}
  notification_email_from Alexandre.Cassen@firewall.loc
  smtp_server 127.0.0.1
  smtp_connect_timeout 30
  router_id kpl1
  vrrp_mcast_group4 224.0.61.61
}

vrrp_instance VI_1 {
state MASTER
interface eno33554984
virtual_router_id 55
priority 100
advert_int 1
notify_master "/etc/keepalived/kmail.sh master"
notify_backup "/etc/keepalived/kmail.sh backup"
notify_fault  "/etc/keepalived/kmail.sh fault"
authentication {
auth_type PASS
auth_pass zE2kNsRQ
}

virtual_ipaddress {
172.16.26.126 dev eno33554984 label eno33554984:0    
}
}
vrrp_instance VI_2 {
state BACKUP
interface eno33554984
virtual_router_id 66
priority 90
advert_int 1
notify_master "/etc/keepalived/kmail.sh master"
notify_backup "/etc/keepalived/kmail.sh backup"
notify_fault  "/etc/keepalived/kmail.sh fault"
authentication {
auth_type PASS
auth_pass zE2kfsRQ
}
virtual_ipaddress {
172.16.26.127 dev eno33554984 label eno33554984:1    
}
}
virtual_server fwmark 3 {
delay_loop 2
lb_algo wrr 
lb_kind DR
nat_mask 255.255.0.0
protocol TCP
sorry_server 127.0.0.1 80

real_server 172.16.251.232 80 {
weight 3
HTTP_GET {
url { 
 path /
 status_code 200  
 }
connect_timeout 2
nb_get_retry 3
delay_before_retry 3
}
}
real_server 172.16.250.159 80 {
weight 1
HTTP_GET {
url { 
path /
status_code 200  u
 }
oconnect_timeout 2e
nb_get_retry n
delay_before_retry 3
}
}

keepalived 2 

# iptables -t mangle -A PREROUTING -d 172.16.26.126 -p tcp --dport 80 -j MARK --set-mark 3
# iptables -t mangle -A PREROUTING -d 172.16.26.127 -p tcp --dport 80 -j MARK --set-mark 3
# vim /etc/keepavlied/keepalived.conf 
! Configuration File for keepalived
global_defs {
  notification_email {
root@localhost
}
  notification_email_from Alexandre.Cassen@firewall.loc
  smtp_server 127.0.0.1
  smtp_connect_timeout 30
  router_id kpl2
  vrrp_mcast_group4 224.0.61.61
}

vrrp_instance VI_1 {
state BACKUP
interface eno33554984
virtual_router_id 55
priority 90
advert_int 1
notify_master "/etc/keepalived/kmail.sh master"
notify_backup "/etc/keepalived/kmail.sh backup"
notify_fault  "/etc/keepalived/kmail.sh fault"
authentication {
auth_type PASS
auth_pass zE2kNsRQ
}
virtual_ipaddress {
172.16.26.126 dev eno33554984 label eno33554984:0    
}
}
vrrp_instance VI_2 {
state MASTER
interface eno33554984
virtual_router_id 66
priority 100
advert_int 1
notify_master "/etc/keepalived/kmail.sh master"
notify_backup "/etc/keepalived/kmail.sh backup"
notify_fault  "/etc/keepalived/kmail.sh fault"
authentication {
auth_type PASS
auth_pass zE2kfsRQ
}
virtual_ipaddress {
172.16.26.127 dev eno33554984 label eno33554984:1    
}
}
virtual_server fwmark 3  {
delay_loop {
lb_algo wrr
lb_kind DR
nat_mask 255.255.0.0
protocol TCP
sorry_server 127.0.0.1 80
real_server 172.16.251.232 80 {
weight 3
HTTP_GET {
url {
 path /
 status_code 200
 }
connect_timeout 2
nb_get_retry 3
delay_before_retry 3
}
}

real_server 172.16.250.159 80 {
weight 1
HTTP_GET {
url {
 path /
 status_code 200
 }
connect_timeout 2
nb_get_retry 3
delay_before_retry 3
}
}

五、测试

此时可以测试其访问


当一个keepavlied停止时

当RS2停止时

当RS1，RS2都停止时

##六、keepavlied配置指令说明

虚拟路由器段

state MASTER：当前节点在虚拟路由器中的初始状态；
interface ETHERCARD: vrrp实际工作的网卡接口
virtual_route_id 51 :虚拟路由器ID，范围0-255；
priority 100 :当前物理节点在此虚拟路由器中的优先级；
advert_int 1:每隔多久发送心跳(通行的时间间隔)
auth_type PASS ：选择认证机制 
auth_pass 1111 ：密码 八位有效
virtual_ipaddress ：定义虚拟IP 
track_interface ： 定义要监控的接口
notify_master <STRING> | <QUOTED-STRING> ：当前节点变为主节点时用STRING脚本通告
notfy_backup<STRING> | <QUOTED-STRING> : 当前节点变为主节时用 STRING脚本通告
notify_fault<STRING> | <QUOTED-STRING> : 当前节点上不了线时用STRING脚本通告
notify<STRING> | <QUOTED-STRING> : 如果三种状态用一个脚本来实现用STRING脚本通告

虚拟服务段

lb_algo rr | wrr|lc|lblc|sh|dh :定义负载均衡调度算法
delay_loop<INT>：：定义服务轮询时间间隔
bl_kind NAT |DR |TUN ：集群的类型
persistence_time_out<INT> ：持久连接时长
protocol TCP : 服务协议
sorry_server<IPADDR><PORT>：所有RS均故障时，提供sorry server的服务器；
real_server<IPADDR><PORT>:
weight<INT>：权重
notify_up<STRING>|<QUOTED-STRING> ： 节点上线通知脚本
notify_down <STRING>|<QUOTED-STRNG>：节点离线通知脚本；
#HTTP_GET|SSL_GET|TCP_CHECK|SMTP_CHECK|MISC_CHECK ：支持的所以健康状态的检测方式
url:健康状态检测时请求的资源的URL 
delay_before_retry<INT> ：两次尝试之间的时间间隔 
connect_timeoute<STRING>：连接的超时时长
connect_ip<IP ADDRESS>：向此处指定的地址发测试请求
connect_port<PORT>：向此处指定的PORT发测试请求
bindto<IP ADDRESS>：指定测试请求报文的源IP 
bind_port<PORT>： 指定测试请求报文的源PORT

更新于：2023年1月16日

linux

sre

keepalived

评论