Keepalived+Haproxy双主高可用负载均衡web和mysql综合实验
日期及版本：2014.5.4v1.0
架构图
实验目的：
1.Haproxy+Keepalived双主双机高可用模型，keepalived为Haproxy主从提供高可用保证haproxy-master若挂掉haproxy-backup能无缝接管，haproxy为后端Web提供负载均衡，缓解并发压力，实现WEB站点负载均衡+高可用性；
2. Haproxy反代web做动静分离；
3. Haproxy反代mysql 算法leastconn和roundrobin的不同效果；
系统环境：
OS：centos6.4 X64
apache: httpd-2.2.15-26.el6.centos.x86_64
haproxy: haproxy-1.4.22-3.el6.x86_64
keepalived：keepalived-1.2.7
基础配置：
Role Out-IP In-IP Host
haproxy VIP: 192.168.1.10
haproxy-master(haproxy1): 192.168.1.11 172.16.100.254 ha1.example.com
haproxy-backup(haproxy2): 192.168.1.12 172.16.100.253 ha2.example.com
apache1: 172.16.100.13 web1.example.com
apache2: 172.16.100.14 web2.example.com
实验步骤：因为都是双机配置，重复内容合并
1、配置后端web1，web2按IP修改
[root@web1 ~]# echo ‘<h1>web1.example.com</h1>‘ > /var/www/html/index.html
[root@ web1 html]# cp index.html index.php
[root@web1 html]# cat >> index.php <<EOF
[root@web1 html]# service httpd restart
2、配置Haproxy反向代理
[root@ha1 ~]# sed -i ‘/net.ipv4.ip_forward/ s/\(.*= \).*/\11/‘ /etc/sysctl.conf
net.ipv4.ip_forward = 1
[root@ha1 ~]# cd /etc/haproxy/
[root@ha1 haproxy]# cp haproxy.cfg haproxy.cfg.bak
用rsyslog的理由：
1.防止系统崩溃无法获取系统日志分享崩溃原因，用rsyslog可以把日志传输到远程的日志服务器上
2.使用rsyslog日志可以减轻系统压力，因为使用rsyslog可以有效减轻系统的磁盘IO
3.rsyslog使用tcp传输非常可靠，可以对日志进行过滤，提取出有效的日志，rsyslog是轻量级的日志软件，在大量日志写的情况下，系统负载基本上在0.1以下
[root@ ha1 haproxy ~]# vim /etc/rsyslog.conf
[root@ha1 haproxy]# service rsyslog restart
3、双机公共设置ha1与ha2相同设置
[root@ha1 haproxy]# vim haproxy.cfg
在web1，web2服务器配置中日志格式添加"X-Forwarded-For"特殊首部，以便后端服务器记录真实发起请求的客户端IP地址
[root@web1 ~]# vim /etc/httpd/conf/httpd.conf
搜LogFormat 第一个字节段插入%{X-Forwarded-For}i
LogFormat "%{X-Forwarded-For}i %h %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
[root@web1 ~]# sed -i ‘/^LogFormat.*combined/ s#"#"\%{X-Forwarded-For}\i#‘ /etc/httpd/conf/httpd.conf
[root@web1 html]# service httpd reload
[root@web2 html]# service httpd restart
验证静态页面的负载均衡，动态页面长连接保持于同一后端服务器，动静分离。
Haproxy反代日志会详细显示前端客户端IP,Haproxy反代IP,根据动静分离ACL选择不同后端服务器，GET到不同页面。后端upstream服务器别名
[root@ha1 haproxy]# tail /var/log/haproxy.log
May 4 01:15:25 localhost haproxy[1819]: 192.168.1.108:50854 [04/May/2014:01:15:25.571] webservers dynamic_servs/dynamic2 24/0/1/2/+27 200 +177 - - ---- 2/2/2/2/0 0/0 {192.168.1.11||http://192.168.1.11/} "GET /index.php?=PHPE9568F34-D428-11d2-A769-00AA001ACF42 HTTP/1.1"
May 4 01:15:25 localhost haproxy[1819]: 192.168.1.108:50855 [04/May/2014:01:15:25.572] webservers dynamic_servs/dynamic2 25/0/0/2/+27 200 +177 - - ---- 1/1/1/1/0 0/0 {192.168.1.11||http://192.168.1.11/} "GET /index.php?=PHPE9568F35-D428-11d2-A769-00AA001ACF42 HTTP/1.1"
May 4 01:15:28 localhost haproxy[1819]: 192.168.1.108:50856 [04/May/2014:01:15:28.552] webservers static_servs/static1 5/0/1/1/+7 304 +147 - - ---- 3/3/1/1/0 0/0 {192.168.1.11||} "GET /index.html HTTP/1.1"
May 4 01:15:29 localhost haproxy[1819]: 192.168.1.108:50857 [04/May/2014:01:15:28.556] webservers static_servs/static2 799/0/1/1/+801 200 +265 - - ---- 2/2/1/1/0 0/0 {192.168.1.11||} "GET /index.html HTTP/1.1"
[root@ha2 haproxy]# tail /var/log/haproxy.log
May 4 01:38:30 localhost haproxy[1733]: 192.168.1.108:51208 [04/May/2014:01:38:30.436] webservers dynamic_servs/dynamic2 24/0/0/4/+28 200 +177 - - ---- 2/2/2/2/0 0/0 {192.168.1.12||http://192.168.1.12/} "GET /index.php?=PHPE9568F34-D428-11d2-A769-00AA001ACF42 HTTP/1.1"
May 4 01:38:30 localhost haproxy[1733]: 192.168.1.108:51209 [04/May/2014:01:38:30.436] webservers dynamic_servs/dynamic2 24/0/0/4/+28 200 +177 - - ---- 2/2/2/1/0 0/0 {192.168.1.12||http://192.168.1.12/} "GET /index.php?=PHPE9568F35-D428-11d2-A769-00AA001ACF42 HTTP/1.1"
May 4 01:39:04 localhost haproxy[1733]: 192.168.1.108:51219 [04/May/2014:01:39:04.178] webservers static_servs/static1 88/0/1/1/+90 200 +265 - - ---- 3/3/1/1/0 0/0 {192.168.1.12||} "GET /index.html HTTP/1.1"
May 4 01:39:38 localhost haproxy[1733]: 192.168.1.108:51229 [04/May/2014:01:39:38.161] webservers static_servs/static2 8/0/2/0/+10 200 +265 - - ---- 3/3/1/1/0 0/0 {192.168.1.12||} "GET /index.html HTTP/1.1"
各个后端服务器日志记录前端客户端IP,Haproxy反代内网卡(内网关)IP,Get到不同页面，动态页面还有haproxy反代外网卡IP
[root@web1 ~]# tail /var/log/httpd/access_log
192.168.1.108172.16.100.254 - - [04/May/2014:01:14:01 +0800] "GET /index.html HTTP/1.1" 304 - "-"
[root@web2 ~]# tail /var/log/httpd/access_log
192.168.1.108172.16.100.253 - - [04/May/2014:01:38:29 +0800] "GET /index.php?=PHPE9568F34-D428-11d2-A769-00AA001ACF42 HTTP/1.1" 200 2524 "http://192.168.1.12/"
192.168.1.108172.16.100.253 - - [04/May/2014:01:38:29 +0800] "GET /index.php?=PHPE9568F35-D428-11d2-A769-00AA001ACF42 HTTP/1.1" 200 2146 "http://192.168.1.12/"
不同http://192.168.1.1x:8080/haproxyadmin?stats访问同一侦听stats主机的指定url,端口经过身份验证，显示管理页面。
配置keepalived主从服务器，主从相同实例高低相反
[root@ha1 ~]# cp /etc/keepalived/keepalived.conf /etc/keepalived/keepalived.conf.bak
192.168.1.11 keepalived主节点配置
[root@ha1 ~]# vim /etc/keepalived/keepalived.conf
[root@ha1 keepalived]# service keepalived start
[root@ha2 keepalived]# vim keepalived.conf
[root@ha2 keepalived]# service keepalived start
[root@ha1 ~]# ip addr | grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 192.168.1.11/24 brd 192.168.1.255 scope global eth0
inet 192.168.1.88/24 scope global secondary eth0
[root@ha2 ~]# ip addr | grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 192.168.1.12/24 brd 192.168.1.255 scope global eth0
inet 192.168.1.188/24 scope global secondary eth0
[root@ha1 ~]# tail /var/log/messages
May 4 04:49:26 station11 Keepalived_vrrp[3973]: VRRP_Instance(VI_1) Transition to MASTER STATE
May 4 04:49:27 station11 Keepalived_vrrp[3973]: VRRP_Instance(VI_1) Entering MASTER STATE
May 4 04:49:35 station11 Keepalived_vrrp[3973]: VRRP_Instance(VI_2) Entering BACKUP STATE
[root@ha2 ~]# tail /var/log/messages
May 4 04:49:35 station12 Keepalived_vrrp[3594]: VRRP_Instance(VI_1) Entering BACKUP STATE
May 4 04:49:35 station12 Keepalived_vrrp[3594]: VRRP_Instance(VI_2) Transition to MASTER STATE
May 4 04:49:36 station12 Keepalived_vrrp[3594]: VRRP_Instance(VI_2) Entering MASTER STATE
停止ha1服务器haproxy服务，keepalived进行重新选举，ha2提升VI_1从备到主，VIP切换到备用服务器
[root@ha1 ~]# service haproxy stop
[root@ha2 ~]# tail /var/log/messages
May 4 04:56:00 station12 Keepalived_vrrp[3594]: VRRP_Instance(VI_1) forcing a new MASTER election
May 4 04:56:01 station12 Keepalived_vrrp[3594]: VRRP_Instance(VI_1) Transition to MASTER STATE
May 4 04:56:02 station12 Keepalived_vrrp[3594]: VRRP_Instance(VI_1) Entering MASTER STATE
[root@ha1 ~]# ip addr | grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 192.168.1.11/24 brd 192.168.1.255 scope global eth0
[root@ha2 ~]# ip addr | grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 192.168.1.12/24 brd 192.168.1.255 scope global eth0
inet 192.168.1.188/24 scope global secondary eth0
inet 192.168.1.88/24 scope global secondary eth0
启动ha1服务器haproxy服务，VIP重新切换回主服务器
[root@ha1 ~]# service haproxy start
[root@ha1 ~]# ip addr | grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 192.168.1.11/24 brd 192.168.1.255 scope global eth0
inet 192.168.1.88/24 scope global secondary eth0
[root@ha2 ~]# ip addr | grep eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
inet 192.168.1.12/24 brd 192.168.1.255 scope global eth0
inet 192.168.1.188/24 scope global secondary eth0
[root@ha1 ~]# tail /var/log/messages
May 4 05:07:41 ha1 Keepalived_vrrp[3973]: VRRP_Instance(VI_1) forcing a new MASTER election
May 4 05:07:42 ha1 Keepalived_vrrp[3973]: VRRP_Instance(VI_1) Transition to MASTER STATE
May 4 05:07:43 ha1 Keepalived_vrrp[3973]: VRRP_Instance(VI_1) Entering MASTER STATE
[root@ha2 ~]# tail /var/log/messages
May 4 05:07:41 station12 Keepalived_vrrp[3594]: VRRP_Instance(VI_1) Entering BACKUP STATE
3、反代mysql 算法leastconn和roundrobin的不同效果？
[root@web1 ~]# yum -y install mysql mysql-server mysql-devel
[root@web1 ~]# service mysqld start
[root@web1 ~]# mysqladmin -u root password redhat
[root@web1 ~]# mysql -uroot -predhat
station14相同
[root@ha1 haproxy]# vim haproxy.cfg
[root@ha1 haproxy]# service haproxy reload
[root@ha2 ~]# mysql -umysqlchecker -h192.168.1.11
mysql> select @@hostname;
| station14.example.com |
[root@ha2 ~]# mysql -umysqlchecker -h192.168.1.11
mysql> select @@hostname;
| station13.example.com |
[root@ha1 haproxy]# vim haproxy.cfg
balance leastconn 修改算法为最少连接数
[root@ha1 haproxy]# service haproxy reload
当前最大连接数
[root@web1 ~]# mysql -uroot -predhat -e "show status like ‘%connections%‘;"
| Connections | 6607 |
| Max_used_connections | 3 |
没有大量客户端进行测试，它只计算活动连接数，哪个后端服务器的连接最少就调度到哪台。
本文出自 “aha45” 博客，请务必保留此出处http://manfred12.blog.51cto.com/137312/1406097