Front end load and high availability configuration (hproxy + keepalived)

osc_ sxdofc9c 2020-11-12 12:09:47
end load high availability configuration

Front end load and high availability configuration (Hproxy+Keepalived)

The topology is as follows :


In order to facilitate future maintenance and operation , Using a relatively simple technology , But understand. , The technology is simple, just relative to the user , Developers simplify user configuration , But it's still powerful . For load balancing , We use haproxy technology , High availability adoption keepalived technology .

notes :Haproxy Can provide high availability 、 Load balancing and based on TCP and HTTP Agent for application , Support virtual host , And it's free .

       Haproxy No multiprocessing or multithreading is used , It's about implementing an event driven 、 Single process model , So you don't have to be limited by memory and so on , But usually , Its scalability will be poor .

       Keepalived Is a similar to layer3,4,5 Software for the exchange mechanism , In other words, it is usually said that 3,4,5 layer

In exchange for .

        Layer3,4&5 Working in IP/TCP The protocol stack IP layer ,TCP layer

        Layer3:keepalived Use layer3 The way you work ,keepalived A... Is sent to the servers in the server cluster on a regular basis ICMP Data packets , therefore ,layer3 The way is to use the server IP Whether the address is valid is taken as the standard of whether the server works normally

        Layer4:Layer4 Mainly with TCP The state of the port determines whether the server is working properly .

        Layer5:Layer5 It's working in a specific application layer , It will check whether the server program is running properly according to the user's settings , If it doesn't match the user's settings , This server will be removed from the server cluster .

        Keepalived It is mainly used for RealServer Health check and LoadBalance The host and Slave Between the host failover The implementation of the

Haproxy To configure (Master)

notes : This experiment is just a test environment , about WEB The server is just a simple implementation .

Connect the server with the environment shown in the figure ,IP etc. , We are going to haproxy—keepalived In order to configure .

Haproxy Service installation

notes : Please download the required software by yourself , It's best to use the latest version .

notes : Turn off before configuring iptables And selinux

1. Change host name

We use The server is Keepalived master server , Name it Master.KPLD; It's a secondary server , Name it :slave.KPLD

[root@localhost ~]# cat /etc/hosts  localhost Master.KPLD localhost4 localhost4.localdomain4

::1        localhost localhost.localdomain localhost6 localhost6.localdomain6

[root@localhost ~]# cat/etc/sysconfig/network



2. Unpack the installation haproxy software package

# tar zxvf haproxy-1.3.20.tar.gz

[root@Master ~]# tar -zxvfhaproxy-1.3.20.tar.gz

[root@Master ~]# cd haproxy-1.3.20

[root@Master haproxy-1.3.20]# uname -r

2.6.32-279.el6.x86_64      // Here to check the kernel version number , install haproxy When , To specify the correct version number

          [root@Master haproxy-1.3.20]# vim Makefile        

             64 PREFIX = /usr/local It is amended as follows : 64PREFIX = /usr/local/haproxy

Because the decompressed file is compiled , Someone uses it directly during installation —prefix= Indicate installation path , I did it a few times , Discovery is not going to work , So you want to put the installation files in a directory , It can only be modified Makefile file

      [root@Master haproxy-1.3.20]# make TARGET=linux26   // Corresponding to the above kernel version

      [root@Master haproxy-1.3.20]# make install

      [root@Master haproxy-1.3.20]# mkdir /usr/local/haproxy/etc   // Create a directory of configuration files

      [root@Master haproxy-1.3.20]# cp -p examples/haproxy.cfg/usr/local/haproxy/etc/    //haproxy There are configuration file templates under the source code , We just need copy In the past , You can modify it.

3.Haproxy Service configuration

[root@Masteretc]# vim haproxy.cfg


      log   local0

       log   local1 notice

       #log loghost    local0 info

       maxconn 4096

       chroot /usr/share/haproxy

       uid 99

       gid 99

       daemon                 // To configure haproxy Enter the background operation mode

       nbproc 2                 // establish 2 Processes enter deamon mode

       pidfile /usr/local/haproxy/




       log    global

       mode   http       // The default mode mode

       option httplog

       option dontlognull   // Automatically end completed Links

       retries 3

       option redispatch       // When serverid The corresponding server is down , Force a redirect to another healthy server

       maxconn 2000

       contimeout      5000

       clitimeout      50000

       srvtimeout      50000

       stats uri /haproxy_stats         // Visit the statistics page url

       stats realm user \ passwd        // Statistics page password box prompt text

       stats auth haproxy:haproxy       // Statistics page user name and password

       stats hide-version               // hide haproxy Version information


       cookie SERVERID rewrite


#       balance source                 // It's best to use this , The same client can be guaranteed , Fixed access to a server

       server cookie check inter 2000 rise 2fall

       server cookie192.168.1.20 check inter 2000 rise 2 fall 5

//  Server definition ,checkinter 2000 It refers to the detection of heart frequency ,rise finger 3 The second positive solution is that the server is available ,fall 5 Failure 5 I think the server is not available , You can also set weights weigth  Numbers

After configuration , You need to create a file , It corresponds to the configuration

[root@Master etc]# mkdir /usr/share/haproxy

4. Log configuration

     [root@Masteretc]# vim /etc/rsyslog.conf      // Log profile

Add these two lines

local0.*/var/log/haproxy.log                // increase local0 Log file , The same below

local1.* /var/log/haproxy.log

             [root@Master etc]# vim/etc/sysconfig/rsyslog

Modify this trip :SYSLOGD_OPTIONS="-r -m 0"

              [root@Master etc]# servicersyslog restart       // Restart the logging process

After restarting, you can see /var/log/ There it is haproxy.log This file

5. start-up haproxy The server

[root@Master ~]#/usr/local/haproxy/sbin/haproxy -f /usr/local/haproxy/etc/haproxy.cfg

Take a look at the process

[root@Mastersbin]# netstat -tulpn | grep haproxy

tcp        0     0*   LISTEN     3451/./haproxy      

udp 0*                              3451/./haproxy      


Start successfully

6. Enable routing and forwarding

[root@Mastersbin]# echo "1">/proc/sys/net/ipv4/ip_forward


Slave Installation and Master The installation is the same , We just need to modify it Slave The host name of , I'm not going to repeat it here

Configure server Web1 And Web2

notes :httpd The server is only used for testing at this stage , To the back end Nginx After the server is set up , this httpd The server is shutting down !

1. modify Web1 And Web2 Host name

[root@localhost~]# hostname Web1.KPLD      // If necessary, it can be configured as a permanent host name

root@Web1 ~]#yum install httpd –y             // install apache

root@Web1 ~]# echo "Web1.KPLD">/var/www/html/index.html

[root@Web1 ~]#service httpd start The configuration is basically the same , You need to change the host name , establish index.html Change the content of the web page

[root@Web2 ~]# echo "Web2.KPLD">/var/www/html/index.html

One . test Haproxy Is the service installed successfully

Enter in the address bar Master/Slave Of ip Address ( there Master And Slave It's right keepalived What high availability says , Yes haproxy Load balancing doesn't matter )


Click refresh


Because we use polling , So it's polling between the two servers .

We can also make the graphical interface view Haproxy The server usage status configured under the service


In higher versions , Added more advanced features , such as , You can take the server off the shelf directly .

Keepalived Service configuration (Master)

We have achieved in haproxy Load balancing of , however , We can't publish two addresses at once , So use keepalived High availability features to achieve virtual ip, To achieve the goal of high availability .

1.keepalived Installation

[root@Master ~]# yum install kernel-devel openssl-devel popt-devel    // Install dependent files

[root@Master ~]# tar -zxvf keepalived-1.2.2.tar.gz

[root@Master ~]# cd keepalived-1.2.2

[root@Master keepalived-1.2.2]# ./configure--with-kernel-dir=/usr/src/kernels/2.6.32-279.el6.x86_64/

If the following prompt appears , That is success



Keepalivedversion       : 1.2.2

Compiler                 : gcc

Compilerflags           : -g -O2

Extra Lib                : -lpopt -lssl -lcrypto

Use IPVSFramework       : Yes

IPVS sync daemonsupport : Yes

IPVS uselibnl           : No

Use VRRPFramework       : Yes

Use Debugflags          : No


[root@Masterkeepalived-1.2.2]# make && make install

[root@Masterkeepalived-1.2.2]# cp keepalived/etc/init.d/keepalived.rh.init/etc/init.d/keepalived                            //copy The startup script

[root@Masterkeepalived-1.2.2]# chmod +x /etc/init.d/keepalived    // Executable rights

[root@Masterkeepalived-1.2.2]# chkconfig --level 35 keepalived on  // stay 35 Level boot up

[root@Masterkeepalived-1.2.2]# cp keepalived/etc/init.d/keepalived.sysconfig/etc/sysconfig/keepalived                        //copy Start initialization file

[root@Masterkeepalived-1.2.2]# cp /usr/local/sbin/keepalived /usr/sbin/    //copy Startup file

2.keepalived Service configuration

[root@Master keepalived-1.2.2]# vim/usr/local/etc/keepalived/keepalived.conf

! ConfigurationFile for keepalived

global_defs {

  notification_email {





  notification_email_from Alexandre.Cassen@firewall.loc


  smtp_connect_timeout 30

  router_id LVS_DEVEL


vrrp_instanceVI_1 {

   state MASTER

   interface eth2

   virtual_router_id 60

   priority 100

   advert_int 1

   authentication {

       auth_type PASS

       auth_pass 1111


   virtual_ipaddress {



virtual_server 80 {

  delay_loop 6

  lb_algo rr

  lb_kind DR


  persistence_timeout 50

  protocol TCP

   real_server 80 {

      weight 1

      TCP_CHECK {

          connect_timeout 3

          nb_get_retry 3

          delay_before_retry 3



  real_server 80 {

      weight 1

      TCP_CHECK {

          connect_timeout 3

          nb_get_retry 3

          delay_before_retry 3




Because of the startup script /etc/init.d/keepalived The configuration of , The default configuration file is config:/etc/keepalived/keepalived.conf, therefore , We're going to create a link

[root@Masterkeepalived]# ln -s /usr/local/etc/keepalived/keepalived.conf/etc/keepalived/keepalived.conf

start-up keepalived that will do

[root@Masterkeepalived-1.2.2]# service keepalived start

Keepalived To configure (Slave)

Slave Configuration and Master The configuration is basically the same , Just modify a few aspects , The full code is no longer posted here , Write only the differences

1. Will be a place state MASTER  It is amended as follows state BACKUP

2. Will be a place priority 100   It is amended as follows priority 50

Back end NGINX Server configuration

The company does not have this server , The company's real servers are IIS, I don't like windows Server side , So it's not used in the test IIS

1. Install dependent software

[root@localhost~]# yum install gcc openssl-devel pcre-devel zlib-devel

2. establish Nginx user

# useradd nginx-s /sbin/nologin

3. Unzip the software and install

# tar -zxvfnginx-1.4.2.tar.gz

# cd nginx-1.4.2

# ./configure--user=nginx --group=nginx --prefix=/usr/local/nginx--with-http_stub_status_module --with-http_ssl_module

# make&& make install

notes :--user,--group: Specify the start of each nginx Accounts and groups for

--with-http_stub_status_module: Install allow state module

--with-http_ssl_module: install ssl modular

4. start-up nginx

#/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf

notes : stay web1 And web2 Make a home page for each ,web1 The content of Web1.KPLD,web2 The content of Web2.KPLD

5. Configure static routing

Because our servers are internal networks , Can't communicate with external networks , however Keepalived We use DR Model , therefore , When a request arrives , It is returned directly to the client by the internal server , therefore , We must ensure that the internal host can be connected to the external network

stay /etc/rc.local Add the following route in the

  route add defaultgw

6. Except for the configuration in web Configuration on the server IP outside , Also need to do for intranet SNAT, It can be on two Keepalived Do on SNAT

iptables -t nat -A POSTROUTING -s -j SNAT--to

test Keepalived

Because the setting is Master by keepalived Primary server , So it can be Master Check to see if there is a virtual address on

[root@Master keepalived]# ip addr

You should be able to find the following

2: eth2:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

   link/ether 00:0c:29:3a:4d:dd brdff:ff:ff:ff:ff:ff

   inet brd scopeglobal eth2

   inet scope global eth2               // It can be seen that , fictitious IP appear .

   inet6 fe80::20c:29ff:fe3a:4ddd/64 scopelink

      valid_lft forever preferred_lft forever

You can also install ipvsadm To view the configuration

     [root@Masterkeepalived]# yum install ipvsadm

Use ipvsadm –L Check it out.

Now? Use virtual ip Address to view the web page


Now let's test high availability , hold Master Of keepalived stop, See if it's going to happen vip The transfer of

stay Master Up close keepalived service , see Slave On ip addr Does it appear? vip

[root@Master keepalived]# servicekeepalived stop

stay Slave Check out

[root@Slave ~]# ip addr

2: eth1:<BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen1000

   link/ether 08:00:27:a5:0e:93 brdff:ff:ff:ff:ff:ff

   inet brd scopeglobal eth1

   inet scope global secondaryeth1          // fictitious IP appear

   inet6 fe80::a00:27ff:fea5:e93/64 scope link

     valid_lft forever preferred_lftforever

Then through the browser to view the web page , To view the .

It's normal , High availability is also successful .

Configuration troubleshooting

Keepalived Configuration problems and solutions

A wrong :

configure: error:

 !!!OpenSSL is not properly installed on your system. !!!

 !!!Can not include OpenSSL headers files.            !!!

install openssl-devel

  yum install openssl-devel

Error 2 :

configure: error: Popt libraries isrequired

install popt Development kit

 yum install popt-devel

Error of three :

[root@Master keepalived-1.2.2]# servicekeepalived start

Starting Keepalived for LVS: /bin/bash:keepalived: command not found


resolvent :

[root@Slave ~]# cp/usr/local/sbin/keepalived /usr/sbin/

Error four :

    Ipvs Protocol not available

  Dec31 10:51:02 Slave Keepalived_healthcheckers: Registering Kernel netlink commandchannel

Dec 31 10:51:02 SlaveKeepalived_healthcheckers: Opening file '/etc/keepalived/keepalived.conf'.

Dec 31 10:51:02 SlaveKeepalived_healthcheckers: Configuration is using : 14529 Bytes

Dec 31 10:51:02 Slave Keepalived:Healthcheck child process(19805) died: Respawning

Dec 31 10:51:02 Slave Keepalived: StartingHealthcheck child process, pid=19807

Dec 31 10:51:02 SlaveKeepalived_healthcheckers: IPVS: Can't initialize ipvs: Protocol not available

Dec 31 10:51:02 SlaveKeepalived_healthcheckers: Netlink reflector reports IP added

resolvent :

Manual loading ip_vs modular

modprobe ip_vs

modprobe ip_vs_wrr

And add boot up

 #cat /etc/rc.local


/sbin/modprobe ip_vs_wrr

Error five :

 Master The server and Slave Virtual servers IP Address , When the primary server is disconnected , Cannot switch from server , The log shows

Dec 18 22:37:24 localhost Keepalived_vrrp:bogus VRRP packet received on eth1 !!!

Dec 18 22:37:24 localhost Keepalived_vrrp:VRRP_Instance(VI_1) ignoring received advertisment...

Dec 18 22:37:25 localhost Keepalived_vrrp:ip address associated with VRID not present in received packet :

Dec 18 22:37:25 localhost Keepalived_vrrp:one or more VIP associated with VRID mismatch actual MASTER advert

The main reason for this is virtual_router_id Incorrect configuration , By default 51, In some cases, however, changes are needed , For example, change it to :virtual_router_id 60 , The switch will succeed

本文为[osc_ sxdofc9c]所创,转载请带上原文链接,感谢

  1. [front end -- JavaScript] knowledge point (IV) -- memory leakage in the project (I)
  2. This mechanism in JS
  3. Vue 3.0 source code learning 1 --- rendering process of components
  4. Learning the realization of canvas and simple drawing
  5. gin里获取http请求过来的参数
  6. vue3的新特性
  7. Get the parameters from HTTP request in gin
  8. New features of vue3
  9. vue-cli 引入腾讯地图(最新 api,rocketmq原理面试
  10. Vue 学习笔记(3,免费Java高级工程师学习资源
  11. Vue 学习笔记(2,Java编程视频教程
  12. Vue cli introduces Tencent maps (the latest API, rocketmq)
  13. Vue learning notes (3, free Java senior engineer learning resources)
  14. Vue learning notes (2, Java programming video tutorial)
  15. 【Vue】—props属性
  16. 【Vue】—创建组件
  17. [Vue] - props attribute
  18. [Vue] - create component
  19. 浅谈vue响应式原理及发布订阅模式和观察者模式
  20. On Vue responsive principle, publish subscribe mode and observer mode
  21. 浅谈vue响应式原理及发布订阅模式和观察者模式
  22. On Vue responsive principle, publish subscribe mode and observer mode
  23. Xiaobai can understand it. It only takes 4 steps to solve the problem of Vue keep alive cache component
  24. Publish, subscribe and observer of design patterns
  25. Summary of common content added in ES6 + (II)
  26. No.8 Vue element admin learning (III) vuex learning and login method analysis
  27. Write a mini webpack project construction tool
  28. Shopping cart (front-end static page preparation)
  29. Introduction to the fluent platform
  30. Webpack5 cache
  31. The difference between drop-down box select option and datalist
  32. CSS review (III)
  33. Node.js学习笔记【七】
  34. Node.js learning notes [VII]
  35. Vue Router根据后台数据加载不同的组件(思考-&gt;实现-&gt;不止于实现)
  36. Vue router loads different components according to background data (thinking - & gt; Implementation - & gt; (more than implementation)
  37. 【JQuery框架,Java编程教程视频下载
  38. [jQuery framework, Java programming tutorial video download
  39. Vue Router根据后台数据加载不同的组件(思考-&gt;实现-&gt;不止于实现)
  40. Vue router loads different components according to background data (thinking - & gt; Implementation - & gt; (more than implementation)
  41. 【Vue,阿里P8大佬亲自教你
  42. 【Vue基础知识总结 5,字节跳动算法工程师面试经验
  43. [Vue, Ali P8 teaches you personally
  44. [Vue basic knowledge summary 5. Interview experience of byte beating Algorithm Engineer
  45. 【问题记录】- 谷歌浏览器 Html生成PDF
  46. [problem record] - PDF generated by Google browser HTML
  47. 【问题记录】- 谷歌浏览器 Html生成PDF
  48. [problem record] - PDF generated by Google browser HTML
  49. 【JavaScript】查漏补缺 —数组中reduce()方法
  50. [JavaScript] leak checking and defect filling - reduce() method in array
  51. 【重识 HTML (3),350道Java面试真题分享
  52. 【重识 HTML (2),Java并发编程必会的多线程你竟然还不会
  53. 【重识 HTML (1),二本Java小菜鸟4面字节跳动被秒成渣渣
  54. [re recognize HTML (3) and share 350 real Java interview questions
  55. [re recognize HTML (2). Multithreading is a must for Java Concurrent Programming. How dare you not
  56. [re recognize HTML (1), two Java rookies' 4-sided bytes beat and become slag in seconds
  57. 【重识 HTML ,nginx面试题阿里
  58. 【重识 HTML (4),ELK原来这么简单
  59. [re recognize HTML, nginx interview questions]
  60. [re recognize HTML (4). Elk is so simple