$theTitle=wp_title(" - ", false); if($theTitle != "") { ?> } else { ?> } ?>
by Andrew Johnstone
In: Linux
14 Jun 2009Recently we had an issue with one of our hosting providers load balancing (LVS), which resulted in some very small outages. As a result we decided to setup our own load balancing that we had full control over, and could manage ourselves. In addition to choosing a better suited weighting algorithm.
Each webserver is setup using ucarp an implementation of Common Address Redundancy Protocol (CARP) allowing failover of a single Virtual IP (VIP) for high availability. We bound multiple VIPs for each host as we noticed some HTTP 1.0 clients incorrectly sending the host address to the server.
There are many ways you can then proxy the webservers and load balance, however we decided to use haproxy. This can also be acheived by pound, apache mod_proxy, mod_backhand etc.
In order to setup ucarp & haproxy:
apt-get install -y haproxy ucarp
Modify /etc/network/interfaces giving each interface a unique ucarp-vid and adjust ucarp-advskew for weighting on each server (increment by one for each server) and set ucarp-master to yes if it is to be the master. Modify the configuration below appropriately.
# The primary network interface
auto eth0
iface eth0 inet static
address 10.10.10.2 # IP address of server
netmask 255.255.255.255
broadcast 10.10.10.10
gateway 10.10.10.1
ucarp-vid 3
ucarp-vip 10.110.10.20 # VIP to listen to
ucarp-password password
ucarp-advskew 10
ucarp-advbase 1
ucarp-facility local1
ucarp-master yes
iface eth0:ucarp inet static
address 10.10.10.20# VIP to listen to
netmask 255.255.255.255
To bring the interface up, simply run the following:
ifdown eth0; ifup etho0
ifdown eth0:ucarp; ifup eth0:ucarp
In order to configure haproxy:
sed -i -e ‘s/^ENABLED.*$/ENABLED=1/’ /etc/default/haproxy
Reconfigure apache to listen only on local interfaces (/etc/apache2/ports.conf):
So replace “Listen 80″ with
Listen 10.10.10.20:80
Listen 10.10.10.2:80
edit /etc/haproxy/haproxy.cfg
listen web 10.10.10.20:80
mode http
balance leastconn
stats enable
stats realm Statistics
stats auth stats:password
stats scope .
stats uri /stats?stats
#persist
server web1 10.10.10.2:80 check inter 2000 fall 3
server web2 10.10.10.3:80 check inter 2000 fall 3
server web3 10.10.10.4:80 check inter 2000 fall 3
server web4 10.10.10.5:80 check inter 2000 fall 3
server web5 10.10.10.6:80 check inter 2000 fall 3
Then restart haproxy with /etc/init.d/haproxy restart
Carp & HA Load Balancing
After changing your DNS to point to 10.10.10.20 you will be able to see the traffic balanced between the servers by going to the URL http://10.10.10.20/stats?stats with the credentials assigned above and see the bytes balanced between the servers listed.
Some other alternatives are:
I have been a developer for roughly 10 years and have worked with an extensive range of technologies. Whilst working for relatively small companies, I have worked with all aspects of the development life cycle, which has given me a broad and in-depth experience.
3 Responses to Load balancing with ucarp & haproxy
Ingus Neilands
June 27th, 2011 at 1:02 pm
Hey, we plan create the same configuration in our environment. Do you have any more comments about this – maybe some pros and cons?
Ingus
Maciej Wiercinski
June 29th, 2011 at 5:31 pm
Hi Ingus,
In above setup you need:
net.ipv4.ip_nonlocal_bind = 1
Andy also posted more verbose configuration (with dynamically spawned HAProxy instances, eliminating need for nonlocal_bind).
http://ajohnstone.com/achives/running-several-vips-on-the-same-interface-with-ucarp-and-haproxy/
andrew.johnstone
June 29th, 2011 at 5:52 pm
We run this configuration on a number of boxes, which works fairly well providing the boxes do not become completely saturated.
A few tips:
* Avoid putting on “iface eth0” if you need to restart the interface its very easy to mess up ifdown eth0 && ifup eth0.
* We have since changed the skew to avoid our monitoring of ucarp tripping…
iface eth0 inet static ucarp-advskew 0 ucarp-advbase 2 .... iface eth0:ucarp inet static iface eth0:1 inet static ucarp-advskew 0 ucarp-advbase 3 .... iface eth0:1:ucarp inet static iface eth0:2 inet static ucarp-advskew 0 ucarp-advbase 3 .... iface eth0:2:ucarp inet static iface eth0:3 inet static ucarp-advskew 0 ucarp-advbase 3 ....We monitor the ucarp instances with mon.
And HA Proxy with
#!/bin/bash logfile="/var/log/mon/haproxy.log" killucarp() { sudo /etc/network/local/clean-vip.sh $1; } haproxy_status() { # try for three times before raising an alarm retry=0; statusok=0; # PID file while [ $retry -lt 3 ]; do if [ -f $2 ] ; then statusok=1; break; fi; echo "No PID file, retry $retry"; let retry=$retry+1; sleep 1; done; if [ $statusok -ne 1 ] ; then echo "HAProxy $2 not running (no PID file)"; killucarp "$1" "$2"; exit 3; fi; retry=0; statusok=0; while [ $retry -lt 3 ] ; do pid=$(cat $2); if [ "$pid" -gt 0 2> /dev/null ] ; then if ps --no-headers p $pid | grep haproxy | grep -q "$2"; then statusok=1; break; else echo "Process $pid does not exist or doesnt match definition $2, retry $retry"; echo "--- `date` --- Process $pid does not exist or doesnt match definition $2, retry $retry" >> $logfile; ps axf >> $logfile; fi; else echo "PID file invalid, retry $retry"; fi; let retry=$retry+1; sleep 1; done; if [ $statusok -ne 1 ] ; then echo "HAProxy $2 not running (no PID file)"; killucarp "$1" "$2"; exit 3; fi; # echo "HaProxy running"; exit 0; } if [ $# -ne 2 ] ; then echo "Usage : $(basename $0) [ IP ] [ pid file ]"; exit 1; fi; if ! ( ip addr show | grep "inet $1" -q ) ; then # no ip , nothing to worry about # echo "no ip"; exit 0; fi; haproxy_status $1 $2;