Avatar
Cluster VIP:172.16.1.213
TODO: VRRP failover setup for load balancer (keepalived)
ALSO: It looks like I configured net.ipv4.ip_forward=1 in load balancer but failed to document it, check this when doing a new load balancer.
#####################Edge01 Configuration (loadbalancer 1)#########################
//keepalived.conf
! Configuration File for keepalived
global_defs {
# notification_email {
# me[at]example.net
# }
# notification_email_from not-replay[at]example.net
# smtp_server 192.168.200.1
# smtp_connect_timeout 30
router_id LVS_DEVEL
}
vrrp_instance VI_1 {
state MASTER
interface eth0
virtual_router_id 51
priority 100
advert_int 1
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
# 172.16.1.213
# 172.16.1.254
}
}
virtual_server 172.16.1.213 8098 {
delay_loop 6
lb_algo wlc
lb_kind DR
nat_mask 255.255.255.0
persistence_timeout 50
protocol TCP
real_server 172.16.1.113 8098 {
weight 1
TCP_CHECK {
connect_timeout 3
connect_port 8098
}
}
real_server 172.16.1.112 8098 {
weight 100
TCP_CHECK {
connect_timeout 3
connect_port 8098
}
}
real_server 172.16.1.111 8098 {
weight 100
TCP_CHECK {
connect_timeout 3
connect_port 8098
}
}
}
//ifconfig (secondary address for VIP)
[acool@edge01 ~]$ ifconfig
eth0 Link encap:Ethernet HWaddr 00:24:54:3A:66:4D
inet addr:172.16.1.200 Bcast:172.16.1.255 Mask:255.255.255.0
inet6 addr: fe80::224:54ff:fe3a:664d/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:385 errors:0 dropped:0 overruns:0 frame:0
TX packets:243 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:37042 (36.1 KiB) TX bytes:35302 (34.4 KiB)
eth0:0 Link encap:Ethernet HWaddr 00:24:54:3A:66:4D
inet addr:172.16.1.213 Bcast:172.16.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:240 (240.0 b) TX bytes:240 (240.0 b)
[acool@edge01 ~]$ ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 00:24:54:3a:66:4d brd ff:ff:ff:ff:ff:ff
inet 172.16.1.200/24 brd 172.16.1.255 scope global eth0
inet 172.16.1.213/24 brd 172.16.1.255 scope global secondary eth0:0
inet6 fe80::224:54ff:fe3a:664d/64 scope link
valid_lft forever preferred_lft forever
#####################Real Server 2 ( 99% same output for other real servers) #########################
//DR-IPVS file
#!/bin/bash
VIP=172.16.1.213
host=`/bin/hostname`
/sbin/ifconfig lo down
/sbin/ifconfig lo up
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
/sbin/ifconfig lo:1 $VIP netmask 255.255.255.255 up
/sbin/route add -host $VIP dev lo:1
exit 1
//ifconfig
[acool@app02 ~]$ ifconfig
ens1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST> mtu 1500
inet 172.16.1.112 netmask 255.255.255.0 broadcast 172.16.1.255
inet6 fe80::216:d4ff:fe45:8ae1 prefixlen 64 scopeid 0x20<link>
ether 00:16:d4:45:8a:e1 txqueuelen 1000 (Ethernet)
RX packets 564 bytes 53931 (52.6 KiB)
RX errors 0 dropped 0 overruns 0 frame 1
TX packets 428 bytes 57911 (56.5 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
device interrupt 16
lo: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 127.0.0.1 netmask 255.0.0.0
inet6 ::1 prefixlen 128 scopeid 0x10<host>
loop txqueuelen 0 (Local Loopback)
RX packets 13 bytes 1360 (1.3 KiB)
RX errors 0 dropped 0 overruns 0 frame 0
TX packets 13 bytes 1360 (1.3 KiB)
TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0
lo:1: flags=73<UP,LOOPBACK,RUNNING> mtu 65536
inet 172.16.1.213 netmask 255.255.255.255
loop txqueuelen 0 (Local Loopback)
[acool@app02 ~]$
[acool@app02 ~]$
[acool@app02 ~]$ ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet 172.16.1.213/32 scope global lo:1
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: ens1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP qlen 1000
link/ether 00:16:d4:45:8a:e1 brd ff:ff:ff:ff:ff:ff
inet 172.16.1.112/24 brd 172.16.1.255 scope global ens1
valid_lft forever preferred_lft forever
inet6 fe80::216:d4ff:fe45:8ae1/64 scope link
valid_lft forever preferred_lft forever
edge01 verification (keealived load balancer):
[acool@edge01 ~]$
[acool@edge01 ~]$
[acool@edge01 ~]$ sudo ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 172.16.1.213:8098 wlc persistent 50
-> 172.16.1.111:8098 Route 100 1 0
-> 172.16.1.112:8098 Route 100 0 0
-> 172.16.1.113:8098 Route 1 0 0
[acool@edge01 ~]$
[acool@edge01 ~]$
[acool@edge01 ~]$
Riak ring verification:
[acool@app02 ~]$
[acool@app02 ~]$
[acool@app02 ~]$ sudo riak-admin member_status
================================= Membership ==================================
Status Ring Pending Node
-------------------------------------------------------------------------------
valid 32.8% -- 'riak@172.16.1.111'
valid 32.8% -- 'riak@172.16.1.112'
valid 34.4% -- 'riak@172.16.1.113'
-------------------------------------------------------------------------------
Valid:3 / Leaving:0 / Exiting:0 / Joining:0 / Down:0
[acool@app02 ~]$
[acool@app02 ~]$
[acool@app02 ~]$
[acool@app02 ~]$ sudo riak-admin ring-status
================================== Claimant ===================================
Claimant: 'riak@172.16.1.113'
Status: up
Ring Ready: true
============================== Ownership Handoff ==============================
No pending changes.
============================== Unreachable Nodes ==============================
All nodes are up and reachable
[acool@app02 ~]$
Finally, storing data in riak:
Angels-Mac-mini:~ webmaster$ curl -v -XPUT 172.16.1.213:8098/buckets/test-bucket/keys/friends?returnbody=true -H "Content-Type: application/json" -d '{"Name":"Angel Cool"}'Fetching data:
http://172.16.1.213:8098/buckets/test-bucket/keys/friends
Or using any of the real servers:
http://172.16.1.111:8098/buckets/test-bucket/keys/friends
Adding another node at a later time:
[webmaster@dev ~]$ sudo riak-admin cluster join riak@172.16.1.111
[webmaster@dev ~]$ sudo riak-admin cluster plan
[webmaster@dev ~]$ sudo riak-admin cluster commit
If plan does not look good:
[webmaster@dev ~]$ sudo riak-admin cluster clear
Riak must be bound to all interfaces: 0.0.0.0:8098
[aesteban@localhost ~]$
[aesteban@localhost ~]$ riak-admin status | grep riak_kv_version
riak_kv_version : <<"1.4.7-0-g5daeedd">>
[aesteban@localhost ~]$
Comments
Comments are not available for this entry.