redis 4.0.9 cluster + failover详解大数据

os: centos7.4
redis:4.0.9
ruby:2.3.4

ip 规划

redis1    192.168.56.101 
redis2    192.168.56.102

用两台虚拟机模拟6个redis节点。
分别为
192.168.56.101 的 9001、9002、9003 端口，
192.168.56.102 的 9001、9002、9003 端口。

连接redis

# redis-cli -h 192.168.56.101 -p 9001 -c 
192.168.56.101:9001> 
192.168.56.101:9001> 
192.168.56.101:9001> cluster info 
cluster_state:ok 
cluster_slots_assigned:16384 
cluster_slots_ok:16384 
cluster_slots_pfail:0 
cluster_slots_fail:0 
cluster_known_nodes:6 
cluster_size:3 
cluster_current_epoch:6 
cluster_my_epoch:1 
cluster_stats_messages_ping_sent:23175 
cluster_stats_messages_pong_sent:25275 
cluster_stats_messages_sent:48450 
cluster_stats_messages_ping_received:25270 
cluster_stats_messages_pong_received:23175 
cluster_stats_messages_meet_received:5 
cluster_stats_messages_received:48450 
 
192.168.56.101:9001> cluster nodes 
cf22edbc6463b8b31536819297ac50e9e39175ad 192.168.56.102:9003@19003 slave 8240d131c58c8b687c11462e03825bdba00af53b 0 1526649623970 6 connected 
8240d131c58c8b687c11462e03825bdba00af53b 192.168.56.101:9001@19001 myself,master - 0 1526649623000 1 connected 0-5460 
12e2353c9f0b085d2684da46ba0dc69575d68603 192.168.56.102:9001@19001 slave e38d78fc68b69d4e5f347cf96f31a9cb6a02dccf 0 1526649621000 7 connected 
2b48fd08a8ec55eba5bfa0d163d02c8462637302 192.168.56.102:9002@19002 slave 0bdd8387a662984e95f5103522f52e75eceaab69 0 1526649624972 5 connected 
0bdd8387a662984e95f5103522f52e75eceaab69 192.168.56.101:9002@19002 master - 0 1526649622000 2 connected 10923-16383 
e38d78fc68b69d4e5f347cf96f31a9cb6a02dccf 192.168.56.101:9003@19003 master - 0 1526649622963 7 connected 5461-10922 

kill某个redis master 进程

# ps -ef|grep -i redis 
root     11301     1  0 21:09 ?        00:00:00 redis-server 192.168.56.101:9001 [cluster] 
root     11330     1  0 21:10 ?        00:00:00 redis-server 192.168.56.101:9002 [cluster] 
root     11340     1  0 21:10 ?        00:00:00 redis-server 192.168.56.101:9003 [cluster] 
root     11358  1851  0 21:11 pts/0    00:00:00 redis-cli -h 192.168.56.101 -p 9001 -c 
root     11484  5873  0 21:21 pts/1    00:00:00 grep --color=auto -i redis 
 
# kill 11301 
 
192.168.56.101:9002> cluster nodes 
8240d131c58c8b687c11462e03825bdba00af53b 192.168.56.101:9001@19001 master,fail - 1526649753764 1526649753061 1 disconnected 
12e2353c9f0b085d2684da46ba0dc69575d68603 192.168.56.102:9001@19001 slave e38d78fc68b69d4e5f347cf96f31a9cb6a02dccf 0 1526649779186 7 connected 
0bdd8387a662984e95f5103522f52e75eceaab69 192.168.56.101:9002@19002 myself,master - 0 1526649778000 2 connected 10923-16383 
e38d78fc68b69d4e5f347cf96f31a9cb6a02dccf 192.168.56.101:9003@19003 master - 0 1526649776000 7 connected 5461-10922 
2b48fd08a8ec55eba5bfa0d163d02c8462637302 192.168.56.102:9002@19002 slave 0bdd8387a662984e95f5103522f52e75eceaab69 0 1526649777173 5 connected 
cf22edbc6463b8b31536819297ac50e9e39175ad 192.168.56.102:9003@19003 master - 0 1526649778179 8 connected 0-5460 

可以看到 192.168.56.101:9001 现在是 master,fail 状态

kill 之前redis的状态

master 192.168.56.101:9001、192.168.56.101:9002、192.168.56.101:9003 
slave  192.168.56.102:9001、192.168.56.102:9002、192.168.56.102:9003

kill 之后redis的状态

master 192.168.56.101:9002、192.168.56.101:9003、192.168.56.102:9003 
slave  192.168.56.102:9001、192.168.56.102:9002 
 
master,fail 192.168.56.101:9001

可以看出，192.168.56.102:9003 充当了新的master，也可以推断kill之前，192.168.56.102:9003 是 192.168.56.101:9001 的slave

从输出求证，确实是这样

cf22edbc6463b8b31536819297ac50e9e39175ad 192.168.56.102:9003@19003 slave 8240d131c58c8b687c11462e03825bdba00af53b 0 1526649623970 6 connected 
8240d131c58c8b687c11462e03825bdba00af53b 192.168.56.101:9001@19001 myself,master - 0 1526649623000 1 connected 0-5460 
 
注意 8240d131c58c8b687c11462e03825bdba00af53b

redis的slave在master down后，可以自动提升为master。这个很重要。

重新启动被kill的那个redis

# redis-server /usr/redis/redis-cluster-data/9001/redis.conf 
[root@pgsql1 9001]# redis-cli -h 192.168.56.101 -p 9001 -c 
192.168.56.101:9001> 
192.168.56.101:9001> cluster nodes 
12e2353c9f0b085d2684da46ba0dc69575d68603 192.168.56.102:9001@19001 slave e38d78fc68b69d4e5f347cf96f31a9cb6a02dccf 0 1526650968543 7 connected 
2b48fd08a8ec55eba5bfa0d163d02c8462637302 192.168.56.102:9002@19002 slave 0bdd8387a662984e95f5103522f52e75eceaab69 0 1526650969546 5 connected 
cf22edbc6463b8b31536819297ac50e9e39175ad 192.168.56.102:9003@19003 master - 0 1526650967000 8 connected 0-5460 
8240d131c58c8b687c11462e03825bdba00af53b 192.168.56.101:9001@19001 myself,slave cf22edbc6463b8b31536819297ac50e9e39175ad 0 1526650967000 1 connected 
e38d78fc68b69d4e5f347cf96f31a9cb6a02dccf 192.168.56.101:9003@19003 master - 0 1526650969000 7 connected 5461-10922 
0bdd8387a662984e95f5103522f52e75eceaab69 192.168.56.101:9002@19002 master - 0 1526650966534 2 connected 10923-16383 
192.168.56.101:9001> set peiyb1 abc 
-> Redirected to slot [6161] located at 192.168.56.101:9003 
OK 
192.168.56.101:9001>  

此时，redis各节点的状态为

master 192.168.56.101:9002、192.168.56.101:9003、192.168.56.102:9003 
slave  192.168.56.102:9001、192.168.56.102:9002、192.168.56.101:9001

可以看出，新启动的redis 192.168.56.101:[email protected] 状态为slave，并没有抢夺master的权限。
此外，在slave上操作 set 时，能自动计算 hash slot ，抛给对应的master执行。