The Strange Case Of Sentinel And ACL

Argument

Strange enough! Sentinel has intriguing bearing on ACL...

I. System setup

According to instructions on RU330, to disable default user in acl.conf

user default off

Have a glimpse of documentation:

# If the master is password protected (using the "requirepass" configuration
# directive below) it is possible to tell the replica to authenticate before
# starting the replication synchronization process, otherwise the master will
# refuse the replica request.
#
# masterauth <master-password>
#
# However this is not enough if you are using Redis ACLs (for Redis version
# 6 or greater), and the default user is not capable of running the PSYNC
# command and/or other commands needed for replication. In this case it's
# better to configure a special user to use with replication, and specify the
# masteruser configuration as such:
#
# masteruser <username>
#
# When masteruser is specified, the replica will authenticate against its
# master using the new AUTH form: AUTH <username> <password>.

Thenceforth, admin user is setup in acl.conf and add masteruser directive to redis.conf of Replica1 and Replica2. It's just fine in basic replication.

However, this is not the case when sentinels come in sight... According to instructions on RU301

# In order for Sentinels to connect to Redis server instances when they are configured with requirepass, the Sentinel configuration must include the sentinel auth-pass directive.
sentinel auth-pass <myprimary> <master-password>

But there no such thing as sentinel auth-user subcommand.

Which means it is forbidden to tamper default user or else failover mechanism would simply fail. Stranger than strange! As a matter fact, when using default user with ACL together, it would left the system unprotected...

Corollary

Don't use ACL, secure default user with strong password. ACL GENPASS

II. Check to see if failover success

make up 
make master 
docker-compose stop primary
make ps 
make master

III. Check to see if default user is password protected

redis-cli -p 6381
ping
auth <master-password>
info replication
exit

IV. Check to see if we can manually failover a server

make sentinel
SENTINEL SENTINELS myprimary
SENTINEL REPLICAS myprimary
SENTINEL GET-MASTER-ADDR-BY-NAME myprimary
SENTINEL FAILOVER myprimary
SENTINEL GET-MASTER-ADDR-BY-NAME myprimary
exit

V. Re-check ex-primary status

make up 
make replication
make role

When our primary was put to death, replica2 is promoted as new primary, replica1 points to new primary flatteringly. As soon as ex-primary revives, for a very short of time, he still believes that he is on the throne of primary even though sentinels report that new primary is elected. Wandering and lingering for a while, he comes to realize the existence of new primary... Daunted by the situation, he joins the new primary as slave without a murmur...

Wel-a-day!, life is capricious...

VI. Restore config

make down 
make reset

EOF (2024/07/08)