Verifying Newly Added Servers

Run consul info on the leader. In this example the leader is server1.example.com, it may be different in your case.

root@server1:~# consul info
...
raft:
    applied_index = 2830
    commit_index = 2830
    fsm_pending = 0
    last_contact = never
    last_log_index = 2830
    last_log_term = 7729
    last_snapshot_index = 0
    last_snapshot_term = 0
    num_peers = 3
    state = Leader
    term = 7729
...

This shows various information about the state of Raft. In particular the last_log_index shows the last log that is on disk.

The same info command can be run on the newly added server to see how far behind it is. Eventually the server will catch up, and the values should match.

root@server4:~# consul info
...
raft:
    applied_index = 2830
    commit_index = 2830
    fsm_pending = 0
    last_contact = 91.32694ms
    last_log_index = 2830
    last_log_term = 7729
    last_snapshot_index = 0
    last_snapshot_term = 0
    num_peers = 3
    state = Follower
    term = 7729
...

The same thing can be done on the 5th server node.

You can see the members of the cluster by asking Consul for its members on any of the machines:

root@server4:~# consul members