HA Perodic Synchronization

[#1]

Article

Components	FortiGate HA Cluster (FortiOS 3.0)
Description	How HA periodic synchronization looks for synchronization problems and fixes them.
Periodic synchronization	Incremental synchronization makes sure that as an administrator makes configuration changes, the configurations of all cluster units remain the same. However, a number of factors could cause one or more cluster units to go out of sync with the primary unit. For example, if you add a new unit to a functioning cluster, the configuration of this new unit will not match the configuration of the other cluster units. Its not practical to use incremental synchronization to change the configuration of the new unit. Periodic synchronization is a mechanism that looks for synchronization problems and fixes them. Every minute the cluster compares the configuration file checksum of the primary unit with the configuration file checksums of each of the subordinate units. If all subordinate unit checksums are the same as the primary unit checksum, all cluster units are considered synchronized. If one or more of the subordinate unit checksums is not the same as the primary unit checksum, the subordinate unit configuration is considered out of sync with the primary unit. The checksum of the out of sync subordinate unit is checked again every 15 seconds. This re-checking occurs in case the configurations are out of sync because an incremental configuration sequence has not completed. If the checksums do not match after 5 checks the subordinate unit that is out of sync retrieves the configuration from the primary unit. The subordinate unit then reloads its configuration and resumes operating as a subordinate unit with the same configuration as the primary unit. The configuration of the subordinate unit is reset in this way because when a subordinate unit configuration gets out of sync with the primary unit configuration there is no efficient way to determine what the configuration differences are and to correct them. Resetting the subordinate unit configuration becomes the most efficient way to resynchronize the subordinate unit. Synchronization requires that all cluster units run the same FortiOS firmware build. If some cluster units are running different firmware builds, then unstable cluster operation may occur and the cluster units may not be able to synchronize correctly.
FortiOS v3.0 console out of synch messages	If you connect to the console of a subordinate unit that is out of synchronization with the primary unit, messages similar to the following are displayed. === Console Output (Synchronization Error Messages) === slave is not in sync with master, sequence:0. (type 0x3) slave is not in sync with master, sequence:1. (type 0x3) slave is not in sync with master, sequence:2. (type 0x3) slave is not in sync with master, sequence:3. (type 0x3) slave is not in sync with master, sequence:4. (type 0x3) global compared not matched Wait for system rebooting... slave is not in sync with master, sequence:0. (type 0x52) slave is not in sync with master, sequence:1. (type 0x52) slave is not in sync with master, sequence:2. (type 0x52) slave is not in sync with master, sequence:3. (type 0x52) slave is not in sync with master, sequence:4. (type 0x52) If synchronization problems occur the console message sequence may be repeated over and over again. The messages all include a type value (in the example type 0x3 and type 0x52). The type value can help Fortinet Support diagnose the synchronization problem. See FortiOS v3.0 HA out of sync messages and the objects that they reference for a complete list of sync object messages (including their type codes) and the object that each message references.