Upgrading Cluster members in SALB(Session Aware Load Balancing) Cluster

[plokesh]

Purpose
The following document outlines the process to upgrade the firmware on FortiGate 5001c blades in an SALB cluster.

Setup:
Inter-chassis Session Aware Load Balancing.

Hardware used:
2*5140B each containing 1 FortiController 5103B and 10 FortiGate 5001c
Current state:  Chassis 1 is master and Chassis 2 in slave mode.

Scope


Expectations, Requirements
Prerequisites: CLI(ssh) session to the FortiController, FortiGate management IP and https into FortiGate's management IP

Expected state after the upgrade: Chassis 1 to be the master and chassis 2 to be in slave state.
HA Override is NOT recommended for inter-chassis HA.

Configuration
Upgrade Process:
 
1) First, make sure if there are any chassis that are forced to be in slave state and if so do a ”diag sys ha force-slave-state clear” from the Forticontroller’s CLI. This should clear any forced slave status that the chassis are in.

If override is enabled under ha settings of the Forticontroller, we will have a failover.
HA Override is NOT recommended for inter-chassis HA.

2) Upload the firmware from the GUI of the Config Sync Master FortiGate and Click on upgrade. This will send the firmware to the blades in chassis 2, upgrade them and then reboot them.

3) During step 2, check in the CLI window for the messages regarding slaves being upgraded and the countdown of when it is completed. Once this is done, it will ask you to force a manual failover.

"All members of the slave chassis are ready for traffic.
You may switch over the master chassis now."

4) Use the command "diag sys ha force-slave-state by-chassis 5 1" from the Chassis 1 FortiController’s CLI to make chassis 1 slave. This command will only work from the active chassis's FortiController

5) At this time, we would have the chassis 1’s blades upgrading automatically. They would reboot and rejoin the cluster.

6)After all the blades are upgraded, check for the checksum to be the same and also verify that the build number is correct.

7)Execute the command "diag sys ha force-slave-state clear" from the FortiController’s CLI

8) Verify that the primary chassis(chassis 1) is the active chassis after the upgrade.
(“diag sys ha status” from FortiController’s CLI)




Verification
The sample output from the Config Sync master FortiGate should look like this.

FGT-01 (global) #
Checking new firmware integrity ... pass
Send image to slave.
Wait for slave to upgrade.
............................................................
Image upgrade in progress. 19 minutes before aborting.
............................................................
Image upgrade in progress. 18 minutes before aborting.
.......................................................
All members of the slave chassis are up.
.....
Image upgrade in progress. 17 minutes before aborting.
.
All members of the slave chassis are ready for traffic.
You may switch over the master chassis now.
..................................
Master chassis switchover is done.
Time to upgrade myself now.



Firmware upgrade in progress ...
Done.


The system is going down NOW !!

Please stand by while rebooting the system.
Restarting system.
FortiGate-5001C (18:16-09.17.2012)
Ver:04000004
Serial number:FG-5KCXXXXXXX
RAM activation
CPU(00:000206d7 bfebfbff): MP initialization
CPU(01:000206d7 bfebfbff): MP initialization
CPU(02:000206d7 bfebfbff): MP initialization
CPU(03:000206d7 bfebfbff): MP initialization
CPU(04:000206d7 bfebfbff): MP initialization
CPU(05:000206d7 bfebfbff): MP initialization
CPU(06:000206d7 bfebfbff): MP initialization
CPU(07:000206d7 bfebfbff): MP initialization
CPU(08:000206d7 bfebfbff): MP initialization
CPU(09:000206d7 bfebfbff): MP initialization
CPU(0a:000206d7 bfebfbff): MP initialization
CPU(0b:000206d7 bfebfbff): MP initialization
CPU(0c:000206d7 bfebfbff): MP initialization
CPU(0d:000206d7 bfebfbff): MP initialization
CPU(0e:000206d7 bfebfbff): MP initialization
CPU(0f:000206d7 bfebfbff): MP initialization
Total RAM: 32768MB
Enabling cache...Done.
Scanning PCI bus...Done.
Allocating PCI resources...Done.
Enabling PCI resources...Done.
Zeroing IRQ settings...Done.
Verifying PIRQ tables...Done.
Boot up, boot device capacity: 30533MB.
Press any key to display configuration menu...
......

Reading boot image 1378629 bytes.
Initializing firewall...
System is starting...