FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
rmetzger
Staff
Staff
Article Id 194172
Description
This article provides an overview of the capabilities of the Shelf Manager card which is used to monitor and manage a FortiGate 5140 chassis.

Solution

   Fortinet documentation references


•    Fortinet Document Web page : http://docs.fortinet.com/fgt_5000.html
-    FortiGate-5000 Series Introduction
-    FortiGate-5140 Chassis Guide

    General information


A FG5140 chassis is shipped with and managed by a Shelf Manager. It is possible to purchase a second Shelf Manager to allow for a fully redundant configuration.

The Shelf Manager is responsible for the chassis management, it:
•    Controls the chassis and its associated components which are called FRU units (Field Replaceable Unit).  The FRU units comprise the PEM power modules, an Alarm panel, the Fan trays and Filters as well as any Fortinet Blades inserted in the chassis. The chassis itself and the FRU units contain sensors which provide information on such items as voltage, temperature and fan speeds. 
•    Monitors and logs alarms: Sensors are associated to different alarm threshold values. When a threshold is reached an alarm is generated and the information stored in the Shelf Manager log.
•    Monitors the FRUs and takes the appropriate action for example, the Shelf Manager can dynamically raise fan speeds if critical temperature thresholds are exceeded and lower them when sensors return to normal values.

The communication between the Shelf Manager and the chassis components is performed through an IPMB bus. Each component has bus access and has a unique address called a “slave IPMB address”.  Each sensor is identified by a sensor ID and its FRU IPMB bus address.  These two identifiers are used by the Shelf Manager to identify the individual FRU sensor which is sending information.

    Accessing the shelf manager

The Shelf Manager can be accessed via the serial port, or by IP. To manage the unit via IP you need to configure an IP address as well as manually configure the physical device. Please refer to the “FortiGate-5140 Chassis Guide” for more information

The Shelf Manager runs a dedicated management operating system, produced by Pigeon Point Software (http://www.pigeonpoint.com).

    Log Alarms and Events

When an alarm is raised, the event is logged in the chassis log stored on the Shelf Manager.
  - Use the command “clia sel” to display all event logs.
  -The log can be cleared with command “clia sel clear

When an alarm condition occurs, a log entry with the keyword “asserted” is raised, once the situation returns to normal a log entry with the keyword “deasserted” is raised.

There will also be event log entries which are not related to alarm conditions.  For example, when a blade is inserted or removed from the chassis, it will generate change state change events however no alarm would be raised in this instance.

Log Analysis

-    extract the log “slave IPMB address” and “sensor ID
-    use the command “clia sensordata” to dump the sensors details
-    use “slave IPMB address” and “sensor ID” pair to identify the sensor that has caused the alarm.

When an alarm has been raised, even if the alarm conditions are no longer present, the corresponding alarm LED will be illuminated on the SAP panel. The alarm indication will remain until it is manually cleared by an operator using the command “clia alarm clear”.


Example Alarm:

# clia sel

Pigeon Point Shelf Manager Command Line Interpreter
.../...
0x00BE: Event: at Mar  2 03:49:29 2010; from:(0x10,0,0); sensor:(0x04,8); event:0x1(asserted): "Upper Critical", Threshold: 0x3b, Reading: 0xff
0x00BF: Event: at Mar  2 03:49:31 2010; from:(0x10,0,0); sensor:(0x04,8); event:0x1(deasserted): "Upper Critical", Threshold: 0x3b, Reading: 0x16



# clia sensordata

Pigeon Point Shelf Manager Command Line Interpreter
.../...
10: LUN: 0, Sensor # 8 ("Fan Tach. 1")
    Type: Threshold (0x01), "Fan" (0x04)
    Belongs to entity (0x1e, 0x60)
    Status: 0xc0
        All event messages enabled from this sensor
        Sensor scanning enabled
        Initial update completed
    Raw data: 22 (0x16)
    Processed data: 7704.160247 RPM
    Status: 0x00


The interpretation of these entires is the following:

An alarm condition occurred for 2 seconds due to a tachometer sensor on the fan tray 0 (left hand side). The threshold value was reached hence it generated a major alarm.

The sensors use internal values (''Raw Data'') for thresholds, however the corresponding meaningful value, is provided as “Processed Data” from the command “clia sensordata

It is possible to obtain the alarm status via snmp on the shelf manager IP address - see SNMP section.

    SNMP Management

It is possible to poll each sensor on the chassis using SNMP via the Shelf Manager IP address.  You will need to use SNMP version 2c and the community string ‘public’.

A MIB for the chassis name “PPS-SENTRY-MIB.dat” is available which contains the OID descriptions required to poll the Shelf Manager. The OID construction to obtain a value from a particular sensor is detailed below.

To illustrate with an example, we will extract the “Current State Mask” information from the “TELCO Alarm” sensor. This value returns the current state of the Alarm of the SAP module (corresponding to the LED indicator state located on the chassis front panel).

20: LUN: 0, Sensor # 131 ("TELCO Alarms")
    Type: Discrete (0x6f), "OEM reserved" (0xdf)
    Belongs to entity (0xf0, 0x01): FRU # 0
    Status: 0xc0
        All event messages enabled from this sensor
        Sensor scanning enabled
        Initial update completed
    Sensor reading: 0x00
    Current State Mask 0x0002             <<<<< This is a major alarm



1. Sensor-entry entry point in the vendor specific MIB:

The sensor object entry point for the OID is:OID= ‘.1.3.6.1.4.1.16394.2.1.1.3.1’  corresponding to  “SNMPv2-SMI::enterprises.pps.products.chassis-management.ipm-sentry-shmm.sensor.sensor-entry

2. Type of information to extract from the sensor following the sensor structure in the MIB:

Val.    Information type                        Format
----------------------------------------------------------------------

 1   sensor-index                               INTEGER,

 2   sensor-sdr-version                         INTEGER,

 3   sensor-record-type                         INTEGER,

 4   sensor-owner-id                            INTEGER,

 5   sensor-owner-lun                           INTEGER,

 6   sensor-number                              INTEGER,

 7   sensor-entity-instance                     INTEGER,

 8   sensor-entity-id                           INTEGER,

 9   sensor-initialization                      INTEGER,

10   sensor-capabilities                        INTEGER,

11   sensor-type                                INTEGER,

12   sensor-event                               INTEGER,

13   sensor-assertion-event-mask                INTEGER,

14   sensor-deassertion-event-mask              INTEGER,

15   sensor-mask                                INTEGER,

16   sensor-unit1                               INTEGER,

17   sensor-unit2                               INTEGER,

18   sensor-unit3                               INTEGER,

19   sensor-linearization                       INTEGER,

20   sensor-M                                   INTEGER,

21   sensor-tolerance                           INTEGER,

22   sensor-B                                   INTEGER,

23   sensor-accuracy                            INTEGER,

24   sensor-accuracy-exp                        INTEGER,

25   sensor-R-exp                               INTEGER, 

26   sensor-B-exp                               INTEGER,

27   sensor-characteristic-flags                INTEGER, 

28   sensor-reading                             INTEGER,

29   sensor-processed-reading                   DisplayString,

30   sensor-nominal-reading                     INTEGER,

31   sensor-nominal-maximum                     INTEGER,

32   sensor-nominal-minimum                     INTEGER,

32   sensor-maximum-reading                     INTEGER,

33   sensor-minimum-reading                     INTEGER,

34   sensor-upper-non-recoverable-threshold     INTEGER,

35   sensor-upper-critical-threshold            INTEGER,

36   sensor-upper-non-critical-threshold        INTEGER,

37   sensor-lower-non-recoverable-threshold     INTEGER,

38   sensor-lower-critical-threshold            INTEGER,

39   sensor-lower-non-critical-threshold        INTEGER,

40   sensor-positive-going-threshold-hysteresis INTEGER,

41   sensor-negative-going-threshold-hysteresis INTEGER,

42   sensor-id-string                           DisplayString,

43   sensor-entire-sensor-data                  OCTET STRING



So the OID is now:

OID= ‘.1.3.6.1.4.1.16394.2.1.1.3.1.29.’

3.  IPMB slave address in decimal format:
For the TELCO Alarms : 0x20 is 32 in decimal so we have :
OID= ‘.1.3.6.1.4.1.16394.2.1.1.3.1.29.32.’

4. Sensor ID in decimal format:
The sensor ID can easily be gathered from the shelf manager command ‘cli sensordata’ where the sensor name and ID is explicit.
OID=‘.1.3.6.1.4.1.16394.2.1.1.3.1.29.32.131.’

Lastly poll the Telco alarm state with the OID
OID=‘.1.3.6.1.4.1.16394.2.1.1.3.1.29.32.131


To obtain the status of the alarm with snmpget:

snmpget -c public -v 2c 192.168.181.98 1.3.6.1.4.1.16394.2.1.1.3.1.29.32.131
SNMPv2-SMI::enterprises.16394.2.1.1.3.1.29.32.131 = STRING: "Current State Mask 0x0002"

Note on the return alarm values:

The returned value is a binary mask where 1=minor, 2=major, 4=critical. So for example, 0x0003, would mean a minor and a critical alarm were raised.

    Backup shelf manager


The backup shelf manager will automatically activate if the master fails.
Note: The chassis can operate without a Shelf Manager, but in this instance, there would be no monitoring of components.  One result of this would be for example the fan trays will automatically set themselves to turn at the maximum speed.

    Blade power control


When a chassis is powered on, all blades will be powered on, however it is possible, to power-off and power-on the Fortigate blades remotely from shelf manager CLI access. 

The blade IPMB address is used to select the blade to power control with CLI commands “clia activate <IPMB_ADDR> 0” and “clia deactivate <IPMB_ADDR> 0”.

The following table provides the matching between the blade slot number and the IPMB address.

slot #    IPMB address        slot #    IPMB address
13          9A                 2         84
11          96                 4         88
9           92                 6         8C
7           8E                 8         90
5           8A                 10        94
3           86                 12        98
1           82                 14        9C


Example:
•    To power off blade inserted in slot 5 :    clia deactivate 8a 0
•    To power on blade inserted in slot 6 :    clia activate 8c 0

    Useful Troubleshooting Commands


The following is is a list of useful troubleshooting commands, to initiate these commands access the Shelf Manager CLI via IP or console access, login with ‘root’ (no password by default)


date                              clia fruinfo 10 0              clia fru                 
version                           clia fruinfo 12 0              clia ipmc -v
clia version                      clia fruinfo 20 0              clia threshold
clia minfanlevel                  clia fruinfo 20 1              clia sensordata
clia fans                         clia fruinfo 20 2             
cat /var/messages
clia getfanlevel 20 3             clia fruinfo 20 3             
clia sel
clia getfanlevel 20 4             clia fruinfo 20 4             
clia getfanlevel 20 5             clia fruinfo 20 5             
ifconfig                          clia fruinfo 20 6
                                  clia fruinfo 20 7
                                  clia fruinfo 20 8


 

Related Articles

Technical Note : Advanced SNMP Trap Configuration for a FortiGate 5140 Shelf Manager

Contributors