Technical Note: How to configure FMC-XH0 card for IPS acceleration on FortiGate 3950B and FMC-C20 card

cborgato_FTNT · ‎06-11-2014

Purpose

The purpose of this document is to show how to configure XH0 card for IPS acceleration on a FortiGate 3950B device.

It is possible to use also FMC-XH0 with FMC-C20 card to increase the number of ports available.

Scope

FortiOS v4.00 MR3, v5, v5.2.
FortiGate 3950B and FortiGate 3951B.
FMC-C20 card to have multiple ports available.
FMC-XH0 card for IPS acceleration.

Diagram

FMC-XH0 card is on FortiGate 3950B slot FMC1.

FMC-C20 card is on FortiGate 3950B slot FMC2.

The diagram shows the C20 interfaces used for this configuration.

Each couple of interface are connect by a single policy with IPS enabled:

fmc2/1 to fmc2/4
fmc2/2 to fmc2/5
fmc2/3 to fmc2/6

Expectations, Requirements

FMC-XH0 will take in charge the IPS processes and the traffic crossing the FortiGate 3950B increasing the performance of the device.

Configuration

Once properly configured on FortiGate 3950B, FMC-XH0 is able to load balance the traffic to all the cores available.

CPU utilization : 0%

cpu 00 - 03 0% 0% 0% 0% <--------------- Cores used for interoperability with other XH0 cards (if any) or management purpose

cpu 04 - 07 0% 0% 0% 0%

cpu 08 - 11 0% 0% 0% 0%

cpu 12 - 15 0% 0% 0% 0%

cpu 16 - 19 0% 0% 0% 0%

cpu 20 - 23 0% 0% 0% 0%

cpu 24 - 27 0% 0% 0% 0%

cpu 28 - 31 0% 0% 0% 0%

If no IPS has been enabled, FMC-XH0 will load balance all the traffic to the 28 available cores (first 4 are reserved).

If IPS has been enabled, FMC-XH0 will load balance not inspected traffic in cores 4,8,12,16,20,24,28, while in all the others will load balance IPS inspected traffic (first 4 cores are reserved).

The following 4 steps will explain and show how to configure and see the different cores usage into the XH0 card.

1/ Configure interfaces on FMC-C20 card and related policy without IPS enable
2/ Configure the FMC-XH0 to accelerate IPS/traffic crossing the interfaces previously configure and check the FMC-XH0's cores
3/ Apply IPS inspection into the policy and check again the FMC-XH0's cores
4/ Possible to use multiple FMC-XH0 cards to load-balance the traffic

1/ Configure the interfaces on C20 card and related policy without IPS enable

Basic C20's Interface configuration :

config system interface
edit "fmc2/1"
set vdom "root"
set ip 192.168.43.254 255.255.252.0
set allowaccess ping https ssh http telnet
set type physical
next
edit "fmc2/2"
set vdom "root"
set ip 192.168.47.254 255.255.252.0
set allowaccess ping https ssh http telnet
set type physical
next
edit "fmc2/3"
set vdom "root"
set ip 192.168.51.254 255.255.252.0
set allowaccess ping https ssh http telnet
set type physical
next
edit "fmc2/4"
set vdom "root"
set ip 192.168.1.254 255.255.252.0
set allowaccess ping https ssh http telnet
set type physical
next
edit "fmc2/5"
set vdom "root"
set ip 192.168.2.254 255.255.252.0
set allowaccess ping https ssh http telnet
set type physical
next
edit "fmc2/6"
set vdom "root"
set ip 192.168.3.254 255.255.252.0
set allowaccess ping https ssh http telnet
set type physical
next
end

Basic policy configuration without IPS enabled:

config firewall policy
edit "1"
set srcintf "fmc2/1"
set dstintf "fmc2/4"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always"
set service "ANY"
next
edit "2"
set srcintf "fmc2/2"
set dstintf "fmc2/5"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always"
set service "ANY"
next
edit "3"
set srcintf "fmc2/3"
set dstintf "fmc2/6"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always"
set service "ANY"
next
end

2/ Configure the FMC-XH0 to accelerate IPS/traffic crossing the interfaces previously configure and check the FMC-XH0's cores

Following step will follow how to enable the XH0 card to accelerate traffic crossing the 6 interface previously configured :

config system sp3-port
set interface "fmc2/1" "fmc2/2" "fmc2/3" "fmc2/4" "fmc2/5" "fmc2/6"
end

NOTE: after pressing 'end', the FortiGate 3950B will be rebooted.

With traffic crossing the policy configured, should be possible to see all cores 4-31 used and 1 of the first 4 cores used for management purpose.

(Please note that just a part of diag command is reported)

#diagnose npu spm status 0
...
CPU utilization : 18%
cpu 00 - 03 0% 0% 33% 0%
cpu 04 - 07 23% 21% 14% 19%
cpu 08 - 11 27% 16% 15% 20%
cpu 12 - 15 14% 27% 20% 16%
cpu 16 - 19 26% 24% 19% 17%
cpu 20 - 23 16% 22% 24% 29%
cpu 24 - 27 14% 22% 24% 23%
cpu 28 - 31 18% 15% 19% 21%
end

3/ Apply IPS inspection into the policy and check again the FMC-XH0's cores

Enable IPS in 2 of the 3 policies in order to have 2/3 of the traffic inspected by the IPS XH0 engine.

config firewall policy
edit "1"
set srcintf "fmc2/1"
set dstintf "fmc2/4"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always"
set utm-status enable
set ips-sensor "all_default"
set profile-protocol-options "default"
next
edit "2"
set srcintf "fmc2/2"
set dstintf "fmc2/5"
set srcaddr "all"
set dstaddr "all"
set action accept
set schedule "always"
set utm-status enable
set ips-sensor "all_default"
set profile-protocol-options "default"
next
end

With traffic crossing the 3 policies, should be possible to see:

1 of the first 4 cores used for management purpose
cores 4,8,12,16,20,24,28 used to load balance no inspected traffic (usually dedicated for Firewall, Dos and IPSec traffic)
cores 5-7,9-11,13-15,17-19,21-23,25-27,29-31 used to load balance IPS inspected traffic

(Please note that just a part of diag command is reported)

#diagnose npu spm status 0
...
CPU utilization : 18%
cpu 00 - 03 0% 0% 41% 0%
cpu 04 - 07 21% 18% 16% 19%
cpu 08 - 11 29% 16% 19% 20%
cpu 12 - 15 18% 22% 21% 23%
cpu 16 - 19 27% 19% 17% 23%
cpu 20 - 23 17% 18% 27% 30%
cpu 24 - 27 19% 25% 23% 27%
cpu 28 - 31 22% 12% 14% 25%
end

4/ Possible to use multiple FMC-XH0 cards to load-balance the traffic

In case multiple FMC-XH0 cards are used, it is possible to have the following configuration:

- 1 Master module (card) responsible to load-balance the traffic between its cores and forwards traffic also to the other FMC-XH0 slave modules' cores;
- up to 4 Slave modules.

(Please note that just a part of diag command from Master module is reported)

#diagnose npu spm status 0
...
Master FMC1, sp_1 Number of Slave Module 1 Switch port ...

This is the default behavior and doesn't require any additional configuration.

Verification
Some complete output of the CLI commands :

FG3K9B # diagnose npu spm list

Available SP Modules:

ID Model Slot Interface

0 xh0 fmc1 none

FG3K9B # diagnose npu spm status 0 <---------- ID of the card (it's possible to have more the one XH0 card on Fortigate 3950B)

XH0

FMC1 : sp_0

Version : 1.317 (Dec 18 2013 16:40:31)

IPS Version : 2.179

Build Time : 01/15/2014 12:00

IPS rule version : 4.345

AV rule version : 10.974

IPS status : STANDBY

IPS mask : 00000008

Serial Number : FMCXH03E12345678

Bootloader : v0001-r26758

Module version : XH0-r1

CPU : XLP432 Rev B0

CPU utilization : 0%

cpu 00 - 03 0% 0% 0% 0%

cpu 04 - 07 0% 0% 0% 0%

cpu 08 - 11 0% 0% 0% 0%

cpu 12 - 15 0% 0% 0% 0%

cpu 16 - 19 0% 0% 0% 0%

cpu 20 - 23 0% 0% 0% 0%

cpu 24 - 27 0% 0% 0% 0%

cpu 28 - 31 0% 0% 0% 0%

Memory utilization : 18%

Second on SP : 2809:2810

Loops on monitor : 2816409:2817349

Master FMC1, sp_0

Number of Slave Module 0

Switch port 22 24 21 23

Other possible option are:

diagnose npu spm datapath Show data path counters

diagnose npu spm dos Display DoS information

diagnose npu spm ipsec Display IPsec information

diagnose npu spm list List all SP modules in system

diagnose npu spm status Show module status

Troubleshooting