FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
sha-1_FTNT
Staff
Staff
Article Id 189917
Purpose

The purpose of this article is to explain the conditions which must be met in order to allow hardware acceleration with redundant or aggregate interfaces.


Scope


Diagram


Expectations, Requirements

> Is it a problem to span interface members of a redundant or aggregate interface over multiple NPU chips?


* NP4 interfaces : there is no restriction as to which NP4 chip the interface members are bound to.

* NP2 interfaces : some restrictions :

Only 4 interfaces can be connected to an NP2 chip.
Two NP2 chips may be inter-connected via an EEI bus which allows traffic to be hardware accelerated between an interface on the first chip and another interface on the second chip.
An EEI bus can only interconnect two NP2 chips
An NP2 chip has only one EEI connection, so one NP2 chip can only be EEI connected to one other NP2 chip.

In case of units with 8 NP2 interfaces (like 310B) :
- four are bound to NP2 chip #0
- four are bound to NP2 chip #1
- NP2 chip #0 and #1 are EEI interconnected

So, with 310B, any NP2 interface can hardware accelerate its traffic to any other NP2 interface.
Consequently, any NP2 interface on a 310B can be combined in a redundant/aggregate interface without compromising the hardware offload eligibility of the redundant/aggregate interface.

In case of units with 16 NP2 interfaces (like 620B or 3016B) :
- four are bound to NP2 chip #0
- four are bound to NP2 chip #1
- four are bound to NP2 chip #2
- four are bound to NP2 chip #3
- NP2 chips #0 and #1 are EEI interconnected
- NP2 chips #2 and #3 are EEI interconnected

Distribution of the interfaces on each NP2 chip can be seen with command 'diagnose npu np2 list'

So, with these units :
- any NP2 interface bound to chip #0 or #1 can hardware accelerate its traffic to any NP2 interface bound to chip #0 or #1
- any NP2 interface bound to chip #2 or #3 can hardware accelerate its traffic to any NP2 interface bound to chip #2 or #3

Hence the limitation : an NP2 interface bound to chip #0 or #1 canNOT hardware accelerate its traffic to an NP2 interface bound to chip #2 or #3

Consequently, on these units, special care must be made to NOT mix interfaces which are bound to NP2 which are NOT EEI interconnected.
Only interface members which are bound to the same NP2 chip or NP2 chips which are EEI interconnected should be mixed in a redundant/aggregate interface if hardware acceleration is desired.


> Are these sessions still accelerated?

The above explanation is about the redundant/aggregate interface itself.
If correctly configured, the redundant/aggregate interface is considered by the kernel as an NPU interface (as shown by 'npu: y' in 'diagnose netlink redundant name <interface-name>')
Though this is a mandatory condition, it is only one of the conditions for a session to be eligible for hardware acceleration.

All conditions are explained in the hardware guide (http://docs.fortinet.com/fgt/handbook/40mr3/fortigate-hardware-40-mr3.pdf), pp.31-32:

"../..
Fast path ready session characteristics are:
• Layer 2 type/length must be 0x0800 (IEEE 802.1q VLAN specification is supported); link aggregation between any network interfaces sharing the same network processor(s) may be used (IEEE 802.3ad specification is supported)
• Layer 3 protocol must be IPv4
• Layer 4 protocol must be UDP, TCP or ICMP
• Layer 3 / Layer 4 header or content modification must not require a session helper (for example, SNAT, DNAT, and TTL reduction are supported, but application layer content modification is not supported)
• FortiGate unit firewall policy must not require antivirus or IPS inspection
• origin must not be local host (the FortiGate unit)
• ingress and egress network interfaces are both attached to the same network processor(s)
../..
Packets within the session must then also meet packet requirements.
• Incoming packets must not be fragmented.
• Outgoing packets must not require fragmentation to a size less than 385 bytes.
Because of this requirement, the configured MTU (Maximum Transmission Unit) for network processors’ network interfaces must also meet or exceed the network processors’ supported minimum MTU of 385 bytes.
../.."


In case of 310B, any NP2 interface can be used as source/destination of the policy. As long as there is no profile (IPS, AV,...), non fragmented traffic will be offloaded to the NP2.
In case of units like 620B/3016B, special care must be done with source/destination interfaces of the policies: they must be both bound to the same NP2 chip or these chips must be EEI interconnected if hardware acceleration is desired.

Again, no limitation with NP4 : any NP4 interface can be used as source or destination in the policy.


>  Recommendations ?

Keep as much as possible traffic which is hardware accelerated inside the same NP2 chip since EEI bus can be overloaded.
When an EEI bus is being overflowed by the traffic, it can be seen in a diag command :

FGT3KB # diagnose hardware deviceinfo nic <NP2-interface-name>
Driver Name: NP2
Version: 0.92
Chip Revision: 1
../..
EEI0 Dropped: 0
EEI1 Dropped: 0
EEI2 Dropped: 0
EEI3 Dropped: 0
../..

Continuous increase of one of the four above counter indicates that the EEI bus is undersized for the traffic it is asked to forward from one NP2 chip to another.
There is nothing which can be done except re-organizing the interface assignment in such a way that heavy traffic no longer cross the EEI bus and is kept inside the same NP2.

Configuration
FG600B-2 # diagnose npu np2 list
ID       PORTS
--       -----
0        port1
0        port2
0        port3
0        port4
ID       PORTS
--       -----
1        port5
1        port6
1        port7
1        port8
ID       PORTS
--       -----
2        port9
2        port10
2        port11
2        port12
ID       PORTS
--       -----
3        port13
3        port14
3        port15
3        port16

------------------------------

FG600B-2 # sh system interface <myredundant>
config system interface
    edit "myredundant"
        set vdom "root"
        set ip 172.31.16.192 255.255.252.0
        set allowaccess ping https ssh http telnet fgfm
        set type redundant
            set member "port2" "port12"
    next
end

FG600B-2 # diagnose netlink redundant name <myredundant>
status: up
npu: n    <--- this redundant interface is not considered as an NPU interface by the kernel because NP2 chip #0 (port2) is not EEI connected to NP2 chip #2 (port12)
ports: 2
MAC addr: 00:09:0f:b7:3c:79
current slave: port2

slave: port2
  link status: up
  link failure count: 0
  permanent MAC addr: 00:09:0f:b7:3c:79

slave: port12
  link status: up
  link failure count: 0
  permanent MAC addr: 00:09:0f:b7:3c:83

------------------------------

FG600B-2 # sh system interface <myredundant>
config system interface
    edit "myredundant"
        set vdom "root"
        set ip 172.31.16.192 255.255.252.0
        set allowaccess ping https ssh http telnet fgfm
        set type redundant
            set member "port2" "port17"
    next
end

FG600B-2 #
FG600B-2 # diagnose netlink redundant name <myredundant>
status: up
npu: n   <--- this redundant interface is not considered as an NPU interface by the kernel because port17 is not an NP2 interface
ports: 2
MAC addr: 00:09:0f:b7:3c:79
current slave: port2

slave: port2
  link status: up
  link failure count: 0
  permanent MAC addr: 00:09:0f:b7:3c:79

slave: port17
  link status: down
  link failure count: 0
  permanent MAC addr: 00:09:0f:b7:3c:88

------------------------------

FG600B-2 # sh system interface <myredundant>
config system interface
    edit "myredundant"
        set vdom "root"
        set ip 172.31.16.192 255.255.252.0
        set allowaccess ping https ssh http telnet fgfm
        set type redundant
            set member "port2" "port3"
    next
end

FG600B-2 # diagnose netlink redundant name <myredundant>
status: up
npu: y  <--- this redundant interface is considered as an NPU interface by the kernel because port2 and port3 are bound to the same NP2 chip
oid: 1
ports: 2
MAC addr: 00:09:0f:b7:3c:79
current slave: port2

slave: port2
  link status: up
  link failure count: 0
  permanent MAC addr: 00:09:0f:b7:3c:79

slave: port3
  link status: up
  link failure count: 0
  permanent MAC addr: 00:09:0f:b7:3c:7a

------------------------------

FG600B-2 # sh system interface <myredundant>
config system interface
    edit "myredundant"
        set vdom "root"
        set ip 172.31.16.192 255.255.252.0
        set allowaccess ping https ssh http telnet fgfm
        set type redundant
            set member "port2" "port5"
    next
end

FG600B-2 #
FG600B-2 # diagnose netlink redundant name <myredundant>
status: up
npu: y   <--- this redundant interface is considered as an NPU interface by the kernel because NP2 chip #0 (port2) is EEI connected to NP2 chip #1 (port5)
oid: 1
ports: 2
MAC addr: 00:09:0f:b7:3c:79
current slave: port2

slave: port2
  link status: up
  link failure count: 0
  permanent MAC addr: 00:09:0f:b7:3c:79

slave: port5
  link status: up
  link failure count: 0
  permanent MAC addr: 00:09:0f:b7:3c:7c

------------------------------

FG600B-2 # diagnose hardware deviceinfo nic port2
Driver Name: NP2
Version: 0.92
Chip Revision: 2
BoardSN: ÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
Module Name: 620B-Rev2
DDR Size: 256 MB
Bootstrap ID: 15
PCIX-64bit-@133MHz bus: 0f:01.0
Admin: up, num=4, duration=2732514
Current_HWaddr: 00:09:0f:b7:3c:79
Permanent_HWaddr: 00:09:0f:b7:3c:79
Link: up, 1
Speed: 1000Mbps
Duplex: Full
Rx Pkts: 1708986
Tx Pkts: 5114
Rx Bytes: 78325760
Tx Bytes: 195584
MAC1 Rx Errors: 0
MAC1 Rx Dropped: 0
MAC1 Tx Dropped: 0
MAC1 FIFO Overflow: 0
MAC1 IP Error: 0
TAE Entry Used: 0
TSE Entry Used: 0
Host Dropped: 0
Shaper Dropped: 0
EEI0 Dropped: 0
EEI1 Dropped: 0
EEI2 Dropped: 0
EEI3 Dropped: 0

IPSEC QFIFO Dropped: 0
IPSEC DFIFO Dropped: 0
PBA: 123/1019/251
Forwarding Entry Used: 0
Offload IPSEC Antireplay ENC Status: Disable
Offload IPSEC Antireplay DEC Status: Enable
Offload Host IPSEC Traffic: Disable
ses mask: 40047dcb



Contributors