Technical Tip: Misconfiguration related to IPpool or VIP causes FortiGate to reset the connection

[ppatel]

Description
A misconfigured IPpool or VIP can create connectivity issues for TCP connections even if there are policies allowing traffic to go through the FortiGate.

 In such a case, it could be noticed that the TCP syn would go through the FortiGate but when receiving the TCP syn/ack, the FortiGate would send back a TCP rst to the originator of the TCP syn/ack.

Solution
A first simple case where this problem can occur is:

client (.100) ---- 10.1.1.x/24 ---- (.1) port1 ---- FGT ---- port5 (.1) ---- 20.1.1.x/24 ---- (.100) server

# show firewall policy
# config firewall policy
    edit 1
        set uuid be82756a-95f7-51e6-aa3b-5a5127e32b55
        set srcintf "port1"
        set dstintf "port5"
        set srcaddr "all"
        set dstaddr "all"
        set action accept
        set schedule "always"
        set service "ALL"
    next
end

# show firewall vip vip1
# config firewall vip
    edit "vip1"
        set uuid 46c513ba-95f8-51e6-564c-cdd05631c9e6
        set extip 10.1.1.100
        set extintf "any"
        set mappedip "30.1.1.1"
    next
end

Note that the VIP1 is just defined but is not used in any policy.

If the client (10.1.1.100) tries to establish a SSH connection to the server (20.1.1.100), it can be noticed:

Sniffer trace.

2016-10-19 14:34:12.189914 port1 in 10.1.1.100.38947 -> 20.1.1.100.22: syn 3654889098
2016-10-19 14:34:12.195421 port5 out 10.1.1.100.38947 -> 20.1.1.100.22: syn 3654889098
2016-10-19 14:34:12.195590 port5 in 20.1.1.100.22 -> 10.1.1.100.38947: syn 2220274952 ack 3654889099
2016-10-19 14:34:12.195627 port5 out 10.1.1.100.38947 -> 20.1.1.100.22: rst 3654889099
2016-10-19 14:34:13.189030 port1 in 10.1.1.100.38947 -> 20.1.1.100.22: syn 3654889098
2016-10-19 14:34:13.189049 port5 out 10.1.1.100.38947 -> 20.1.1.100.22: syn 3654889098
2016-10-19 14:34:13.189421 port5 in 20.1.1.100.22 -> 10.1.1.100.38947: syn 2235797277 ack 3654889099
2016-10-19 14:34:13.189436 port5 out 10.1.1.100.38947 -> 20.1.1.100.22: rst 3654889099
2016-10-19 14:34:15.192915 port1 in 10.1.1.100.38947 -> 20.1.1.100.22: syn 3654889098
2016-10-19 14:34:15.192931 port5 out 10.1.1.100.38947 -> 20.1.1.100.22: syn 3654889098
2016-10-19 14:34:15.193262 port5 in 20.1.1.100.22 -> 10.1.1.100.38947: syn 2267105566 ack 3654889099
2016-10-19 14:34:15.193277 port5 out 10.1.1.100.38947 -> 20.1.1.100.22: rst 3654889099

Debug flow.

id=20085 trace_id=239 func=print_pkt_detail line=4471 msg="vd-root received a packet(proto=6, 10.1.1.100:38947->20.1.1.100:22) from port1. flag [S], seq 3654889098, ack 0, win 29200"
id=20085 trace_id=239 func=init_ip_session_common line=4624 msg="allocate a new session-000651da"
id=20085 trace_id=239 func=vf_ip4_route_input line=1596 msg="find a route: flags=00000000 gw-20.1.1.100 via port5"
id=20085 trace_id=239 func=fw_forward_handler line=686 msg="Allowed by Policy-1:"
id=20085 trace_id=240 func=print_pkt_detail line=4471 msg="vd-root received a packet(proto=6, 20.1.1.100:22->10.1.1.100:38947) from port5. flag [S.], seq 2220274952, ack 3654889099, win 28960"
id=20085 trace_id=240 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-000651da, reply direction"
id=20085 trace_id=241 func=print_pkt_detail line=4471 msg="vd-root received a packet(proto=6, 10.1.1.100:38947->20.1.1.100:22) from local. flag [R], seq 3654889099, ack 0, win 0"
id=20085 trace_id=241 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-000651da, original direction"
id=20085 trace_id=242 func=print_pkt_detail line=4471 msg="vd-root received a packet(proto=6, 10.1.1.100:38947->20.1.1.100:22) from port1. flag [S], seq 3654889098, ack 0, win 29200"
id=20085 trace_id=242 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-000651da, original direction"
id=20085 trace_id=242 func=vf_ip4_route_input line=1596 msg="find a route: flags=00000000 gw-20.1.1.100 via port5"
id=20085 trace_id=243 func=print_pkt_detail line=4471 msg="vd-root received a packet(proto=6, 20.1.1.100:22->10.1.1.100:38947) from port5. flag [S.], seq 2235797277, ack 3654889099, win 28960"
id=20085 trace_id=243 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-000651da, reply direction"
id=20085 trace_id=244 func=print_pkt_detail line=4471 msg="vd-root received a packet(proto=6, 10.1.1.100:38947->20.1.1.100:22) from local. flag [R], seq 3654889099, ack 0, win 0"
id=20085 trace_id=244 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-000651da, original direction"
id=20085 trace_id=245 func=print_pkt_detail line=4471 msg="vd-root received a packet(proto=6, 10.1.1.100:38947->20.1.1.100:22) from port1. flag [S], seq 3654889098, ack 0, win 29200"
id=20085 trace_id=245 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-000651da, original direction"
id=20085 trace_id=245 func=vf_ip4_route_input line=1596 msg="find a route: flags=00000000 gw-20.1.1.100 via port5"
id=20085 trace_id=246 func=print_pkt_detail line=4471 msg="vd-root received a packet(proto=6, 20.1.1.100:22->10.1.1.100:38947) from port5. flag [S.], seq 2267105566, ack 3654889099, win 28960"
id=20085 trace_id=246 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-000651da, reply direction"
id=20085 trace_id=247 func=print_pkt_detail line=4471 msg="vd-root received a packet(proto=6, 10.1.1.100:38947->20.1.1.100:22) from local. flag [R], seq 3654889099, ack 0, win 0"
id=20085 trace_id=247 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-000651da, original direction"

The problem can be fixed by disabling arp-reply in VIP1 or deleting VIP1 .

___________________________________

The same problem can occur if the client uses IP address utilized in the IP pool (the IP pool also does not have to be necessarily utilized in a firewall policy).

# show firewall ippool pool1
# config firewall ippool
    edit "pool1"
        set startip 10.1.1.100
        set endip 10.1.1.100
    next

The problem can be fixed again by disabling arp-reply in pool1 or deleting it.

___________________________________
There is a second case where a similar problem can occur.

client (.100) ---- 10.1.1.x/24 ---- (.1) port1 ---- vdom V1 - ivl - vdom V2 ---- port5 (.1) ---- 20.1.1.x/24 ---- (.100) server

Here, there is a policy in VDOM V1 which source nat the traffic from client to server using an IPpool (10.1.1.100 ---> 11.1.1.1) . 
In VDOM V2 there is an accept all policy to forward the traffic to the server. 

A VIP is configured with extip = 11.1.1.1 is VDOM V2.  This VIP is not used in any policy.  The same symptoms are noticed

Sniffer trace.

2016-10-18 15:03:50.232571 port1 in 10.1.1.100.38938 -> 20.1.1.100.22: syn 1793278706
2016-10-18 15:03:50.232625 V1-V2-0 out 11.1.1.1.38938 -> 20.1.1.100.22: syn 1793278706
2016-10-18 15:03:50.232625 V1-V2-1 in 11.1.1.1.38938 -> 20.1.1.100.22: syn 1793278706
2016-10-18 15:03:50.248890 port5 out arp who-has 20.1.1.100 tell 20.1.1.1
2016-10-18 15:03:50.249186 port5 in arp reply 20.1.1.100 is-at 0:50:56:1:68:60
2016-10-18 15:03:50.249194 port5 out 11.1.1.1.38938 -> 20.1.1.100.22: syn 1793278706
2016-10-18 15:03:50.249399 port5 in 20.1.1.100.22 -> 11.1.1.1.38938: syn 4083851017 ack 1793278707
2016-10-18 15:03:50.249505 port5 out 11.1.1.1.38938 -> 20.1.1.100.22: rst 1793278707

Debug flow.

id=20085 trace_id=11 func=print_pkt_detail line=4471 msg="vd-V1 received a packet(proto=6, 10.1.1.100:38938->20.1.1.100:22) from port1. flag [S], seq 1793278706, ack 0, win 29200"
id=20085 trace_id=11 func=init_ip_session_common line=4624 msg="allocate a new session-00037c06"
id=20085 trace_id=11 func=vf_ip4_route_input line=1596 msg="find a route: flags=00000000 gw-20.1.1.100 via V1-V2-0"
id=20085 trace_id=11 func=fw_forward_handler line=686 msg="Allowed by Policy-1: SNAT"
id=20085 trace_id=11 func=__ip_session_run_tuple line=2593 msg="SNAT 10.1.1.100->11.1.1.1:38938"
id=20085 trace_id=12 func=print_pkt_detail line=4471 msg="vd-V2 received a packet(proto=6, 11.1.1.1:38938->20.1.1.100:22) from V1-V2-1. flag [S], seq 1793278706, ack 0, win 29200"
id=20085 trace_id=12 func=init_ip_session_common line=4624 msg="allocate a new session-00037c07"
id=20085 trace_id=12 func=vf_ip4_route_input line=1596 msg="find a route: flags=00000000 gw-20.1.1.100 via port5"
id=20085 trace_id=12 func=fw_forward_handler line=686 msg="Allowed by Policy-1:"
id=20085 trace_id=13 func=print_pkt_detail line=4471 msg="vd-V2 received a packet(proto=6, 20.1.1.100:22->11.1.1.1:38938) from port5. flag [S.], seq 4083851017, ack 1793278707, win 28960"
id=20085 trace_id=13 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-00037c07, reply direction"
id=20085 trace_id=14 func=print_pkt_detail line=4471 msg="vd-V2 received a packet(proto=6, 11.1.1.1:38938->20.1.1.100:22) from local. flag [R], seq 1793278707, ack 0, win 0"
id=20085 trace_id=14 func=resolve_ip_tuple_fast line=4534 msg="Find an existing session, id-00037c07, original direction"