Technical Tip: VIP not working with SD-WAN reply traffic causing issue

[sagha]

Description
This article describes the scenario where when trying to access VIP in a setup where SD-WAN, it does not work anymore.

Solution
Explanation.

The traffic reaches inbound on FortiGate correctly and correct DNAT takes place.

id=20085 trace_id=1131 func=print_pkt_detail line=5693 msg="vd-root:0 received a packet(proto=6, 10.10.20.1:64457->10.10.10.1:443) from wan1. flag [S], seq 2537036893, ack 0, win 64240"
id=20085 trace_id=1131 func=init_ip_session_common line=5864 msg="allocate a new session-10d7fav2"
id=20085 trace_id=1131 func=fw_pre_route_handler line=182 msg="VIP-10.179.5.4:443, outdev-wan1"
id=20085 trace_id=1131 func=__ip_session_run_tuple line=3485 msg="DNAT 10.10.10.1:443->10.179.5.4:443"
id=20085 trace_id=1131 func=vf_ip_route_input_common line=2584 msg="find a route: flag=00000000 gw-10.179.5.4 via internal1"
id=20085 trace_id=1131 func=fw_forward_handler line=796 msg="Allowed by Policy-10:"
id=20085 trace_id=1131 func=ipd_post_route_handler line=490 msg="out MGMT-FortiMgmt vwl_zone_id 0, state2 0x1, quality 0.

The return traffic however seems to cause the issue.
Traffic once reaches FortiGate, looks for the best route available via SD-WAN and forwards the traffic accordingly.

In the below scenario, the traffic originally was received on wan1 interface, however, for return traffic the best route is via WAN2.

id=20085 trace_id=1132 func=print_pkt_detail line=5693 msg="vd-root:0 received a packet(proto=6, 10.179.5.4:443->10.10.20.1:64457) from internal1. flag [S.], seq 698453379, ack 2537036894, win 29200"
id=20085 trace_id=1132 func=resolve_ip_tuple_fast line=5774 msg="Find an existing session, id-10d7fad1, reply direction"
id=20085 trace_id=1132 func=vf_ip_route_input_common line=2584 msg="find a route: flag=04000000 gw-10.10.30.1 via wan2"
id=20085 trace_id=1132 func=npu_handle_session44 line=1160 msg="Trying to offloading session from internal1 to wan2, skb.npu_flag=00000400 ses.state=04010204 ses.npu_state=0x01000000"
2id=20085 trace_id=1132 func=fw_forward_dirty_handler line=394 msg="state=04010204, state2=00000001, npu_state=01000000"
id=20085 trace_id=1132 func=__ip_session_run_tuple line=3471 msg="SNAT 10.179.5.4->10.10.10.1:443"
id=20085 trace_id=1132 func=ipd_post_route_handler line=490 msg="out wan2 vwl_zone_id 2, state2 0x1, quality 1.

This causes VIP configured for wan1 not work correctly and hence it fails.

Solution.

In such a scenario, we need to check if auxiliary session is enable or disable on FortiGate.

If the reply traffic is returning on the same interface, FortiGate handles the traffic as follow:

With Auxiliary sessions disabled.

# config system settings
     set auxiliary-session disable
end

The reply to the client egresses on the original incoming interface, Port1.
If policy routes or SD-WAN rules are configured, there are not checked.

Auxiliary sessions enabled.

# config system settings
     set auxiliary-session enable
end

The reply to the client egresses on the best route in the routing table:
- If the best route is WAN1, then reply traffic will egress on WAN1.
- If the best route is WAN2, then reply traffic will egress on WAN2.

For using VIP, to use the same incoming interface for reply traffic, the auxiliary session needs to be disabled so the traffic uses the same path and does not calculate a new path.

Solution2.

Also check if asymmetric routing is enabled on FortiGate.

# config system setting
    set asymroute enable
end

If this is enabled, disable it.

# config system setting
    set asymroute disable
end