Technical Tip: BGP weight attribute in redistribution scenario

[gmanea]

Description
This article is explaining following scenario:
FortiGate has BGP peering with neighbor and receiving X.X.X.X/X subnet.
In case of failure of the BGP peering, FortiGate has static route with higher administrative distance to mitigate outage.
At the same time, FortiGate is redistributing all static routes into BGP. After BGP is restored, with default settings, subnet X.X.X.X/X that will be installed in routing-table is not one received from BGP peer.

Solution
Problem.

Default configuration on FortiGate:

rimmer-esx27 (root) # show router bgp
# config router bgp
    set as 65000
    set router-id 10.109.20.6
    # config neighbor
        edit "10.109.16.250"
            set soft-reconfiguration enable
            set remote-as 4500
        next
    end
    # config redistribute "static"
        set status enable

We are receiving route 10.110.0.0/20 from BGP and we have static route in case of failure. Static route is configured with higher AD than BGP. In this case, we have eBGP so AD on static route needs to be higher than 20.
In normal state, FortiGate has in routing-table BGP route and static route is in database only:

rimmer-esx27 (root) # get router info routing-table all
Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default
S*      0.0.0.0/0 [1/0] via 5.5.5.254, port3
                  [1/0] via 10.109.31.254, port1
C       5.5.5.0/24 is directly connected, port3
C       10.32.1.0/24 is directly connected, Vlan321
C       10.109.16.0/20 is directly connected, port1
B       10.110.0.0/20 [20/0] via 10.109.16.250, port1, 00:01:43 <<< Our route
C       10.218.0.0/24 is directly connected, Vlan218

rimmer-esx27 (root) # get router info routing-table data
Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       > - selected route, * - FIB route, p - stale info

S    *> 0.0.0.0/0 [1/0] via 5.5.5.254, port3
     *>           [1/0] via 10.109.31.254, port1
C    *> 5.5.5.0/24 is directly connected, port3
C    *> 10.32.1.0/24 is directly connected, Vlan321
C    *> 10.109.16.0/20 is directly connected, port1
S       10.110.0.0/20 [150/0] via 10.109.16.250, port1 <<< static route that is not installed yet
B    *> 10.110.0.0/20 [20/0] via 10.109.16.250, port1, 00:02:36
C    *> 10.218.0.0/24 is directly connected, Vlan218

And attributes of the BGP route:

rimmer-esx27 (root) # get router info bgp network 10.110.0.0/20
BGP routing table entry for 10.110.0.0/20
Paths: (1 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  4500
    10.109.16.250 from 10.109.16.250 (10.109.16.250)
      Origin IGP metric 0, localpref 100, valid, external, best
      Last update: Tue Apr 13 07:08:24 2021

Now, there is a network failure. BGP is going down, static route is installed in routing-table.

rimmer-esx27 (root) # get router info bgp sum
BGP router identifier 10.109.20.6, local AS number 65000
BGP table version is 3
1 BGP AS-PATH entries
0 BGP community entries
Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.109.16.250   4       4500    7731    7749        0    0    0 00:00:14 Active     
Total number of neighbors 1

rimmer-esx27 (root) # get router info routing-table all

Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default
S*      0.0.0.0/0 [1/0] via 5.5.5.254, port3
                  [1/0] via 10.109.31.254, port1
C       5.5.5.0/24 is directly connected, port3
C       10.32.1.0/24 is directly connected, Vlan321
C       10.109.16.0/20 is directly connected, port1
S       10.110.0.0/20 [150/0] via 10.109.16.250, port1 << Static route installed.
C       10.218.0.0/24 is directly connected, Vlan218

However, after BGP is restored, BGP route is not installed back in routing-table but still the static route:

rimmer-esx27 (root) # get router info bgp sum
BGP router identifier 10.109.20.6, local AS number 65000
BGP table version is 3
2 BGP AS-PATH entries
0 BGP community entries
Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.109.16.250   4       4500    7735    7753        3    0    0 00:01:04        1
Total number of neighbors 1


rimmer-esx27 (root) # get router info routing-table all
Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default
S*      0.0.0.0/0 [1/0] via 5.5.5.254, port3
                  [1/0] via 10.109.31.254, port1
C       5.5.5.0/24 is directly connected, port3
C       10.32.1.0/24 is directly connected, Vlan321
C       10.109.16.0/20 is directly connected, port1
S       10.110.0.0/20 [150/0] via 10.109.16.250, port1 <<< Still static route
C       10.218.0.0/24 is directly connected, Vlan218

And if BGP attributes of the route is verified:

rimmer-esx27 (root) # get router info bgp network 10.110.0.0/20
BGP routing table entry for 10.110.0.0/20
Paths: (2 available, best #2, table Default-IP-Routing-Table)
  Not advertised to any peer
  4500
    10.109.16.250 from 10.109.16.250 (10.109.16.250)
      Origin IGP metric 0, localpref 100, valid, external
      Last update: Tue Apr 13 07:18:12 2021
  Local
    10.109.16.250 from 0.0.0.0 (10.109.20.6)
      Origin incomplete, localpref 100, weight 32768, valid, sourced, best
      Last update: Tue Apr 13 07:16:38 2021

rimmer-esx27 (root) # get router info bgp network
BGP table version is 3, local router ID is 10.109.20.6
Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,
              S Stale
Origin codes: i - IGP, e - EGP, ? - incomplete
   Network          Next Hop            Metric LocPrf Weight RouteTag Path
*  10.110.0.0/20    10.109.16.250            0             0        0  4500 i
*>                        10.109.16.250                      32768           0 ?

Solution.
As you can see, we are receiving 2 routes from the BGP, but with different attributes. By default, redistributed route is having weight set to 32768, while receiving route from BGP neighbor has 0.
As a result, from BGP perspective, the best route for 10.110.0.0/20 is route with higher weight, which is locally originated route and that’s because routing-table will not converge to original state and will keep static route in routing-table. This problem will be visible also if you have OSPF running and you are receiving route from OSPF and you are redistributing OSPF into BGP.
Solution for this is to set weight for incoming routes from BGP neighbor that is higher than the weight for locally-originated route (usually higher than 32768).
There 2 ways how to do this: with route-map or set weight for neighbor.

1) Route-map option.

rimmer-esx27 (root) # show router route-map
# config router route-map
    edit "set-weight"
        # config rule
            edit 1
                set set-weight 40000
            next
        end
    next
end

# config router bgp
# config neighbor
        edit "10.109.16.250"
            set route-map-in "set-weight"
        next
    end

And do not forget to soft-clear the session:

# execute router clear bgp all soft

Now, after restoration of BGP neighbor:

rimmer-esx27 (root) # get router info routing-table all
Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S*      0.0.0.0/0 [1/0] via 5.5.5.254, port3

                  [1/0] via 10.109.31.254, port1
C       5.5.5.0/24 is directly connected, port3
C       10.32.1.0/24 is directly connected, Vlan321
C       10.109.16.0/20 is directly connected, port1
S       10.110.0.0/20 [150/0] via 10.109.16.250, port1 << Still static route
C       10.218.0.0/24 is directly connected, Vlan218

rimmer-esx27 (root) # get router info bgp sum
BGP router identifier 10.109.20.6, local AS number 65000
BGP table version is 6
2 BGP AS-PATH entries
0 BGP community entries
Neighbor        V         AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
10.109.16.250   4       4500    7750    7772        5    0    0 00:00:02        1
Total number of neighbors 1

rimmer-esx27 (root) # get router info routing-table all
Routing table for VRF=0
Codes: K - kernel, C - connected, S - static, R - RIP, B - BGP
       O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, L1 - IS-IS level-1, L2 - IS-IS level-2, ia - IS-IS inter area
       * - candidate default

S*      0.0.0.0/0 [1/0] via 5.5.5.254, port3
                  [1/0] via 10.109.31.254, port1
C       5.5.5.0/24 is directly connected, port3
C       10.32.1.0/24 is directly connected, Vlan321
C       10.109.16.0/20 is directly connected, port1
B       10.110.0.0/20 [20/0] via 10.109.16.250, port1, 00:00:01 <----- BGP is up, BGP route installed.
C       10.218.0.0/24 is directly connected, Vlan218

rimmer-esx27 (root) # get router info bgp network 10.110.0.0
BGP routing table entry for 10.110.0.0/20
Paths: (1 available, best #1, table Default-IP-Routing-Table)
  Not advertised to any peer
  4500
    10.109.16.250 from 10.109.16.250 (10.109.16.250)
      Origin IGP metric 0, localpref 100, weight 40000, valid, external, best
      Last update: Tue Apr 13 07:33:01 2021

2) Neighbor weight option

# config router bgp
    set as 65000
    set router-id 10.109.20.6
    # config neighbor
        edit "10.109.16.250"
            set soft-reconfiguration enable
            set remote-as 4500
            set weight 40000                                    <----- Weight for neighbor.
        next

This setting will set weight for all incoming routes received from the neighbor.
The result is the same as with route-map.