FortiGate
FortiGate Next Generation Firewall utilizes purpose-built security processors and threat intelligence security services from FortiGuard labs to deliver top-rated protection and high performance, including encrypted traffic.
nvisentin_FTNT
Article Id 196225
Description
This article explains how to analyze and troubleshoot multicast registration failure in PIM Spare Mode environment on a FortiGate using FortiOS 5.0, 5.2 and 5.4.

Solution
In the following diagram, B is the source sending multicast traffic toward group 239.1.1.1.  R4 is the 1st hop router in charge of sending the PIM register message to the Rendezvous Point (RP).  R3's Loopback (3.3.3.3) is the RP.  All Multicast routers are configured with 3.3.3.3 as static RP address.

nvisentin_FD39227_tn_FD39227-1.jpg

In this example, there was no receiver.  The source registers first.

Once the source (B) starts sending multicast traffic toward 239.1.1.1, the Rendezvous Point (R3) should have a (10.4.0.10,239.1.1.1) entry in its multicast routing table.

1. Multicast routing table on R3 (RP)
(R3) # get router info multicast pim sparse-mode table 239.1.1.1
IP Multicast Routing Table

(*,*,RP) Entries: 0
(*,G) Entries: 2
(S,G) Entries: 5
(S,G,rpt) Entries: 9
FCR Entries: 0

There is no (10.4.0.10,239.1.1.1) entry in the multicast routing table.

Remember that the PIM register message (IP Protocol 103) sent by the 1st hop router (R4) is sent to the unicast address of the RP (3.3.3.3).

2. Start a sniffer on R3 (RP) to make sure it receives the PIM register
(R3) # diag sniffer packet r3_r4_0 'proto 103' 4 0 l 
interfaces=[r3_r4_0]
filters=[proto 103]
2016-07-06 06:17:07.644360 r3_r4_0 in 10.4.0.4 -> 3.3.3.3:  ip-proto-103 68
2016-07-06 06:17:08.639835 r3_r4_0 in 10.4.0.4 -> 3.3.3.3:  ip-proto-103 68
2016-07-06 06:17:09.639791 r3_r4_0 in 10.4.0.4 -> 3.3.3.3:  ip-proto-103 68
2016-07-06 06:17:10.642411 r3_r4_0 in 10.4.0.4 -> 3.3.3.3:  ip-proto-103 68
2016-07-06 06:17:11.639761 r3_r4_0 in 10.4.0.4 -> 3.3.3.3:  ip-proto-103 68

The PIM register seems to be received on R3 (RP), but R3 (RP) should also send a PIM Register-stop back toward R4 as there is no receiver at the moment.

3. Start a debug flow with a filter on destination address 3.3.3.3
diagnose debug enable
diagnose debug flow show console enable
diagnose debug flow show function-name enable
diagnose debug console timestamp enable
diagnose debug flow filter daddr 3.3.3.3
diagnose debug flow trace start 100
2016-07-06 06:34:22 id=20085 trace_id=102 func=print_pkt_detail line=4478 msg="vd-R3 received a packet(proto=103, 10.4.0.4:0->3.3.3.3:0) from r3_r4_0. "
2016-07-06 06:34:22 id=20085 trace_id=102 func=init_ip_session_common line=4629 msg="allocate a new session-000056af"
2016-07-06 06:34:22 id=20085 trace_id=102 func=fw_local_in_handler line=385 msg="iprope_in_check() check failed on policy 0, drop"

It seems that the PIM Register message is denied by Firewall policies, check the Firewall policies on R3 (RP):
(R3) # show firewall policy
config firewall policy
    edit 1
        set uuid 4ab60d1a-435c-51e6-76b7-39c0c045e9e6
        set srcintf "r3_r4_0" "r1_r3_1"
        set dstintf "lo3"
        set srcaddr "R4_Loopback" 
        set dstaddr "R3_Loopback"
        set action accept
        set schedule "always"
        set service "PIM"
    next
end

(R3) # show firewall address R4_Loopback
config firewall address
    edit "R4_Loopback"
        set uuid 57546916-2bfd-51e6-c15c-a8f5ff213544
        set subnet 4.4.4.4 255.255.255.255
    next
end

PIM is allowed from 4.4.4.4. R4 initiates the PIM register with 10.4.0.4 as source (IP of the interface closest to the source), thus it is denied in R3 (RP) Firewall policies.

By default, R4 initiates the PIM register message with the IP of the interface closest to the source, in this case 10.4.0.4. This can be modified as follows:
config router multicast
    set multicast-routing enable
        config pim-sm-global
            set register-source ip-address
            set register-source-ip 4.4.4.4
        end
    end

Start a sniffer to make sure the source of the PIM register is 4.4.4.4:
(R3) # diag sniffer packet r3_r4_0 'proto 103' 4 0 l
interfaces=[r3_r4_0]
filters=[proto 103]
2016-07-06 06:22:58.167509 r3_r4_0 in 4.4.4.4 -> 3.3.3.3:  ip-proto-103 68
2016-07-06 06:22:58.167649 r3_r4_0 out 3.3.3.3 -> 4.4.4.4:  ip-proto-103 18

2 packets received by filter
0 packets dropped by kernel

Check the multicast routing table on R3 (RP):
FGT_1 (R3) # get router info multicast pim sparse-mode table 239.1.1.1
IP Multicast Routing Table

(*,*,RP) Entries: 0
(*,G) Entries: 2
(S,G) Entries: 6
(S,G,rpt) Entries: 10
FCR Entries: 0

(10.4.0.10, 239.1.1.1)
RPF nbr: 0.0.0.0
RPF idx: None
SPT bit: 0
Upstream State: NOT JOINED
 Local:
 Joined:
 Asserted:
 Outgoing:

(10.4.0.10, 239.1.1.1, rpt)
RP: 0.0.0.0
RPF nbr: 0.0.0.0
RPF idx: None
Upstream State: RPT NOT JOINED
 Local:
 Pruned:
 Outgoing:

The PIM registration process is now working and the RP is aware of the source IP which sends traffic to 239.1.1.1.

Note that the Upstream State is “NOT JOINED” because there is no receiver.

Contributors