Technical Note: LDAP Nested Group settings and changes in FortiOS 5.6, 6.0 and 6.2
Products
FortiGate v5.6
FortiGate v6.0
FortiGate v6.2
Description

This article describes how to modify the LDAP Nested group settings. 

In order to authenticate user via LDAP while the user is not a direct member of the group, but member of nested group, set FortiGate in the way it will be able to check for nested groups inside LDAP.

By default, any nested group check support is disabled.

On the previous versions of FortiOS, 'set search-type nested' option was available, in which on FortiOS 5.6 and 6.0 are not available.

In FortiOS 5.6 and 6.0,
'search-type nested' was replaced with 'extended group-filter'.

However,
in FortiOS 6.2, 'search-type recursive' is available.

Note: The option 'nested' is replaced with 'recursive'

Solution
On FortiOS 5.6 and 6.0 the 'search-type nested' option was replaced with more flexible group-filter setting in LDAP user configuration.
config user ldap
    edit "LDAP-AD-SERVER"
        set <other necessary settings like server,dn etc>
        set group-filter "(|(&(objectclass=group)(member:1.2.840.113556.1.4.1941:=%u))(&(objectClass=group)(member:1.2.840.113556.1.4.1941:=%pg)))"
end
Since FortiOS 6.2.0, there is 'search-type recursive' in user LDAP config and this option is still in CLI only.
It brings similar results as older 'nested' option, but in contrast to group-filter the results do not contain AD Builtin (CN=Builtin,DC=YourDomain..) user groups. See examples in verification section.

config user ldap
    edit "LDAP-AD-SERVER"
        set search-type recursive
end

FortiOS upgrade to 6.2:

1) Group-filter option is not automatically changed to recursive search-type during upgrade process, because group-filter might be more customized than the above mentioned simple example.

2) Group-filter option used in FortiOS 5.6-6.0 is still valid in FortiOS 6.2 and works as before, so there is no changes needed manually.

3) Once search-type is set, group-filter option will be hidden and disabled! So use either simple search-type or complex group filter.

4) Another point of view is CPU load on DC, as group-filter is passed to LDAP in query and all the group processing is done by DC, which might be CPU intensive if the filter is not specific enough. Search-type option make fnbamd to issue subsequent queries to resolve group nesting, and this is lighter on DC, but more intensive in network traffic and CPU usage on FortiGate side.

5) Search-type recursive at the moment do not resolve groups nested under primary user group. If it is needed, use group filter as mentioned above, even on FortiOS 6.2.

Following is an example of the configuration.

Diagram:

User "Test-nested" is member of group "grp-nested-L2".
Group "grp-nested-L2" is member of "grp-nested-L1".
Group "grp-nested-L1" is member of "grp-nested-L0".

Configuration GUI:

Group filtering is not configurable in GUI, it can only be configured in the CLI.

Configuration CLI - FortiOS 5.6 and 6.0:
config user ldap
    edit "C24_ALFA.XSILVER.ORG"
        set server "10.109.19.88"
        set cnid "cn"
        set dn "dc=alfa,dc=xsilver,dc=org"
        set type regular
        set username "cn=administrator,cn=users,dc=alfa,dc=xsilver,dc=org"
        set password supersecretpassword
        set group-filter "(|(&(objectclass=group)(member:1.2.840.113556.1.4.1941:=%u))(&(objectClass=group)(member:1.2.840.113556.1.4.1941:=%pg)))"
    next
end
Configuration CLI - FortiOS 6.2:
config user ldap
    edit "C24_ALFA.XSILVER.ORG"
        set server "10.109.19.88"
        set cnid "cn"
        set dn "dc=alfa,dc=xsilver,dc=org"
        set type regular
        set username "cn=administrator,cn=users,dc=alfa,dc=xsilver,dc=org"
        set password supersecretpassword
        set search-type recursive
    next
end

Verification of Configuration and troubleshooting:

Use the configuration in the example of FortiOS 5.6/6.0 and described LDAP structure. When doing test authserver LDAP with the group filter handling nested groups, all the groups from nested chain will be displayed.

diag test authserver ldap C24_ALFA.XSILVER.ORG test-nested tester

authenticate 'test-nested' against 'C24_ALFA.XSILVER.ORG' succeeded!

Group membership(s) - CN=Domain Users,CN=Users,DC=alfa,DC=xsilver,DC=org
                      CN=Users,CN=Builtin,DC=alfa,DC=xsilver,DC=org
                      CN=Remote Desktop Users,CN=Builtin,DC=alfa,DC=xsilver,DC=org
                      CN=grp-nested-L0,CN=Users,DC=alfa,DC=xsilver,DC=org
                      CN=grp-nested-L1,CN=Users,DC=alfa,DC=xsilver,DC=org
                      CN=grp-nested-L2,CN=Users,DC=alfa,DC=xsilver,DC=org
Without group-filter, it will only display groups that the user is direct MemberOf (grp-nested-L2 especially in our test case)
diag test authserver ldap C24_ALFA.XSILVER.ORG test-nested tester

authenticate 'test-nested' against 'C24_ALFA.XSILVER.ORG' succeeded!

Group membership(s) - CN=grp-nested-L2,CN=Users,DC=alfa,DC=xsilver,DC=org
                      CN=Domain Users,CN=Users,DC=alfa,DC=xsilver,DC=org
When group-filter is removed and search-type recursive is used, as in config example from FortiOS 6.2, it will display all the custom and nested groups as before. However, it will not display any AD Builtin groups (compare outputs and pay attention to 'CN=Users,CN=Builtin' and 'CN=Remote Desktop Users,CN=Builtin' groups).
diag test authserver ldap LDAP-C24-ALFA test-nested tester

authenticate 'test-nested' against 'LDAP-C24-ALFA' succeeded!

Group membership(s) - CN=grp-nested-L2,CN=Users,DC=alfa,DC=xsilver,DC=org
                      CN=Domain Users,CN=Users,DC=alfa,DC=xsilver,DC=org
                      CN=grp-nested-L1,CN=Users,DC=alfa,DC=xsilver,DC=org
                      CN=grp-nested-L0,CN=Users,DC=alfa,DC=xsilver,DC=org
Sample fnbamd debug output is as follows:
diag debug application fnbamd 7
diag debug enable
diag test authserver ldap LDAP-C24-ALFA test-nested tester

[2273] handle_req-Rcvd auth req 588851107 for test-nested in LDAP-C24-ALFA opt=0000001b prot=0
[614] fnbamd_pop3_start-test-nested
[1042] __fnbamd_cfg_get_ldap_list_by_server-Loading LDAP server 'LDAP-C24-ALFA'
[1662] fnbamd_ldap_init-search filter is: cn=test-nested
[1671] fnbamd_ldap_init-search base is: dc=alfa,dc=xsilver,dc=org
[1019] __fnbamd_ldap_dns_cb-Resolved LDAP-C24-ALFA(idx 0) to 10.109.19.88
[1087] __fnbamd_ldap_dns_cb-Still connecting.
[557] create_auth_session-Total 1 server(s) to try
[969] __ldap_connect-tcps_connect(10.109.19.88) is established.
[204] __ldap_build_bind_req-Binding to 'administrator@alfa.xsilver.org'
[937] fnbamd_ldap_send-Request is sent. ID 1
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:1, type:bind
[864] fnbamd_ldap_parse_response-ret=0
[910] __ldap_rxtx-Change state to 'DN search'
[592] fnbamd_ldap_build_dn_search_req-base:'dc=alfa,dc=xsilver,dc=org' filter:cn=test-nested
[937] fnbamd_ldap_send-Request is sent. ID 2
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:2, type:search-entry
[864] fnbamd_ldap_parse_response-ret=0
[1180] __fnbamd_ldap_dn_entry-Get DN 'CN=test-nested,CN=Users,DC=alfa,DC=xsilver,DC=org'
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:2, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:2, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:2, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:2, type:search-result
[864] fnbamd_ldap_parse_response-ret=0
[910] __ldap_rxtx-Change state to 'User Binding'
[437] fnbamd_ldap_build_userbind_req-Trying DN 'CN=test-nested,CN=Users,DC=alfa,DC=xsilver,DC=org'
[204] __ldap_build_bind_req-Binding to 'CN=test-nested,CN=Users,DC=alfa,DC=xsilver,DC=org'
[937] fnbamd_ldap_send-Request is sent. ID 3
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:3, type:bind
[864] fnbamd_ldap_parse_response-ret=0
[910] __ldap_rxtx-Change state to 'Attr query'
[490] fnbamd_ldap_build_attr_search_req-Adding attr 'memberOf'
[502] fnbamd_ldap_build_attr_search_req-base:'CN=test-nested,CN=Users,DC=alfa,DC=xsilver,DC=org' filter:cn=*
[937] fnbamd_ldap_send-Request is sent. ID 4
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:4, type:search-entry
[864] fnbamd_ldap_parse_response-ret=0
[553] __get_member_of_groups-Get the memberOf groups.
[519] __retrieve_group_values-Get the memberOf groups.
[530] __retrieve_group_values- attr='memberOf', found 1 values
[539] __retrieve_group_values-val[0]='CN=grp-nested-L2,CN=Users,DC=alfa,DC=xsilver,DC=org'
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:4, type:search-result
[864] fnbamd_ldap_parse_response-ret=0
[1260] __fnbamd_ldap_attr_next-Entering CHKPRIMARYGRP state
[910] __ldap_rxtx-Change state to 'Primary group query'
[526] fnbamd_ldap_build_primary_grp_search_req-starting primary group check...
[530] fnbamd_ldap_build_primary_grp_search_req-number of sub auths 5
[548] fnbamd_ldap_build_primary_grp_search_req-base:'dc=alfa,dc=xsilver,dc=org' filter:(&(objectclass=group)(objectSid=\01\05\00\00\00\00\00\05\15\00\00\00\bb\f4\90\f4\30\bd\1f\79\0b\35\92\63\01\02\00\00))
[937] fnbamd_ldap_send-Request is sent. ID 5
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:5, type:search-entry
[864] fnbamd_ldap_parse_response-ret=0
[470] __get_one_group-group: CN=Domain Users,CN=Users,DC=alfa,DC=xsilver,DC=org
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:5, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:5, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:5, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:5, type:search-result
[864] fnbamd_ldap_parse_response-ret=0
[910] __ldap_rxtx-Change state to 'User recursive group query'
[735] fnbamd_ldap_build_recursive_grp_search_req-base:'dc=alfa,dc=xsilver,dc=org' filter:(&(objectclass=group)(|(distinguishedName=CN=grp-nested-L2,CN=Users,DC=alfa,DC=xsilver,DC=org)))
[937] fnbamd_ldap_send-Request is sent. ID 6
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:6, type:search-entry
[864] fnbamd_ldap_parse_response-ret=0
[519] __retrieve_group_values-Get the memberOf groups.
[530] __retrieve_group_values- attr='memberOf', found 1 values
[539] __retrieve_group_values-val[0]='CN=grp-nested-L1,CN=Users,DC=alfa,DC=xsilver,DC=org'
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:6, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:6, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:6, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:6, type:search-result
[864] fnbamd_ldap_parse_response-ret=0
[372] __ldap_grp_list_need_more_check-Group 'CN=grp-nested-L1,CN=Users,DC=alfa,DC=xsilver,DC=org' is not checked
[910] __ldap_rxtx-Change state to 'User recursive group query'
[735] fnbamd_ldap_build_recursive_grp_search_req-base:'dc=alfa,dc=xsilver,dc=org' filter:(&(objectclass=group)(|(distinguishedName=CN=grp-nested-L1,CN=Users,DC=alfa,DC=xsilver,DC=org)))
[937] fnbamd_ldap_send-Request is sent. ID 7
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:7, type:search-entry
[864] fnbamd_ldap_parse_response-ret=0
[519] __retrieve_group_values-Get the memberOf groups.
[530] __retrieve_group_values- attr='memberOf', found 1 values
[539] __retrieve_group_values-val[0]='CN=grp-nested-L0,CN=Users,DC=alfa,DC=xsilver,DC=org'
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:7, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:7, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:7, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:7, type:search-result
[864] fnbamd_ldap_parse_response-ret=0
[372] __ldap_grp_list_need_more_check-Group 'CN=grp-nested-L0,CN=Users,DC=alfa,DC=xsilver,DC=org' is not checked
[910] __ldap_rxtx-Change state to 'User recursive group query'
[735] fnbamd_ldap_build_recursive_grp_search_req-base:'dc=alfa,dc=xsilver,dc=org' filter:(&(objectclass=group)(|(distinguishedName=CN=grp-nested-L0,CN=Users,DC=alfa,DC=xsilver,DC=org)))
[937] fnbamd_ldap_send-Request is sent. ID 8
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:8, type:search-entry
[864] fnbamd_ldap_parse_response-ret=0
[519] __retrieve_group_values-Get the memberOf groups.
[524] __retrieve_group_values-attr='memberOf' - found 0 values
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:8, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:8, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:8, type:search-reference
[864] fnbamd_ldap_parse_response-ret=0
[829] fnbamd_ldap_parse_response-Got one MESSAGE. ID:8, type:search-result
[864] fnbamd_ldap_parse_response-ret=0
[1420] __fnbamd_ldap_recursive_grp_next-Auth accepted
[910] __ldap_rxtx-Change state to 'Done'
[937] fnbamd_ldap_send-Request is sent. ID 9
[3053] fnbamd_ldap_result-Result for ldap svr 10.109.19.88 is SUCCESS
[3067] fnbamd_ldap_result-Skipping group matching
[989] find_matched_usr_grps-Skipped group matching
[182] fnbamd_comm_send_result-Sending result 0 (error 0, nid 0) for req 588851107
[713] destroy_auth_session-delete session 588851107

authenticate 'test-nested' against 'LDAP-C24-ALFA' succeeded!

Group membership(s) - CN=grp-nested-L2,CN=Users,DC=alfa,DC=xsilver,DC=org

                      CN=Domain Users,CN=Users,DC=alfa,DC=xsilver,DC=org
                      CN=grp-nested-L1,CN=Users,DC=alfa,DC=xsilver,DC=org
                      CN=grp-nested-L0,CN=Users,DC=alfa,DC=xsilver,DC=org





Related Articles
Technical Note: LDAP Nested Group authentication for SSLVPN
Last Modified Date: 04-30-2019 Document ID: FD41657