Technical Tip: Web filtering FortiGuard category does not block web sites as expected when custom certificate port is used

[cborgato_FTNT]

Description
This article describes with an example what is the possible cause of ‘WebFiltering FortiGuard Category may no block websites’ and how the WAD process works in such cases with web filter feature.

Solution
After an upgrade from 5.4 (or older releases) to 5.6 (or newer releases), web sites blocked using FortiGuard WebFilter categories are not blocked anymore even if the FortiGate reaches correctly FortiGuard services.

In other worlds, web filter with FortiGuard category is not blocking anymore web site accesses.

NetWork Design.

A condition for this particular issue is to have an external web Ppoxy involved.

Client PC -> FortiGate -> WebProxy -> Internet.

Configuration of Interest.

In the following example the following configuration is coming from older version (only most interested parts are reported!).

- Web filtering in proxy mode blocking FortiGuard categories.

# config webfilter profile
    edit "WAF_Categ_Profile" <<< (Default Inspection Mode: Proxy)
        # config ftgd-wf
            # config filters
                …
                edit 14
                    set category 14 <<< ("Pornography" FortiGuard category)
                    set action block
                next
                …
            next
        end
    end
end

- Certificate inspection with custom port HTTPS 8080.

# config firewall ssl-ssh-profile
    edit "SSL_Certificate_Profile"
        # config https
            set ports 8080
            set status certificate-inspection
            set allow-invalid-server-cert enable
            …
        end
    next
end

- Protocol option with custom HTTP port 8080.

# config firewall profile-protocol-options
    edit "Protocol_Profile”
        # config http
            set ports 8888 8080 8088
            unset options
            unset post-lang
            set scan-bzip2 disable
        end
        …
    next
end

Debug to Use.

# diagnose debug application urlfilter -1                            
# diagnose debug urlfilter test-url www.youporn.com

Test with issue does not show any message related to youporn.

How to fix it.

Leave the certificate inspection with default HTTPS 443.

# config firewall ssl-ssh-profile
    edit "SSL_Certificate_Profile"
        # config https
            set ports 443
            set status certificate-inspection
            set allow-invalid-server-cert enable
            …
        end
    next
end

The debugs shoul then show:

msg="received a request /tmp/.wad_325_0_0.url.socket, addr_len=31: d=youporn.com:80, id=41, vfname='root', vfid=4, profile='WAF_Categ_Profile', type=0, client=1.2.3.4, url_source=1, url="/"
msg="Cache miss" user="N/A" src=1.2.3.4 sport=52419 dst=216.5.6.7 dport=80 service="http" hostname="youporn.com" url="/"

Explanation.

On FOS 5.6, process WAD is on charge of all the 'checks' steps (in this case is web filtering, SSL inspection and proxy).
In FOS 5.0 those 'checks' were handled in different way by multiple processes (proxy worker, Url filtering and SSLxx process).

The way WAD works does not require to specify port 8080 on certificate inspection.

When Client tries to reach the website that is blocked (for example you.porn.com) using proxy, it will send.

- HTTP to proxy with dport 8080 with 'HTTP CONNECT www.youporn.com:443"'in HTTP header.
- Then proxy will do HTTPS to the website (www.youporn.com:443).

WAD knows that must inspect traffic on port 8080 thanks to 'protocol option with custom HTTP port 8080' (in this case was ' Protocol_Profile’) between client and proxy.

When it starts to check the HTTP content, it sees 'HTTP Connect www.youporn.com:443'.
 
In order to analyze the HTTPS, WAD needs to use the certificate Inspection profile (in this case was SSL_Certificate_Profile) that says to inspect SSL on port 8080.

WAD tries to inspect 8080 for SSL, but actually the one used is 443.

Workarounds.

- Leave the certificate inspection to default HTTPS 443.
- Change the inspection mode to flow mode instead of proxy mode in order to use IPS engine.

Related Articles

Troubleshooting Tip: FortiGuard Web Filtering problems

Technical Tip: Change the security profile inspection mode

Technical Tip: Inspection Mode Changes