Technical Tip: Configuring WCCP interception of HTTP traffic to a squid proxy using GRE tunneling

cgustave · ‎05-20-2009

Purpose

This article describes how to setup WCCP on the FortiGate to intercept and redirect HTTP traffic (TCP port 80) to a Linux squid transparent proxy.

GRE encapsulation is used to tunnel intercepted traffic to the squid. This article covers the Linux and squid configuration. Key concepts of the architecture are discussed. Verification and troubleshooting information is provided.

Scope
All FortiGates or VDOMs in NAT mode.
Diagram

Note :

We intentionally use a router between the FortiGate and the squid even if it is not a requirement. This is a key differentiator between the 2 WCCP encapsulation methods "GRE" and "L2 Forwarding". Unlike "L2 Forwarding", the GRE method allows to cross routers between the FortiGate and the caches.
The connection with the cache is using a specific interface in this example as this is the most common practice.

Expectations, Requirements

Expectations:

- uranium-mvm08 requests HTTP traffic on the Internet using TCP port 80 (browser is not configured to use a proxy)
- FGT3600 intercepts the traffic with a WCCP policy and redirects traffic to uranium-mvm03 squid transparent proxy through a GRE tunnel
- uranium-mvm03 squid gets the content from the Internet if not already in the cache via FortiGate port1
- uranium-mvm03 squid returns the content towards the FortiGate for uranium-mvm08 using the GRE tunnel
- FGT3600 receives the content from GRE and hooks it back to uranium-mvm08 using the WCCP policy

WCCP highlights:

WCCP stands for "Web Cache Coordinator Protocol". The idea is to transparently redirect user traffic to a cache (that is: no explicit web proxy configured on client web browser).

Client traffic going through the FortiGate is intercepted and redirected to a web proxy.
The web proxy gets the content if required and services the client by sending the content back to the FortiGate.
The FortiGate retransmitted the content from the web proxy to the client.

- WCCP is defined as an RFC draft
- It has built-in load-balancing and fault tolerance
- WCCP protocol uses UDP port 2048 to communicate between FortiGate and Caches
- WCCP has 2 types of services (well-known service id=0 for http and dynamic services (from id 51 to 255)
- WCCP allows 2 types of tunneling mechanisms ( GRE and L2 Forwarding)
Configuration

FortiGate

(only the relevant configuration is shown)

config system interface
    edit "port1"
        set vdom "root"
        set ip 10.120.0.225 255.255.254.0
        set allowaccess ping https ssh http telnet
        set type physical
        set wccp enable (#1)
    next
    edit "internal"
        set vdom "root"
        set ip 10.100.0.225 255.255.254.0
        set allowaccess ping https ssh http telnet
        set type physical
    next
    edit "external"
        set vdom "root"
        set ip 192.168.182.225 255.255.254.0
        set allowaccess ping https ssh http telnet
        set type physical
    next
end

config firewall address
    edit "all"
    next
    edit "squid-proxy"
        set associated-interface "port1"
        set subnet 10.102.0.83 255.255.255.255
    next
    edit "client-subnet"
        set associated-interface "internal"
        set subnet 10.100.0.0 255.255.255.0
    next
end
config firewall policy
    edit 3
        set srcintf "internal"
        set dstintf "external"
            set srcaddr "client-subnet"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "HTTP"
        set wccp enable (#2)
        set comments "HTTP TRAFFIC TO INTERCEPT"
        set nat enable
    next
    edit 1
        set srcintf "internal"
        set dstintf "external"
            set srcaddr "all"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "DNS" "PING"
        set comments "CLIENT NON HTTP TRAFFIC DIRECT"
        set nat enable
    next
    edit 2
        set srcintf "port1"
        set dstintf "external"
            set srcaddr "squid-proxy"
            set dstaddr "all"
        set action accept
        set schedule "always"
            set service "DNS" "HTTP" "PING"
        set comments "PROXY TRAFFIC"
        set nat enable
    next
end
config router static
    edit 1
        set device "external"
        set gateway 192.168.183.254
    next
    edit 2
        set device "port1"
        set dst 10.102.0.0 255.255.254.0
        set gateway 10.120.0.76
    next
end

config system wccp
    edit "0" (#3)
        set router-id 10.120.0.225 (#4)
        set server-list 10.102.0.83 255.255.255.255 (#5)
        set authentication enable (#6)
        set password ENC /pSXIHLNBkcXgQD5XOAhnq0dfgOcHsFwIAp3yDVWa6zWMZTM+y1dolT5yxO3GZgKAQsOluZ21HNzi+qxxOJ0hUfrwDhEJwFFbuVzAXEP9XtZPdBh (#7)
    next
end

Keys of the configuration:

#1 : the interface toward the proxy must be 'set wccp enable' to designate on which interface the GRE tunnel should be starting
#2 : the policy intercepting the traffic must be marked 'set wccp enable'
#3 : in WCCP '0' refers to the standard service (tcp port 80) so it is required to have edit "0" in this case
#4 : router-id used in WCCP (generally the interface ip address towards the cache)
#5 : defines the authorized cache IP addresses (it could be also 10.102.0.0 255.255.255.0 in this example)
#6 : use a shared WCCP secret between the FortiGate and the cache for authentication (this is optional)
#7 : the shared secret key if authentication is enabled (step #6) . We use 'fortinet' in this example.

LINUX + Squid server

(only the relevant configuration is shown)

In the first place we focus on the networking settings for the Linux server, then we highlight the relevant parts of the squid configuration.

# ip addresses and default route :
ifconfig eth0 10.102.0.83 netmask 255.255.254.0
route add default gw 10.102.0.76

# GRE Tunnel :
modprobe ip_gre
ip tunnel add wccp0 mode gre remote 10.120.0.225 local 10.102.0.83 dev eth0
ip addr add 10.102.0.83/32 dev wccp0
ip link set wccp0 up

# Route to send the content back to the GRE tunnel
route add -net 192.168.182.225/32 dev wccp0

# Disabling reverse path filtering and enable routing in the kernel
echo 0 > /proc/sys/net/ipv4/conf/wccp0/rp_filter
echo 1 > /proc/sys/net/ipv4/ip_forward

# Setup the redirection of traffic from the GRE tunnel to squid port 3128
iptables -t nat -F
iptables -t nat -A PREROUTING -i wccp0 -m tcp -p tcp -j REDIRECT --to-ports 3128

/etc/squid/squid.conf

# setup transparent squid on port 3128
http_port 3128 transparent

# Allow clients to request contents
http_access allow all

# FortiGate interface of wccp
wccp2_router 10.120.0.225

# wccp version 2 configuration for standard service HTTP on tcp port 80 (service 0) with authentication password 'fortinet'
wccp2_service standard 0 password=fortinet

# tunneling method GRE for forward traffic
wccp2_forwarding_method 1

# tunneling method GRE for return traffic
wccp2_return_method 1

# Assignemment method (default), only relevant if multiple caches used
wccp2_assignment_method 1

# wccp weight (default) ,only relevant if multiple caches used
wccp2_weight 10000

# which interface to use for WCCP (0.0.0.0 determines the interface from routing)
wccp2_address 0.0.0.0

Verification

WCCP debug

diagnose test application wccpd 1 (statistics)

FG3600-4 # diagnose test application wccpd 1
vdoms=1
pkts=0

1 vdom using wccp

diagnose test application wccpd 2 (wccp configuration)

FG3600-4 # diagnose test application wccpd 2
vdom-root: working NAT first_phy_id=2
interface list:
    intf=port1, gid=2 phy_id=2
service list:
    service: 0, router_id=10.120.0.225, group=0.0.0.0, auth(fortinet)
      access:10.102.0.83/255.255.255.255 forward=1
      return=1, assign=1.
      erouter_id=10.120.0.225

wccp configuration showing the wccp interface, the service id and router-id, authentication active with password 'fortinet'
cache ip in GRE forward method

diagnose test application wccpd 3 (servers)

FG3600-4 # diagnose test application wccpd 3
service-0 in vdom-root: num=1, usable=1
cache server ID:
len=44, addr=10.102.0.83, weight=4135, status=0
rcv_id=51, usable=1, fm=1, nq=0, dev=2(k2), to=10.120.0.225
ch_no=0, num_router=1:
10.120.0.225

Server (caches) available and usable

diagnose test application wccpd 4 (services)

FG3600-4 # diagnose test application wccpd 4
service-0 in vdom-root:
total_servers=1, usable_servers=1, assign_m=1, rtun_m=1, wcid_len=48, rcv_id=54, ch_no=2
ID=0, type=0, pri=0, pro=0 f=00000000
Port: 0 0 0 0 0 0 0 0 num-routers=1:
10.120.0.225

Active services details (id=0 for http on port tcp 80) with servers that can handle the service

diagnose test application wccpd 5 (assignement)

FG3600-4 # diagnose test application wccpd 5
service-0 in vdom-root:

key: ip=10.102.0.83, change-number=1
cache_list: 1
0. 10.102.0.83
primary assignment:
key=10.102.0.83 change-number=1
num_routers=1
router element[0]: router_id=10.120.0.225, receive_id=4, ch_no=2
cache-server-num=1:
10.102.0.83
buckets:
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Assignment : defines a cache id (here only 0 as we have only 1 cache) for each WCCP bucket.
This is only relevant if multiple caches are used.
This assignment is determinate by the proxies and sent to the FortiGate via WCCP protocol

firewall session entry for wccp interception

You can identify from the session list (in CLI only using 'diag sys session list') a firewall session which is intercepted by wccp and determinate which cache is used as target. It also confirms that the WCCP peering between the FortiGate and the cache is active.

session info: proto=6 proto_state=01 expire=3599 timeout=3600 flags=00000000 sockflag=00000000 sockport=0 av_idx=0 use=4
origin-shaper=
reply-shaper=
ha_id=0 hakey=53608
policy_dir=0 tunnel=/
state=wccp may_dirty nlb rem
statistic(bytes/packets/allow_err): org=598/4/1 reply=1967/4/1 tuples=2
orgin->sink: org pre->post, reply pre->post dev=3->4/4->3 gwy=192.168.183.254/10.100.0.88
hook=post dir=org act=snat 10.100.0.88:1752->4.71.209.16:80(192.168.182.225:50780)
hook=pre dir=reply act=dnat 4.71.209.16:80->192.168.182.225:50780(10.100.0.88:1752)
pos/(before,after) 0/(0,0), 0/(0,0)
misc=0 policy_id=3 auth_info=0 chk_client_info=0 vd=0 serial=000091f5 tos=ff/ff app=0
dd_type=0 dd_rule_id=0
wccp: server=10.102.0.83, rid=10.120.0.225, dev=2, pri=180, alt=0 GRE-F L2-R std

Cache 10.102.0.83 is used to deliver the content via wccp via FortiGate interface 10.120.0.225 using GRE encapsulation
Note : L2-R is irrelevant in this output

Note : In this example, if the WCCP peering goes down (for instance if the cache is not reachable), user traffic would be sent directly to the Internet with no interception. If the preferred behavior is not to deliver the traffic to user if the cache is not reachable, a simple trick is to disable natting on policy 3. Without nat and wccp active traffic can only be served to client via the cache only.

SQUID access logs

Activity of the proxy is traceable from the squid logs

/var/log/squid/access.log
1242672135.232      1 10.100.0.88 TCP_MEM_HIT/200 3548 GET http://search.cpan.org/author/ - NONE/- text/html
1242672135.388     39 10.100.0.88 TCP_MEM_HIT/200 1998 GET http://st.pimg.net/tucs/style.css - NONE/- text/css
1242672135.401     12 10.100.0.88 TCP_MEM_HIT/200 627 GET http://st.pimg.net/tucs/print.css - NONE/- text/css
1242672135.401     12 10.100.0.88 TCP_MEM_HIT/200 2362 GET http://st.pimg.net/tucs/img/DC-LOGO-S.gif - NONE/- image/gif
1242672135.402     13 10.100.0.88 TCP_MEM_HIT/200 5610 GET http://st.pimg.net/tucs/img/cpan_banner.png - NONE/- image/png
1242672135.575      1 10.100.0.88 TCP_HIT/200 9617 GET http://www.google-analytics.com/ga.js - NONE/- text/javascript
1242672135.782     13 10.100.0.88 TCP_MEM_HIT/200 772 GET http://search.cpan.org/favicon.ico - NONE/- image/x-icon
1242672487.062    144 10.100.0.88 TCP_MISS/404 2927 GET http://search.cpan.org/search?query=&mode=author - DIRECT/84.45.68.23 text/html

Troubleshooting
debug flow

Output example of 'diag debug flow' showing in this example how client traffic is processed by the FortiGate

id=20085 trace_id=100222 func=resolve_ip_tuple_fast line=3196 msg="vd-root received a packet(proto=6, 10.100.0.88:1658->84.45.68.23:80) from internal."
id=20085 trace_id=100222 func=resolve_ip_tuple line=3308 msg="allocate a new session-00001c3b"
id=20085 trace_id=100222 func=vf_ip4_route_input line=1607 msg="find a route: gw-192.168.183.254 via external"
id=20085 trace_id=100222 func=get_new_addr line=1481 msg="find SNAT: IP-192.168.182.225, port-34558"
id=20085 trace_id=100222 func=fw_forward_handler line=366 msg="Allowed by Policy-3: SNAT"
id=20085 trace_id=100222 func=__ip_session_run_tuple line=1663 msg="SNAT 10.100.0.88->192.168.182.225:34558"
id=20085 trace_id=100222 func=wccp_output line=222 msg="gre_forward"
id=20085 trace_id=100222 func=wccp_output line=264 msg="send packet via dev-port1"
id=20085 trace_id=100223 func=resolve_ip_tuple_fast line=3196 msg="vd-root received a packet(proto=47, 10.102.0.83:0->10.120.0.225:0) from port1."
id=20085 trace_id=100223 func=wccp_gre_decap line=380 msg="feed to dev=external"
id=20085 trace_id=100223 func=__ip_session_run_tuple line=1677 msg="DNAT 192.168.182.225:34558->10.100.0.88:1658"
id=20085 trace_id=100223 func=vf_ip4_route_input line=1607 msg="find a route: gw-10.100.0.88 via internal"
id=20085 trace_id=100224 func=resolve_ip_tuple_fast line=3196 msg="vd-root received a packet(proto=6, 10.100.0.88:1658->84.45.68.23:80) from internal."
id=20085 trace_id=100224 func=resolve_ip_tuple_fast line=3224 msg="Find an existing session, id-00001c3b, original direction"
id=20085 trace_id=100224 func=__ip_session_run_tuple line=1663 msg="SNAT 10.100.0.88->192.168.182.225:34558"
id=20085 trace_id=100224 func=wccp_output line=222 msg="gre_forward"
id=20085 trace_id=100224 func=wccp_output line=264 msg="send packet via dev-port1"
id=20085 trace_id=100225 func=resolve_ip_tuple_fast line=3196 msg="vd-root received a packet(proto=6, 10.100.0.88:1658->84.45.68.23:80) from internal."
id=20085 trace_id=100225 func=resolve_ip_tuple_fast line=3224 msg="Find an existing session, id-00001c3b, original direction"
id=20085 trace_id=100225 func=__ip_session_run_tuple line=1663 msg="SNAT 10.100.0.88->192.168.182.225:34558"
id=20085 trace_id=100225 func=wccp_output line=222 msg="gre_forward"
id=20085 trace_id=100225 func=wccp_output line=264 msg="send packet via dev-port1"
id=20085 trace_id=100226 func=resolve_ip_tuple_fast line=3196 msg="vd-root received a packet(proto=47, 10.102.0.83:0->10.120.0.225:0) from port1."
id=20085 trace_id=100226 func=wccp_gre_decap line=380 msg="feed to dev=external"