Home > Linux, Mobile Tele-communication, Networking > Setting up crazy caching for orange 3G modem on FreeBSD

Setting up crazy caching for orange 3G modem on FreeBSD

This is a guide to illustrate how to set up a 3G USB modem on FreeBSD. Due to the cap limit on the 3G plans on the market, there is need to maximize link usage hence the guide illustrates how to setup squid + squidGuard + bind caching server. On capped links you need to monitor usage of the link, so darkstat will come in handy.

In this setup I used an old ASUS board PC with Celeron processor 500 MHz, 256MB RAM + 150GB disk I had stashed away; it is the perfect piece for the Job. To be able to access internet all around the house, I purchased a cheap TP Link wireless ADSL 2+ modem router, it goes for around UGX 100,000, that’s around EUR 37.

I mostly use Debian and Fedora derivatives, but in this setup the choice of operating system is FreeBSD reason being it can be easily setup to have a small resource footprint based on the hardware specs above and I have taken about two years without using FreeBSD. So I needed something to get my hands dirty with. In this setup I used the 7.1 release of FreeBSD.

Home network structure

Home network structure

1. Install FreeBSD

When installing FreeBSD create a separate partition for the squid cache. The mount point of that partition can be /usr/local/squid/cache but I set mine to /var/spool/cache. You can choose any, u just have to remember to update the cache_dir directive in the squid.conf after installation.

2. Configure Network Devices

I use the Myson ethernet card, so to decide the driver to load for my network card. I used the shortcut of going through the /boot/defaults/loader.conf looking for the comments relating to my card. My network card driver is if_my. If you have not found your driver, check the FreeBSD handbook.

Add the directive to load the network interface card driver during boot in the /boot/loader.conf

if_my_load=”YES”

My network device is my0. We need to configure the ip address of the ethernet card. so add:-

ifconfig_my0=”inet 192.168.1.1 netmask 255.255.255.0″
gateway_enable=”YES”
defaultrouter=”192.168.1.1″

to the /etc/rc.conf
For the HUAWEI 3G Modem, FreeBSD comes with a u3g driver for 3G usb modems. To load the u3g driver add u3g_load=”YES” to the /boot/loader.conf. I made use of Nick Hibma’s guide of setting up the modem and ppp. In /boot/loader.conf, add

u3g_load=”YES”

Net we need to configure ppp to add a new orange profile. Modify the /etc/ppp/ppp.conf to look as below:-

default:
set device /dev/cuaU0.0

orange:
set speed 115200
set timeout 0
set authname “”
set authkey “”

set dial “ABORT BUSY TIMEOUT 2 \
\”\” \
AT OK-AT-OK \
AT+CFUN=1 OK-AT-OK \
AT+CMEE=2 OK-AT-OK \
AT+CSQ OK \
AT+CGDCONT=1,\\\”IP\\\”,\\\”orange.ug\\\” OK \
AT+CGACT? OK-AT-OK \
AT+CGATT? OK \
AT+CGCLASS? OK \
AT+COPS? OK \
ATD*99***1# CONNECT”

set crtscts on
disable vjcomp
disable acfcomp
disable deflate
disable deflate24
disable pred1
disable protocomp
disable mppe
disable ipv6cp
disable lqr
disable echo
nat enable yes

set ifaddr 10.0.1/0 10.0.0.1/0 255.255.255.255 0.0.0.0
add! default HISADDR # See ppp.link*

create a new file /etc/ppp/ppp.linkup and add the following lines

orange:
shell route delete default
shell route add default -interface INTERFACE

Create a new file /etc/ppp/ppp.linkdown and add the following lines

orange:
shell route delete default

When a user attaches the usbmodem, we need to initiate a ppp link.
Connect the modem and execute command usbdevs -v, it should display details about usb devices connected. You get output such as:-

port 1 addr 2: full speed, self powered, config 1, HUAWEI Mobile(0x1003), HUAWEI Technology(0x12d1), rev 0.00

With vender 0x12d1 and product 0x1003, edit the /etc/devd.conf and add the folowing config

attach 100 {
match “device-name” “ucom[0-9]+”;
match “vendor” “0x12d1”;
match “product” “0x1003”;
match “devclass” “0x00”;
action “/usr/sbin/ppp -ddial orange”;
};

The config is for devd. When a device that matches the name, vendor and product is attached (which is the modem), ppp is executed and uses the orange profile in the ppp.conf.
At the end of this step we should be able to test that ppp works by executing ppp -ddial orange. A well configured ppp interface (tun0 or tun1 or tun?) should be up and running.
You will not be able to ping any domain since ppp will not update nameserver list in /etc/resolv.conf. If your not planning to setup a caching only local server, you can go back to editing /etc/ppp/ppp.conf and add enable dns under the orange profile, then when ppp receives nameserver records from the ISP, it will update /etc/resolv.conf.

2. Setup Bind caching name server
Bind is going to be used as caching only name server.
So you only need to modify a few directives as below.


listen-on { 127.0.0.1; 192.168.1.1; };
forward first;
forwarders { 41.202.229.144;41.202.229.140; };

Bind is going to listen to the loopback interface and internal ip address in the interface card.
The list of forwarders above are orange DNS servers.
forward first causes the server to query the forwarders list above before it looks up the answer itself.

Update the /etc/resolv.conf to have only one nameserver directive pointing to 127.0.0.1 (the bind server setup)


nameserver 127.0.0.1

We need to enable bind to start during boot so add named_enable=”YES” to the /etc/rc.conf
2. Setup a transparent Squid proxy caching server
To install squid change to the squid port directory /usr/ports/www/squid and type make install.

After installation changes made to squid.conf include:-

  1. Configuring squid as a transparent proxy.

    http_port 192.168.1.1:3128 transparent
    http_port 127.0.0.1:3128 transparent

  2. Modifying refresh patterns to increase the time objects expire in the cache.
    I made use of the refresh patterns in Solomon Asare’s post on Speed up your Internet access using Squid’s refresh patterns. I only increased the maximum cache object age (in minutes). With the final list looking like

    refresh_pattern ^ftp: 20160 20% 43200
    refresh_pattern ^gopher: 1440 0% 43200
    refresh_pattern -i (wikimedia\.org|wikipedia\.org|howstuffworks\.com|wiki|about.com) 40320 100% 43200 override-expire ignore-no-cache ignore-private override-lastmod ignore-reload ignore-stale-while-revalidate
    refresh_pattern -i \.(gif|png|jpg|jpeg|ico) 10080 90% 43200 override-expire ignore-no-cache ignore-private
    refresh_pattern -i \.(iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv) 43200 90% 432000 override-expire ignore-no-cache ignore-private
    refresh_pattern -i \.(deb|rpm|exe|zip|tar|tgz|ram|rar|bin|ppt|doc|tiff) 10080 90% 43200 override-expire ignore-no-cache ignore-private
    refresh_pattern -i index\.(html|htm|shtml) 10080 40% 20160
    refresh_pattern -i \.(html|htm|css|js) 10080 40% 40320
    refresh_pattern (cgi-bin|\?) 1440 40% 20160
    refresh_pattern . 1440 75% 40320

  3. To further my crazy caching
    I added or modified the squid.conf directives as below

    max_stale 4 week
    maximum_object_size 20480 KB
    cache_dir ufs /var/spool/squid/cache 15000 128 512
    vary_ignore_expire on
    offline_mode on

    Also to avoid squid from making direct dns queries. Modify squid.conf dns nameservers directive to:-

    dns_nameservers 127.0.0.1

  4. Configure squidGuard redirector.
    To avoid loading of embedded videos and ads in webpages. I decided to add a squidGuard redirector to squid to block ads and audio or video sites. To install squidGuard change directory to /usr/ports/www/squidguard and type make install. After installation, modify squid.conf by adding

    redirect_program /usr/local/bin/squidGuard -c /usr/local/etc/squid/squidGuard.conf

    I made use of MESD blacklists that are extracted to /var/db/squidGuard.
    In the squidGuard conf file define categoris for ads and audiovideo to block.
    The final squidGuard.conf looks like :-


    dbhome /var/db/squidGuard
    logdir /var/spool/squid/log


    dest ads {
    domainlist blacklists/ads/domains
    urllist blacklists/ads/urls
    }


    dest audiovideo {
    domainlist blacklists/audiovideo/domains
    urllist blacklists/audiovideo/urls
    }


    acl {
    default {
    pass !ads !audiovideo all
    redirect http://192.168.1.1:667
    }
    }

  5. Since we need squid to start at boot time I modify /etc/rc.conf to include squid_enable=”YES”
  6. Squid operates transparently so that means we need to redirect traffic destined for port 80 to squid’s listening port 3128.
    To archive this we use the IPFIREWALL that comes with FreeBSD. To be able to redirect or forward traffic some options have to be enabled for the module. That means recompiling the kernel
    So change directory to /usr/src/sys/i386/conf/ make a copy of the GENERIC file using cp GENERIC MYGENERIC
    Edit MYGENERIC and append the lines below

    options IPFIREWALL
    options IPFIREWALL_VERBOSE
    options IPFIREWALL_DEFAULT_TO_ACCEPT
    options IPDIVERT
    options IPFIREWALL_FORWARD

    recompile the kernel using the command :-

    make buildkernel KERNCONF=MYGENERIC

    install the kernel using

    make installkernel KERNCONF=MYGENERIC

    If you need more help about compiling the kernel you can check out the handbook.

    To redirect traffic to squid first enable the IPFIREWALL from /etc/rc.conf and add the following options

    firewall_enable=”YES”
    firewall_type=”open”
    firewall_iif=”my0″

    Add a redirect rule in /etc/rc.firewall. I added it under the prototype setups Case open section. Or if its hard for you to find the section. Just add the line at the top.

    ${fwcmd} add 1000 fwd 192.168.1.1,3128 tcp from any to any 80 in via ${firewall_iif}

    The line above tells the firewall to redirect traffic destined for port 80 on the internet coming in from the internal interface to squid listening on port 3128.

  7. Configure DHCP Server
    I found problems with configuring the TP Link DHCP server. Since I can’t specify the FreeBSD box as the router. TP Link automatically assigns it self as the router when running as a dhcp server. So I decided to disable the TP Link dhcp server and enable it on the FreeBSD box. To install DHCP server go to /usr/ports/net/isc-dhcp30-server and type make install. After installation, edit the /usr/local/etc/dhcpd.conf directives as below:-

    option domain-name “freebsd.test”;
    option domain-name-servers 192.168.1.1;
    option routers 192.168.1.1;
    ddns-update-style none;

    subnet 192.168.1.0 netmask 255.255.255.0 {
    range 192.168.1.100 192.168.1.254;
    }

    To enable the dhcp server to start at boot time add dhcpd_enable=”YES” to /etc/rc.conf.

Finally to be able to monitor data transfers. I installed a simple network management tool called Darkstart. Its also include in the freebsd ports. So just go to /usr/ports/net-mgmt/darkstat/ and type make install to install it. To configure the darkstart and start it on boot time add the following to /etc/rc.conf

darkstat_interface=”my0″
darkstat_enable=”yes”

And thats all. Reboot. Now you can enjoy the orange 3G internet connection without worrying how much ur using always. The caching times are just crazy. U can see you usage by accessing the port 667 darkstart listens on e.g http://192.168.1.1:667

Below are some of the screenshots.

Advertisements
  1. nasser
    March 22, 2010 at 9:56 pm

    man this balistic.. am coming for free bsd .ya

  2. June 23, 2010 at 8:56 am

    You help me so much! Thank you for that!

  3. Joachim
    August 29, 2010 at 7:15 pm

    Impressive work.

  4. Joseph Ssenyange
    January 13, 2011 at 7:03 am

    Since 1st January 2011 I work from home whole day, and I have really been spending higher to pay for the data. I have updated the acls to block images too, to cut on data and seems am saving alot.

    acl imageflashcontent url_regex -i \.(gif|png|jpg|jpeg|ico|swf|iso|avi|wav|mp3|mp4|mpeg|swf|flv|x-flv)

    http_access deny imageflashcontent

    acl deny_rep_mime rep_mime_type -i video/.*
    acl deny_rep_mime rep_mime_type -i image/.*

    http_reply_access deny deny_rep_mime

  1. No trackbacks yet.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: