12/06/2013

How to use pfSense with BT Infinity FTTC and other ISPs PPPOE and static range of IP addresses

As more and more areas in the UK are being covered on a daily by FTTC (fiber-to-the-cabinet) providing very good speeds up to 78Mbit doenstream and 20Mbit Upstream with a very attractive price, I'm seeing a challenge with the current set up BT and all others ISPs are having. As you need the OpenReach white VDSL modem and then another router to actually do the routing between your LAN (you can use a VDSL router but again, I require a proper firewall), you always end up with an annoying set up of having 2 boxes, sometimes even 3 if you use a separated firewall appliance.

BT Openreach FTTC modem

I always favour practicality and tidiness, so the least devices I can use the better to maintain, manage and of course, less clutter of cables.

Messy Cables in Office

One good solution for this is to use pfSense firewall to authenticate the PPPOE connection, route and be the firewall. Also all of this can be done with pfSense installed as a virtual appliance. I have for example at home pfSense running on VMware workstation as a service so the VM start and shutdown with the host. I have a media centre on 24/7 so for me it is perfect. Of course you can install pfSense on a old PC, nettop or even on some thinclient like the NEOware E14 (limited firewall features though)

My problem was that with the PPPOE set up, the ISP assigns automatically the IP address to the firewall and in my case as we have a block of IP addresses /29 I wanted to be able to use all of them. Normally you would do this simply by going to Firewall>Virtual IPs>Add IP Alias. This works fine when you use static IP address on your WAN like when you have a ADSL router and it has one of the public IP addresses on it, and forward everything else ahead so you can simply set up the Virtual IPs on pfSense and voila, all done. For FTTC as we use PPPOE this changes a alot as the WAN interface gets the IP address from the ISP automatically, and if you set the IP address as Virtual IP, nothing happens.

To troubleshoot this I had to check what was happening, are the packets arriving on the WAN interface? Are they being dropped? Are the IPs actually showing up on the WAN interface? I contacted my ISP (zen.co.uk) just to confirm the block of IPs /29 is available, they confirmed and then I had to run some packet analysis on my side to understand why it was still not working.

Running the command ifconfig -a will show all the IP addresses assigned to all interfaces and you will notice that when authanticatin using PPPOE the other virtual IP addresses allocated to the WAN interface simply don't show up. 

After this I decided to check if traffic to the other IP addresses were actually being routed to my firewall by my ISP on the /29 network. I used the Packet capture feature on the same diagnostics tab from pfSense and enabled only ICMP capture, then from outside the network I run a ping against every single IP address I have on the range and by my surprise, I got them on pfSense as you can see: (IPs hidden for security purposes)

14:55:21.737881 IP 92.x.x.x > 88..x.x.x: ICMP echo request, id 35801, seq 28, length 40
14:55:21.737948 IP 92.x.x.x > 88..x.x.x: ICMP echo request, id 35801, seq 28, length 40
14:55:26.545222 IP 92.x.x.x > 88..x.x.x: ICMP echo request, id 34275, seq 29, length 40

So pings are received but simply dropped. The firewall doesn't have the configuration to route this information anywhere as we have seen with ifconfig-a, no IP address besides the one assigned through PPPOE is assigned to any interface.

I tried assigning the Virtual IPs then instead of WAN, to the localhost interface.

For my surprise, after testing this all IP addresses were working just fine and I manage to ping them from outside the network, establish IPSEC tunnels as if they were being assigned to the WAN interface, exactly like that.

It is weird, but it works and everything else works like it should, all IPs added as Virtual IPs.

Tricky to find out why, but at least I got it working. I hope this help for all FTTC ISPs in the UK using PPPOE authentication with several static IPs allocated for you :)


8 comments:

  1. This comment has been removed by the author.

    ReplyDelete
  2. Alex. If at all possible could you give an example configuration for your pfsense box and change the external IP addresses for security for yourself. I am currently using an hgv2700 for the gateway routing / NAT ip addressing. It would be great to use the fttc box direct with pfsense. Any further help would be great. I have a block of 5 but getting them to work with pfsense is trick as BT have a way of doing things arse about face.

    ReplyDelete
  3. Have you got a PPPOE username and password from BT? Is BT actually your ISP? If you have these details it won't be difficult to set up, but unfortunately I can't send my config file. I can give you a hand with the set up of course. Let me know exactly what is the problem so far so I can help.

    ReplyDelete
    Replies
    1. Thanks for the reply Alex. Yes I have a PPPOE username and password. The wan port gets a dynamic address and connects successfully using the fttc box. I stumble though at this point. The lan traffic appears as the dynamic address. I would however wish to have the lan traffic appear as the first ip address in my BT ip block. The office lan traffic needs to always appear as a fixed address.

      Delete
    2. The way it works with PPPOE it always gets the last IP of the block for the device. You can always add the other IPs as aliases and they work just fine. This seems to be the ISP and not the firewall doing, perhaps the ISP can change their IP assignment rules to fit your needs, but them they need to do this as they assign the address for you. Remember it is not a dynamic address, it is just assigned automatically, but fixed. If you get dynamic IP address, then you need to request your ISP for a static IP address for block of addresses.

      Delete
  4. Hey, I only got a "5C" Master socket installed from phone line so I don't have a BT modem just a Home hub. Do I need to buy a HG612 and install that then run the rJ45 from it to my pfsense?

    ReplyDelete
  5. you need a VDSL modem, be the HG612 or a Draytek 130, it needs to be a Modem or a router configured in bridge mode and then connect it to pfSense

    ReplyDelete
  6. Just over 5 years later and this article just saved the day for me. Thanks very much for sharing this info. It got me going in the right direction and finally online.

    My steps were slightly different, but just adding them here in case anyone else finds it useful.

    - Create your PPPoE interface and let it bind to the WAN as normal (setup wizard sorts this)
    - Interfaces > assignments > Create OPT1 interface and bind it to the WAN network card
    - Edit it and give it a static network IP address (normally first usable in the range)
    - Remember to enable it
    - Leave everything else as default
    - Firewall > Virtual IPs > add your IP alias for each IP and bind it to the OPT1 interface.

    Thanks again.

    ReplyDelete