4/01/2014

Problems with PPTP and pfSense running as a virtual machine on VMware ESXi

Before you throw some stones saying why the hell are you using PPTP knowing it is legacy and easy to hack, think about Windows XP. It has been around for more than 10 years, it is full of malware around for it and there are applications that simply don't run any more on it, but several companies still have it. I was flying to Sydney last month from London and noticed that the airport in Sydney has several terminal running Windows XP. It is not just about what we want to tell our customers what is best or not, it is all down to costs. Some of my clients have firewalls that are a bit dated, some others want to connect to their network with their WIndows client without any need to 3rd party software like OpenVPN. Some I managed to get around using L2TP, but not always.

pfSense has a built in PPTP server which works fine when needed, also pass forward PPTP traffic just fine and it has worked like this for a long time, until I upgraded my clients to ESXi 5, then I started to have all sort of problems. I spent several months with pfSense support trying to get this sorted, not just about PPTP port forwarding or PPTP server itself on pfSense, but also dialing out PPTP connections behind pfSense.

Not a single machine behind pfSense that was running as a virtual machine on ESXi managed to dial out PPTP connections or be able to host PPTP servers, but this took me a whiloe to figure out as I had a mix of ESXi 4, 5 and 5.5 around. Even at my home server pfSense run virtualised, but inside VMware workstation running on top of a Windows server. It runs just fine inside VMware Workstation, so I assumed that it was not a problem with virtualisation. Going forward with pfSense support, we did many tests, captured loads of traffic and did everything that was available to troubleshoot the problem. In the end I was nearly giving up but then found some hints on forums regarding problems with the E1000 network drivers for freebsd and ESXi, plus some people saying the latest build of ESXi 5.5 sorted out some problems with PPTP connections.

I tested the VMXNET3 drivers for freebsd and for my surprise, indeed PPTP worked like a charm once again, but I didn't want to get into replacing the VMware tools manually across all the pfSense instances I have. It was better to update all hosts to the latest version of ESXi 5.5, build 1623387. I just went to https://www.vmware.com/patchmgr/findPatch.portal

The zip file you download can be uploaded to your ESXi host using WinSCP to the datastore, and then you can use putty to SSH into the ESXi host and run the update with this command:

esxcli software vib update -d /tmp/esxiupdate.zip (change to the location where you dumped the file)

You need to reboot the server afterwards, and hopefully pfSense or any other firewall that uses PPTP or even PPTP port forwarding will work fine. It worked for me.