Setup NtopNG on PfSense the Correct Way
Background
Ntopng is a great tool for diagnosing and monitoring your network. It is available on the pfSense firewall through the built-in package manager. Unfortunately, the pfSense port of the ntopng package, which is installed through the GUI package manager, has been broken for a long time.
In the latest pfSense 2.5 release, they updated ntopng to 4.2, which is great, but it contains a lot of bugs. Sometimes ntopng keeps restarting on itself, other times it seems very slow, and I personally faced an issue where whenever an ntopng service restart occurs, all the package’s config gets wiped out — so any modifications you make, like renaming an interface or adding alert endpoints and recipients, get lost on the next service restart. Also, in the previous version of pfSense, which had ntopng 3.8, geolocation data was not being reported correctly. This all makes it useless to put into production environments.
Maybe the most optimal way to set up ntopng is to separate it from the firewall and use a dedicated box to record and analyze network traffic by using a port mirror. However, sometimes you are in a circumstance where it is not feasible to have a separate machine, or maybe the firewall box that you are using is powerful enough to add an active network monitoring function to it.
Tested On:
pfSense 2.5, ntopng 4.2
TL;DR
The better way to integrate ntopng with pfSense is by installing the ntopng package directly from the command line. These are the needed steps in short:
1- On pfSense 2.5, install ntopng and the Redis database using the shell
pkg install ntopng redis2- Enable ntopng and Redis services
sysrc redis_enable="YES"
sysrc ntopng_enable="YES"3- Install the Shellcmd package from the pfSense GUI package manager
4- Add the below entries to Shellcmd settings and save
service redis start
service ntopng start5- Configure ntopng geolocation by downloading the relevant .mmdb files from MaxMind and placing them in the folder “/usr/local/share/ntopng/httpdocs/geoip”
6- Edit the startup script of ntopng to add your custom command arguments located in “/usr/local/etc/rc.d/ntopng”
7- Start Redis and ntopng services, or simply reboot pfSense
service start redis
service start ntopngDetails
First of all, you need to decide whether you are comfortable using the official but unstable ntopng development build from the original author’s package repository (packages.ntop.org), or otherwise the official stable FreeBSD port maintained by the FreeBSD developer madpilot (pkg.freebsd.org). However, the one in the FreeBSD repo sometimes falls too far behind in development. Unfortunately, ntop.org does not provide a stable build, only a development snapshot (as of the time of this writing).
If you decide to choose the stable build from pkg.freebsd.org, you’re in luck — pfSense uses that repository by default, so you do not need to add any additional sources. All you need to do is install ntopng and Redis. To do that, connect to your pfSense using SSH or use the console and open the shell prompt.
pkg install ntopng redisIf you decide to install the latest but unstable build, then follow the instructions at packages.ntop.org/FreeBSD, but use FreeBSD 12 if you are on pfSense 2.5 or later.
After installing ntopng and Redis, you need to make them run automatically on boot. The way we do this is by using a package called Shellcmd, available in the pfSense package manager. Simply head over to the System menu, then Package Manager, and install Shellcmd. Once installed, go to the Services menu, then Shellcmd, and add entries to start ntopng and Redis on system boot.
Geolocation
Next, you need to set up geolocation databases if you want ntopng to report geolocation data. Refer to this GitHub README on how and why you need to do this: https://github.com/ntop/ntopng/blob/dev/doc/README.geolocation.md.
Unfortunately, there is no pre-built geoipupdate package for FreeBSD, so you will have to download and update the databases manually, or you could look up some scripts online that automate the process of downloading the databases from MaxMind for you. “/usr/local/share/ntopng/httpdocs/geoip” is where to put the downloaded database files.
Startup-Command Options
Now comes the part where you modify the startup script for ntopng to put whatever argument option you need. Edit the file located on “/usr/local/etc/rc.d/ntopng” using any text editor and go to the line that starts with “command_args=” and add the arguments that you need. You can refer to the official ntop docs here: (https://www.ntop.org/guides/ntopng/cli_options.html).
Ntopng is not designed to be used as an aggregator for network traffic data over long periods of time — instead, it is best used for live traffic monitoring. That is why, if you want to keep network traffic for data retention, it is best to send flows to other databases that are more efficient for storing data over longer periods.
For example, you could instruct ntopng to dump flows to an elasticsearch database using the argument:
--dump-flows 'es;flows;index_name%Y.%m.%d;http://ip_of_es:9200/_bulk;'Another thing to note: if you want to enable HTTPS for the ntopng access portal, after adding the command option –https-port “[port_number]”, ntopng tries to find an SSL certificate but doesn’t find one. The easiest way to correct this is to find and rename the provided dummy certificate to the .pem file format. The dummy cert is located at “/usr/local/share/ntopng/httpdocs/ssl/ntopng-cert.pem.dummy”.
Finally, once everything is set up correctly, you can start ntopng with
service redis start
service ntopng startor simply rebooting pfSense should bring those services up.
Now you can head over to http(s)://ip-of-pfsense:3000 or whatever port you defined in the startup options.
