Basic Information
Ipfixprobe supports various plugins, allowing you to customize the probe for your specific use case.
To use any plugin other than the (default), you must install Ipfixprobe from source. Refer to our installation guide for more details.
To enable a plugin:
- Install the necessary dependencies.
- Run cmake in the build folder with the -D flag and the plugin name. (e.g., for PCAP: cmake -DENABLE_INPUT_PCAP=ON ..)
- Compile and install Ipfixprobe.
➡️ For plugin-specific details, refer to the sections below.
List of Plugins
| INPUT | STORAGE | PROCESS (default) | OUTPUT |
|---|---|---|---|
| RAW_SOCKET (default) | CACHE (default) | HTTP | TXT (default) |
| BENCHMARK (default) | NETTISA | IPFIX (default) | |
| DPDK | DNS | UniRec (NEMEA) | |
| DPDK RING | PSTATS | ||
| PCAP | –others– | ||
| NDP (FPGA) |
- (default) - plugins installed by default
Plugin Details
This documentation is written primarily for RHEL distributions.
If you're having trouble installing dependencies, the package names might differ on different distributions. (e.g., libpcap-devel → libpcap-dev) or
check out CESNET COPR repository (you can add necessary copr repositories or build dependencies from source).
You may also need to enable certain repositories, for example:
- for RHEL - sudo yum-config-manager --add-repo https://yum.oracle.com/repo/OracleLinux/OL9/codeready/builder/x86_64/
INPUT
Input plugins define the source of incoming packets.
⚠️ You can choose only one type of input plugin (although more sources at one are possible)
Command line
Use -i to specify the input plugin.
Systemd service
To specify input in the configuration file, use input_plugin: followed by the plugin name (e.g., raw:, pcap_file:, pcap_live:, ndp:, dpdk_ring:, dpdk:) on a new line. After that, you can specify parameters for the plugin.
Example of configuration file
input_plugin:
raw:
interface: eth0
ℹ️ Run ip link show to see possible IFC values. (for example eth0)
Plugins description
Input plugin for reading packets from a raw interface.
- Works by capturing sockets directly from the INC by bypassing the TCP stack.
- A simple but not very efficient way to quickly check small networks.
- Does NOT require any additional libraries.
- Good for sniffing specific packets, not for general network monitoring.
This plugin is installed by default. Recommended for small networks with simple monitoring setup.
- i or ifc : Defines network interface name.
- b or blocks : Defines number of packet blocks.
- f or fanout : Enables packet fanout.
- p or pkts : Defines number of packets in block.
- l or list : Print list of available interfaces.
Read packets from the interface specified by the IFC value.
ipfixprobe -i 'raw;ifc=IFC;' -s 'cache'
Capture from IFC interface and scale packet processing using 2 instances of raw plugin, send flow to ifpfix collector using UDP.
ipfixprobe -i 'raw;ifc=IFC;f' -i 'raw;ifc=IFC;f' -o 'ipfix;u;host=collector.example.com;port=4739'
input_plugin:
raw:
interface: (name) # Network interface name to capture traffic from [required]
blocks_count: (number) # Number of blocks in the circular buffer (must be a power of 2)
packets_in_block: (number) # Number of packets per block (must be a power of 2)
Pcap (Packet Capture) provides an API for capturing network traffic. Ipfixprobe can use libpcap to quickly capture packets from an INC or a binary .pcap file format.
- Recommended option for monitoring small to medium networks (up to 1 Gbps).
- Exporting into .pcap file is NOT supported.
- If you encounter performance problems (many dropped packets ..), try limiting the number of process plugins or move to a faster input plugin (DPDK).
sudo dnf install ipfixprobe-input-pcap
dnf install libpcap libpcap-devel
cd build
cmake -DENABLE_INPUT_PCAP=ON ..
cd ..
make -j2
sudo make install
- f or file : Defines path to pcap file.
- i or ifc : Defines interface name.
- F or filter : Defines filter string.
- s or snaplen : Defines snapshot length in bytes (live capture only).
- l or list : Print list of available interfaces.
Process the pcap file and output to the terminal.
ipfixprobe -s cache -i "pcap;file=PATH_TO_PCAP_FILE" -o "text;m"
Read packets from the interface specified by the IFC value.
ipfixprobe -i 'pcap;i=IFC;' -s 'cache'
Capture from IFC interface using pcap plugin, split biflows into flows and prints them to console without mac addresses, telemetry data is exposed via the appFs library in /var/run/ipfixprobe directory.
ipfixprobe -i 'pcap;ifc=IFC' -s 'cache;split' -o 'text;m' -t /var/run/ipfixprobe
Read packets from a pcap file, enable 4 processing plugins, send L7 HTTP extended biflows to the unirec interface named http, and data from 3 other plugins to the stats interface.
ipfixprobe -i 'pcap;file=pcaps/http.pcap' -p http -p pstats -p idpcontent -p phists -o 'unirec;i=u:http:timeout=WAIT,u:stats:timeout=WAIT;p=http,(pstats,phists,idpcontent)'
input_plugin:
pcap_file:
file: (path) # Path to the PCAP file to read from [required]
bpf_filter: null # Optional BPF filter (e.g., "port 80"), null = no filter
# OR
pcap_live:
interface: (name) # Network interface for live packet capture [required]
bpf_filter: null # Optional BPF filter (null = no filter)
snap_length: (number) # Maximum packet capture length
DPDK (Data Plane Development Kit) is an ultrafast way to capture packets.
- Very efficient setup suitable for high-speed networks (up to 400 Gbps).
- Works by bypassing the kernel layer, offloading TCP packet processing to user space.
⚠️ Setting up DPDK is difficult and should not be used unless necessary. Try PCAP first and switch to DPDK later if PCAP is not enough.
ℹ️ This manual is simplified to get you up and running with DPDK. For detailed instruction see offitial DPDK manual
1️⃣ Install the required DPDK tools and libraries described here (section 2.2).
2️⃣ Install DPDK.
sudo dnf install dpdk dpdk-devel dpdk-tools
1️⃣ Download the latest stable version of DPDK from here.
2️⃣ Install the required DPDK tools and libraries described here (section 2.2).
3️⃣ Extract sources.
tar xJf dpdk-<version>.tar.xz
cd dpdk-<version>
4️⃣ Build libraries, drivers, and test applications.
meson setup <options> build
cd build
ninja
meson install
ldconfig
ℹ️ See mason setup options here.
⚠️ On some Linux distributions, such as Fedora or RedHat, paths in /usr/local are not in the default paths for the loader. Therefore, on these distributions, /usr/local/lib and /usr/local/lib64 should be added to a file in /etc/ld.so.conf.d/ before running ldconfig.
sudo dnf install ipfixprobe-dpdk
cd build
cmake -DENABLE_INPUT_DPDK=ON ..
cd ..
make -j
sudo make install
There are a few steps needed to set up DPDK correctly.
ℹ️ DPDK provides a few useful tools (dpdk-hugepages.py, dpdk-devbin.py, …) for configuration. Depending on your chosen installation, you can find them either on PATH or in build files. DPDK tools
Check if your kernel and glibc versions are up to date.
Kernel version >= 4.19- see:
uname -r
glibc >= 2.7 - see:
ldd --version
Hugepages allocation is needed to reduce TLB miss rate (that would occur when using standard 4k pages). This can greatly increase performance.
See /sys/kernel/mm/hugepages/ to see what sizes of huge pages are available.
See /sys/devices/system/nodes/ to see the number of nodes you have on the NUMA (Non-uniform memory access) system. (Only 1 node (node0) means NUMA is not used).
For a 64-bit system, it is recommended to use 1G hugepages if available. Some kernel versions may NOT allow reserving 1 GB hugepages at run time, so reserving them at boot time may be the only option.
There are two ways you can allocate hugepages.
Use this to allocate 1024 hugepages of size 2M at run time.
On NUMA, this command allocates hugepages equally across nodes.
echo 1024 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
You can also reserve hugepages for nodes (node0, node1, node2, …)
echo 1024 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages
Allocation at boot time helps prevent fragmentation of memory when allocating lots of continuous memory.
To reserve hugepages at boot time, a parameter is passed to the Linux kernel on the kernel command line.
- default_hugepagesz set default hugepage size for the system (important later)
- hugepagesz size of hugepages to use
- hugepages number of hugepages to allocate
default_hugepagesz=1G hugepagesz=1G hugepages=4
If secondary process is NOT required, you can use DPDK (ela) parameter --in-memory.
If secondary process is required, you need to use a mount point for hugepages at /dev/hugepages/. This mount point is using the system's DEFAULT hugepage size.
If you want to use hugepages other than DEFAULT, you need to create a mount point at /mnt/huge/.
mkdir /mnt/huge
mount -t hugetlbfs pagesize=1GB /mnt/huge
Or a permanent mount point by adding this line to /etc/fstab/
nodev /mnt/huge hugetlbfs pagesize=1GB 0 0
1️⃣ Figure out what NICs are available (use dpdk-tools).
dpdk-devbind.py --status
➡️ This should return a table of available NICs with their drives.
2️⃣ Find the NIC/s you want to use for DPDK and check if they support Bifurcated drives. (how???)
- If yes, skip to the next part..???
- If they do NOT, you need to unbind this NIC from the kernel.
ℹ️ If your NIC supports Bifurcated drives, it can operate both with the kernel and the DPDK application. It is important to NOT unbind such NIC from the kernel because DPDK expects it not to be.
⚠️ If you unbind your NIC from the kernel, it will no longer be visible from your system, and no other application can use it.
3️⃣ Bind your NIC to PMD (Pull Mode Driver)
There are two options to choose from if your NIC does not support Bifurcated drives
This option SHOULD be chosen in all cases if available. VIFO is robust and secure driver that relies on IOMMU protection.
3️⃣.1️⃣ Load vfio-pci module.
sudo modprobe vfio-pci
⚠️ Make sure that both kernel and BIOS support IO virtualization.
3️⃣.2️⃣ Figure out whether your system supports IOMMU (Input-Output Memory Management Unit)
In most cases, specifying iommu=on as a kernel parameter should be enough to configure the Linux kernel to use IOMMU.
You can use both VIFO with or without IOMMU mode. If your system does NOT support IOMMU, use:
echo 1 > /sys/module/vfio/parameters/enable_unsafe_noiommu_mode
⚠️ No IOMMU mode is inherently unsafe. Be careful with the usage.
3️⃣.3️⃣ Bind the NIC to vfio-pci (use dpdk-devbind.py --status to see NIC IDs)
dpdk-devbind.py --bind=vfio-pci 04:00.1
OR
dpdk-devbind.py --bind=vfio-pci eth1
Memory mappings are limited by the kernel. There are two main limitations.
This limit defines how much data can process lock (rlimit). You may want to increase it from the default limit if you need more memory for DPDK process.
ulimit -l <new_limit>
or temporarily with /etc/security/limits.conf.
This limit defines how many DMA map entries DPDK can make. You might want to change this limit if you are not using huge pages --no-huge.
echo 512000 > /sys/module/vfio_iommu_type1/parameters/dma_entry_limit
- b or bsize : Size of the MBUF packet buffer. (Default: 64).
- p or port : DPDK port to be used as an input interface.
- m or mem : Size of the memory pool for received packets. (Default: 16384).
- q or queue : Number of RX queues. (Default: 1).
- e or eal : DPDK parameters.
- M or mtu : Input interface MTU. (Default: 1518).
Read packets using DPDK input interface and 1 DPDK queue, enable plugins for basic statistics, http and tls, output to IPFIX on a local machine DPDK EAL parameters are passed in e, eal parameters DPDK plugin configuration has to be specified in the first input interface. The following dpdk interfaces are given without parameters; their configuration is inherited from the first one. Example for the queue of 3 DPDK input plugins (q=3):
ipfixprobe -i 'dpdk;p=0;q=3;e=-c 0x1 -a <[domain:]bus:devid.func>' -i dpdk -i dpdk -p http -p bstats -p tls -o 'ipfix;h=127.0.0.1'
Same example for the multiport read from ports 0 and 1, note comma-separated ports:
ipfixprobe -i 'dpdk;p=0,1;q=3;e=-c 0x1 -a <[domain:]bus:devid.func>' -i dpdk -i dpdk -p http -p bstats -p tls -o 'ipfix;h=127.0.0.1'
dpdk:
allowed_nics: "0000:00:00.0,0000:00:00.1" # List of allowed NICs (PCI addresses) [required]
burst_size: 64 # Number of packets processed in each burst (cycle)
mempool_size: 8192 # Size of the memory pool (must be a power of 2)
rx_queues: 1 # Number of RX (receive) queues
workers_cpu_list: [0] # List of CPU cores assigned to RX queues (must match rx_queues)
eal_opts: null # EAL options (null = default options)
mtu: null # Maximum Transmission Unit (defaults to RTE_ETHER_MAX_LEN)
This plugin is installed with the DPDK input plugin.
- b or bsize : Size of the MBUF packet buffer. (Default: 64).
- r or ring : Name of the ring to read packets from. Need to be specified explicitly, thus no default provided.
- e or eal : DPDK eal.
Read packets using DPDK input interface as a secondary process with shared memory (DPDK rings) - in this case, 4 DPDK rings are used.
ipfixprobe -i 'dpdk-ring;r=rx_ipfixprobe_0;e= --proc-type=secondary' -i 'dpdk-ring;r=rx_ipfixprobe_1' -i 'dpdk-ring;r=rx_ipfixprobe_2' -i 'dpdk-ring;r=rx_ipfixprobe_3' -o 'text'
dpdk_ring:
ring_name: ring0 # Name of the shared DPDK ring buffer [required]
eal_opts: null # EAL options (null = default options)
burst_size: 64 # Number of packets processed in each burst (cycle)
Input plugin for reading packets directly from an FPGA-based NIC. Unless you are using Netcope (CESNET FPGA cards) hardware, this plugin is NOT the right option.
sudo dnf install epel-release
sudo dnf copr enable @CESNET/nfb-framework
sudo dnf install nfb-framework numactl-devel
sudo dnf install ipfixprobe-input-ndp
You can also build nfb-framework from source from this repo.
sudo dnf install epel-release
sudo dnf copr enable @CESNET/nfb-framework
sudo dnf install nfb-framework numactl-devel
cd build
cmake -DENABLE_INPUT_NFB=ON ..
cd ..
make -j
sudo make install
- d or dev : Path to a device file.
- I or id : Link identifier number.
Capture from a COMBO card using the NDP plugin sends IPFIX data to 127.0.0.1:4739 using TCP by default.
ipfixprobe -i 'ndp;dev=/dev/nfb0:0' -i 'ndp;dev=/dev/nfb0:1' -i 'ndp;dev=/dev/nfb0:2' -o 'ipfix;h=127.0.0.1;p=4739'
input_plugin:
ndp:
device: PATH # Device for packet capture (e.g., /dev/nfb0) [required]
queues: "0-15" # List of queue IDs (e.g., "0,1,2-3,7") [required]
ℹ️ You can run 'ipfixprobe -h input' for installed input plugins
STORAGE
Storage plugins define how flows are internally stored. The CACHE plugin is activated by default.
Command line
Used -s to specify storage plugin.
Systemd service
To specify storage in the configuration file, use storage:
Plugin description
Currently, only available storage plugin.
- A hash table is used to keep flows.
- The hash table is divided into rows.
- Each row is managed as LRU (Least Recently Used).
This plugin is installed by default.
- s or size : Defines the count of flows that are kept in the cache at once. Cache size is 2^s.
- l or line : Defines length of the cache line. Line length is 2^l.
- a or active : Defines active timeout. When there is a flow that is active for more than -a seconds, it's exported.
- i or inactive : Defines inactive timeout. When there is a flow that is inactive for more than -i seconds, it's exported.
- S or split : Boolean flag. Defines if the bidirectional flow between two nodes is split into 2 separate unidirectional flows.
- fe or frag-enable : Used to enable completing fragmented packets into one packet.
- fs or frag-size : Size of fragmentation cache.
- ft or frag-timeout : The Timeout to consider fragments belonging to the same packets.
Store flows using cache split into two unidirectional flows.
ipfixprobe -s 'cache;split' -i 'pcap;file=PATH'
### Systemd parameters
storage:
cache:
size_exponent: (number) # Cache size as a power of 2 (e.g., 2^20)
line_size_exponent: (number) # Cache line size as a power of 2 (e.g., 2^4)
timeouts:
active: (number) # Active timeout duration (in seconds)
inactive: (number) # Inactive timeout duration (in seconds)
split_biflow: (bool) # Whether to split biflow into uniflow (true/false)
fragmentation_cache:
enabled: (bool) # Enable fragmentation cache (true/false)
size: (number) # Fragmentation cache size (minimum 1)
timeout: (number) # Timeout for fragments in the cache (in seconds)
ℹ️ You can run 'ipfixprobe -h storage' to see installed storage plugins.
PROCESS
Process plugins can specify what data to export.
By default, the Basic plugin is activated, giving you core information about network traffic like (i/o: mac address, IP, transmitted bits …, for a complete list see below).
⚠️ Every process plugin adds additional information on top of the Basic plugin. This comes with a big performance hit, so ALWAYS enable the minimum amount of plugins required.
Installation
Most process plugins are installed by default. Some experimental plugins (sip, rtsp, mpls, ntp, nettisa) must be installed separately.
Binary Packages
sudo dnf install ipfixprobe-nemea ipfixprobe-nemea-process-experimental
Build from source
cd build
cmake -DENABLE_PROCESS_EXPERIMENTAL=ON -DENABLE_NEMEA=ON ..
cd ..
make -j
sudo make install
Command line
Use -p to specify the process plugin.
Example usage
ipfixprobe -i raw;ifc=IFC' -s cache -p basicplus -p vlan -o text
Systemd service
Use process_plugins: to specify process plugins in a configuration file.
Example usage
process_plugins:
- http # HTTP protocol plugin
- dns # DNS protocol plugin
- quic # QUIC protocol plugin
- tls # TLS protocol plugin
# ...
Unirec and IPFIX fields
Each process plugin exports additional Unirec or IPFIX fields with flow data. The difference between Unirec and IPFIX fields comes down to a few points.
Unirec is our in-house developed standard, mainly used for communication between NEMEA modules.
IPFIX probe is standardized by the IETF.
- Fields with ID != 0 and IPFIX Enterprise number = 0 are standardized, and you can read more about them here if needed.
- Fields with ID = 0 and IPFIX Enterprise number (PEN) != 0 are custom-defined by the Ipfixprobe plugin.
Plugins description
This plugin is activated by default, any other process plugin adds on top of this one. These fields are also exported on interfaces where HTTP, DNS, SIP, and NTP plugins are active.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| DST_MAC | macaddr | 0 | 80 | destination MAC address |
| SRC_MAC | macaddr | 0 | 56 | source MAC address |
| DST_IP | ipaddr | 0 | 12 or 28 | destination IP address |
| SRC_IP | ipaddr | 0 | 8 or 27 | source IP address |
| BYTES | uint64 | 0 | 1 | number of bytes in data flow (src → dst) |
| BYTES_REV | uint64 | 29305 | 1 | number of bytes in data flow (dst → src) |
| LINK_BIT_FIELD or ODID | uint64 or uint32 | - | - | exporter identification |
| TIME_FIRST | time | 0 | 152 | first time stamp |
| TIME_LAST | time | 0 | 153 | last time stamp |
| PACKETS | uint32 | 0 | 2 | number of packets in data flow (src → dst) |
| PACKETS_REV | uint32 | 29305 | 0 | number of packets in data flow (dst → src) |
| DST_PORT | uint16 | 0 | 11 | transport layer destination port |
| SRC_PORT | uint16 | 0 | 7 | transport layer source port |
| DIR_BIT_FIELD | uint8 | 0 | 10 | bit field for determining outgoing incoming traffic |
| PROTOCOl | uint8 | 0 | 60 | transport protocol |
| TCP_FLAGS | uint8 | 0 | 6 | TCP protocol flags (src → dst) |
| TCP_FLAGS_REV | uint8 | 26305 | 6 | TCP protocol flags (dst→ src) |
List of fields exported together with BASIC on the interface.
- REV = source flow fields
- without REV = opposite direction flow
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| IP_TTL | uint8 | 0 | 192 | IP TLL field (destination) |
| IP_TTL_REV | uint8 | 29305 | 192 | IP TLL field (source) |
| IP_FLG | uint8 | 0 | 197 | IP FLAGS (destination) |
| IP_FLG_REV | uint8 | 29305 | 197 | IP FLAGS (source) |
| TCP_WIN | uint16 | 0 | 186 | TCP window size (destination) |
| TCP_WIN_REV | uint16 | 29305 | 186 | TCP window size (source) |
| TCP_OPT | uint64 | 0 | 209 | TCP options bitfield (destination) |
| TCP_OPT_REV | uint64 | 29305 | 209 | TCP options bitfield (source) |
| TCP_MSS | uint32 | 8057 | 900 | TCP maximum segment size (destination) |
| TCP_MSS_REV | uint32 | 8057 | 901 | TCP maximum segment size (source) |
| TCP_SYN_SIZE | uint16 | 8057 | 902 | TCP SYN packet size |
List of fields exported together with BASIC on the interface.
- BSTATS_MAXELENCOUNT (15 by default)
- Number of bursts in each direction exported by the plugin
- The bursts are computed separately for each direction.
- MINIMAL_PACKETS_IN_BURST (3 by default)
- When minimal packets are not reached, the fields are not exported to reduce output bandwidth.
- MAXIMAL_INTERPKT_TIME (1000 ms by default)
- Time between packets to be included in a burst.
Use for example: -DBSTATS_MAXELENCOUNT=30 while recompiling with cmake to change
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| SBI_BRST_PACKETS | uint32* | 0 | 291 | Number of packets transmitted in ith burst (src → dst) |
| DBI_BRST_PACKETS | uint32* | 0 | 291 | Number of packets transmitted in ith burst (dst → src) |
| SBI_BRST_BYTES | uint32* | 0 | 291 | Number of bytes transmitted in ith burst (src → dst) |
| DBI_BRST_BYTES | uint32* | 0 | 291 | Number of bytes transmitted in ith burst (dst → src) |
| SBI_BRST_TIME_START | time* | 0 | 291 | Start time of the ith burst (src → dst) |
| DBI_BRST_TIME_START | time* | 0 | 291 | Start time of the ith burst (dst → src) |
| SBI_BRST_TIME_STOP | time* | 0 | 291 | End time of the ith burst (src → dst) |
| DBI_BRST_TIME_STOP | time* | 0 | 291 | End time of the ith burst (dst → src) |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| DNS_ID | uint16 | 8057 | 10 | transaction ID |
| DNS_ANSWERS | uint16 | 8057 | 14 | number of DNS answer records |
| DNS_RCODE | uint8 | 8057 | 1 | response code field |
| DNS_NAME | string | 8057 | 2 | request domain name |
| DNS_QTYPE | uint16 | 8057 | 3 | request type field |
| DNS_CLASS | uint16 | 8057 | 4 | class field of DNS question |
| DNS_RR_TTL | uint32 | 8057 | 5 | resource record TTL field |
| DNS_RLENGTH | uint16 | 8057 | 6 | length of DNS_RDADT |
| DNS_RDATA | bytes | 8057 | 7 | resource record specific data |
| DNS_PSIZE | uint16 | 8057 | 8 | requestors payload size |
| DNS_DO | uint8 | 8057 | 9 | DNSSEC OK bit |
List of fields exported together with BASIC on the interface.
- t or txt : [FILE] Activates processing of all txt records. Allow to specify whitelist TXT records file (file line format: service.domain,txt_key1,txt_key2,…)
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| DNSSD_QUERIES | string | 8057 | 826 | list of queries for services |
| DNSSD_RESPONSES | string | 8057 | 827 | list of advertised services |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| FLOW_ID | uint64 | 0 | 148 | Hash of the flow - unique flow id |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| HTTP_DOMAIN | string | 39499 | 1 | HTTP request host |
| HTTP_URI | string | 39499 | 2 | HTTP request url |
| HTTP_USERAGENT | string | 39499 | 20 | HTTP request user agent |
| HTTP_REFERER | string | 39499 | 3 | HTTP request referer |
| HTTP_STATUS | uint16 | 39499 | 12 | HTTP response code |
| HTTP_CONTENT_TYPE | string | 39499 | 10 | HTTP content type |
| HTTP_METHOD | string | 39499 | 200 | HTTP request method |
| HTTP_SERVER | string | 39499 | 201 | HTTP response server |
| HTTP_SET_COOKIE_NAMES | string | 39499 | 202 | HTTP response all set-cookie names separated by a delimiter |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| L4_ICMP_TYPE_CODE | uint16 | 0 | 32 | ICMP type (MSB) and code (LSB) |
List of fields exported together with BASIC on the interface.
- IDPCONTENT_SIZE (100 by default)
- Number of bytes exported from the first data packet in the SRC → DST direction, and the first data packet in the DST → SRC direction.
Use -DIDPCONTENT_SIZE=number while recompiling with cmake to change.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| IDP_CONTENT | bytes | 8057 | 850 | Content of first data packet (src → dst) |
| IDP_CONTENT_REV | bytes | 8057 | 851 | Content of first data packet (dst → src) |
⚠️ This plugin is experimental and must be installed separately. For more details, check the installation information above.
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| MPLS_TOP_LABEL_STACK_SECTION | bytes | 0 | 70 | MPLS label section (without TTL), always 3 bytes |
List of fields exported together with BASIC on the interface.
- tc or topiccount : Export the first tc topics from the Publish packet header. Topics are separated by #. The default value is 0.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| MQTT_TYPE_CUMULATIVE | uint16 | 8057 | 1033 | types of packets and session present flag cumulative |
| MQTT_VERSION | uint8 | 8057 | 1034 | MQTT version |
| MQTT_CONNECTION_FLAGS | uint8 | 8057 | 1035 | last CONNECT packet flags |
| MQTT_KEEP_ALIVE | uint16 | 8057 | 1036 | last CONNECT keep alive |
| MQTT_CONNECTION_RETURN_CODE | uint8 | 8057 | 1037 | last CONNECT return code |
| MQTT_PUBLISH_FLAGS | uint8 | 8057 | 1038 | cumulative of PUBLISH packet flags |
| MQTT_TOPICS | string | 8057 | 1039 | topics from PUBLISH packets headers |
⚠️ This plugin is experimental and must be installed separately. For more details, check the installation information above.
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| NTP_LEAP | uint8 | 8057 | 18 | NTP leap field |
| NTP_VERSION | uint8 | 8057 | 19 | NTP message version |
| NTP_MODE | uint8 | 8057 | 20 | NTP mode field |
| NTP_STRATUM | uint8 | 8057 | 21 | NTP stratum field |
| NTP_POLL | uint8 | 8057 | 22 | NTP poll interval |
| NTP_PRECISION | uint8 | 8057 | 23 | NTP precision field |
| NTP_DELAY | uint32 | 8057 | 24 | NTP root delay |
| NTP_DISPERSION | uint32 | 8057 | 25 | NTP root dispersion |
| NTP_REF_ID | string | 8057 | 26 | NTP reference ID |
| NTP_REF | string | 8057 | 27 | NTP reference timestamp |
| NTP_ORIG | uint8 | 8057 | 28 | NTP origin timestamp |
| NTP_RECV | uint8 | 8057 | 29 | NTP receive timestamp |
| NTP_SENT | uint8 | 8057 | 30 | NTP transmit timestamp |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| NB_NAME | string | 8057 | 831 | NetBIOS Name Service name |
| NB_SUFFIX | uint8 | 8057 | 832 | NetBIOS Name Service suffix |
⚠️ This plugin is experimental and must be installed separately. For more details, check the installation information above.
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| NTS_MEAN | float | 8057 | 1020 | The mean of the payload lengths of packets |
| NTS_MIN | uint16 | 8057 | 1021 | Minimal value from all packet payload lengths |
| NTS_MAX | uint16 | 8057 | 1022 | Maximal value from all packet payload lengths |
| NTS_STDEV | float | 8057 | 1023 | The standard deviation is a measure of the variation of data from the mean |
| NTS_KURTOSIS | float | 8057 | 1024 | The kurtosis is the measure describing the extent to which the tails of a distribution differ from the tails of a normal distribution. |
| NTS_ROOT_MEAN_SQUARE | float | 8057 | 1025 | The measure of the magnitude of payload lengths of packets |
| NTS_AVERAGE_DISPERSION | float | 8057 | 1026 | The average absolute difference between each payload length of a packet and the mean value |
| NTS_MEAN_SCALED_TIME | float | 8057 | 1027 | The scaled times are defined as sequence s(t) = t_1 - t_1, t_2 - t_1, … , t_n - t-1. We compute the mean of the value with the same method as for feature Mean |
| NTS_MEAN_DIFFTIMES | float | 8057 | 1028 | The time differences are defined as sequence d_t = t_j : j = i + 1, i in 1, 2, … n - We compute the mean of the value with the same method as for feature Mean |
| NTS_MIN_DIFFTIMES | float | 8057 | 1029 | Minimal value from all time differences, i.e., min space between packets |
| NTS_MAX_DIFFTIMES | float | 8057 | 1030 | Maximal value from all time differences, i.e., min space between packets |
| NTS_TIME_DISTRIBUTION | float | 8057 | 1031 | Describes the distribution of time differences between individual packets |
| NTS_SWITCHING_RATIO | float | 8057 | 1032 | Represents a switching ratio between different values of the sequence of observation |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| PROGRAM_NAME | string | 8057 | 852 | The name of the program that handles the connection |
| USERNAME | string | 8057 | 853 | The name of the user who starts the process |
| OS_NAME | string | 8057 | 854 | Distribution or product name |
| OS_MAJOR | uint16 | 8057 | 855 | Major release version |
| OS_MINOR | uint16 | 8057 | 856 | Minor release version |
| OS_BUILD | string | 8057 | 857 | Optional build-specific or variant string |
| OS_PLATFORM | string | 8057 | 858 | OS Platform or ID |
| OS_PLATFORM_LIKE | string | 8057 | 859 | Closely related platforms |
| OS_ARCH | string | 8057 | 860 | OS Architecture |
| KERNEL_VERSION | string | 8057 | 861 | Kernel version |
| SYSTEM_HOSTNAME | string | 8057 | 862 | Network hostname including domain |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| OVPN_CONF_LEVEL | uint8 | 8057 | 828 | Level of confidence that the flow record is an OpenVPN tunnel |
List of fields exported together with BASIC on the interface.
- The plugin exports the histograms of Payload sizes and Inter-Packet-Times for each direction.
- The histograms bins are scaled logarithmically and are shown in the following table.
- i or includezeroes : Include zero payload packets.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| S_PHISTS_IPT | uint32* | 0 | 291 | Histogram of interpacket times (src → dst) |
| D_PHISTS_IPT | uint32* | 0 | 291 | Histogram of interpacket times (dst → src) |
| S_PHISTS_SIZE | uint32* | 0 | 291 | Histogram of packet sizes (src → dst) |
| D_PHISTS_SIZE | uint32* | 0 | 291 | Histogram of packet sizes (dst → src) |
List of fields exported together with BASIC on the interface.
- This plugin gathers statistics for the first PSTATS_MAXELEMCOUNT packets in the biflow record (30 by default).
- Use -DPSTATS_MAXELEMCOUNT=number while recompiling with cmake to change.
- i or includezerous : Include zero payloud packets.
- s or skipdup : Skip duplicate TCP packets.
The following fields are UniRec arrays (or basicList in IPFIX)
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| PPI_PKT_LENGTHS | uint16* | 0 | 291 | Sizes of the first packets |
| PPI_PKT_TIMES | time* | 0 | 291 | Timestamps of the first packets |
| PPI_PKT_DIRECTIONS | int8* | 0 | 291 | Directions of the first packets |
| PPI_PKT_FLAGS | uint8* | 0 | 291 | TCP flags for each packet |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| DNS_ID | uint16 | 8057 | 10 | transaction ID |
| DNS_ATYPE | uint8 | 8057 | 11 | response record type |
| DNS_NAME | string | 8057 | 2 | question domain name |
| DNS_RR_TTL | uint32 | 8057 | 5 | resorce record TTL field |
List of fields exported together with BASIC on the interface.
- -with-quic-ch-full-tls-ext
- enables the extraction of all TLS extensions in the Client Hello.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| QUIC_SIN | string | 8057 | 890 | Decrypted server name |
| QUIC_USER_AGENT | string | 8057 | 891 | Decrypted user agent |
| QUIC_VERSION | uint32 | 8057 | 892 | QUIC version from first server long header packets |
| QUIC_CLIENT_VERSION | uint32 | 8057 | 893 | QUIC version from first client long header packet |
| QUIC_TOKEN_LENGTH | uint64 | 8057 | 894 | Token length from Initial and Retry packets |
| QUIC_OCCID | bytes | 8057 | 895 | Source Connection ID from first client packet |
| QUIC_OSCID | bytes | 8057 | 896 | Destination Connection Id from first client packet |
| QUIC_SCID | bytes | 8057 | 897 | Source Connection ID from first server packet |
| QUIC_RETRY_SCID | bytes | 8057 | 898 | Source Connection ID from Retry packet |
| QUIC_MULTIPLEXED | uint8 | 8057 | 899 | > 0 if multiplexed (at least two different QUIC_OSCIDs or SNIs) |
| QUIC_ZERO_RTT | uint8 | 8057 | 889 | Number of 0-RTT packets in flow |
| QUIC_SERVER_PORT | uint16 | 8057 | 887 | TODO Server Port determined by packet type and TLS message |
| QUIC_PACKETS | uint8* | 0 | 291 | QUIC long header packet type (v1 encoded), version negotiation, QUIC bit |
| QUIC_CH_PARSED | uint8 | 8057 | 886 | > 0 if TLS Client Hello parsed without errors |
| QUIC_TLS_EXT_TYPE | uint16* | 0 | 291 | TLS extensions in the TLS Client Hello |
| QUIC_TLS_EXT_LEN | uint16* | 0 | 291 | Length of each TLS extension |
| QUIC_TLS_EXT | string | 8057 | 883 | Payload of all / application_layer_protocol_negotiation and quic_transport params TLS extension |
⚠️ This plugin is experimental and must be installed separately. For more details, check the installation information above.
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| RTSP_REQUEST_METHOD | string | 16982 | 600 | RTSP request method name |
| RTSP_REQUEST_AGENT | string | 16982 | 601 | RTSP request user agent |
| RTSP_REQUEST_URI | string | 16982 | 602 | RTSP request URI |
| RTSP_RESPONSE_STATUS_CODE | uint16 | 16982 | 603 | RTSP response status code |
| RTSP_RESPONSE_CONTENT_TYPE | string | 16982 | 604 | RTSP response content type |
| RTSP_RESPONSE_SERVER | string | 16982 | 605 | RTSP response server field |
⚠️ This plugin is experimental and must be installed separately. For more details, check the installation information above.
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| SIP_MSG_TYPE | uint16 | 8057 | 100 | SIP message code |
| SIP_STATUS_CODE | uint16 | 8057 | 101 | status of the SIP request |
| SIP_CSEQ | string | 8057 | 108 | CSeq field of SIP packet |
| SIP_CALLING_PARTY | string | 8057 | 103 | calling party (from) URI |
| SIP_CALLED_PARTY | string | 8057 | 104 | called party (to) URI |
| SIP_CALL_ID | string | 8057 | 102 | call ID |
| SIP_USER_AGENT | string | 8057 | 106 | user agent field of SIP packet |
| SIP_REQUEST_URI | string | 8057 | 107 | SIP request URI |
| SIP_VIA | string | 8057 | 105 | via field of SIP packet |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| SMTP_2XX_STAT_CODE_COUNT | uint32 | 8057 | 816 | number of 2XX status codes |
| SMTP_3XX_STAT_CODE_COUNT | uint32 | 8057 | 817 | number of 3XX status codes |
| SMTP_4XX_STAT_CODE_COUNT | uint32 | 8057 | 818 | number of 4XX status codes |
| SMTP_5XX_STAT_CODE_COUNT | uint32 | 8057 | 819 | number of 5XX status codes |
| SMTP_COMMAND_FLAGS | uint32 | 8057 | 810 | bit array of commands present |
| SMTP_MAIL_CMD_COUNT | uint32 | 8057 | 811 | number of MAIL commands |
| SMTP_RCPT_CMD_COUNT | uint32 | 8057 | 812 | number of RCPT commands |
| SMTP_STAT_CODE_FLAGS | uint32 | 8057 | 815 | bit array of status codes present |
| SMTP_DOMAIN | string | 8057 | 820 | domain name of the SMTP client |
| SMTP_FIRST_SENDER | string | 8057 | 813 | first sender in MAIL command |
| SMTP_FIRST_RECIPIENT | string | 8057 | 814 | first recipient in RCPT command |
List of fields exported together with BASIC on the interface. The detector searches for the SYN SYN-ACK ACK pattern in packet lengths. Multiple occurrences of this pattern suggest a tunneled connection.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| SSA_CONF_LEVEL | uint8 | 8057 | 903 | 1 if SSA sequence detected, 0 otherwise |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| SSDP_LOCATION_PORT | uint16 | 8057 | 821 | service port |
| SSDP_NT | string | 8057 | 824 | list of advertised service urns |
| SSDP_SERVER | string | 8057 | 822 | server info |
| SSDP_ST | string | 8057 | 825 | list of queried service urns |
| SSDP_USER_AGENT | string | 8057 | 823 | list of user agents |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| TLS_SNI | string | 8057 | 808 | TLS server name indication field from client |
| TLS_ALPN | string | 39499 | 337 | TLS application protocol layer negotiation field from server |
| TLS_VERSION | uint16 | 39499 | 333 | TLS client protocol version |
| TLS_JA3 | string | 39499 | 357 | TLS client JA3 fingerprint |
| TLS_EXT_TYPE | uint16 | 0 | 291 | TLS extensions in the TLS Client Hello |
| TLS_EXT_LEN | uint16 | 0 | 291 | Length of each TLS extension |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| VLAN_ID | uint16 | 0 | 58 | Vlan ID (used in flow key) |
List of fields exported together with BASIC on the interface.
| Output field | Type | IPFIX Enterprise number | ID | Description |
|---|---|---|---|---|
| WG_CONF_LEVEL | uint8 | 8057 | 1100 | Level of confidence that the flow record is a WireGuard tunnel |
| WG_SRC_PEER | uint32 | 8057 | 1101 | Ephemeral SRC peer identifier |
| WG_DST_PEER | uint32 | 8057 | 1102 | Ephemeral DST peer identifier |
ℹ️ You can run 'ipfixprobe -h process' to see installed process plugins.
OUTPUT
Output plugins define how flows are exported.
⚠️ You can choose only one type of output plugin
Command line
Use -o to specify the output plugin.
Systemd service
To specify the output plugin in the configuration file, use output_plugin: followed by the plugin name (e.g., ipfix:, unirec:, text:) on a new line. After that, you can specify parameters for the plugin.
Example of configuration file
input_plugin:
ipfix:
collector:
host: localhost
port: 4739
Plugins description
Use the text plugin to output directly to the terminal or text file.
This plugin is installed by default.
- f or file : Defines path to save file to write output in instead of stdout.
- m or mac : Boolean flag. MAC addresses are hidden if set.
Print exported flows to the terminal without MAC addresses
ipfixprobe -o 'text;mac' -i 'pcap;file=...;' -s 'cache'
Print exported flows to the FILE
ipfixprobe -o 'text;f=FILE' -i 'pcap;file=...;' -s 'cache'
output_plugin:
text:
file: (path) # Path to output file (use null for stdout)
IP Flow Information Export is a standard protocol for exporting network flow information from probes running on routers, switches, firewalls, directly to collectors on the local network. We recommend using our collector ipfixcol2, but you can use any collector with support for IPFIX input.
This plugin is installed by default.
- h or host : Defines IP address of the remote collector.
- p or port : Defines the collector port to send data to.
- m or mtu : Defines the maximum size of IPFIX packet payload sent.
- u or udp : Boolean flag. UDP is used if set.
- n or non-blocking-tcp : Boolean flag. Non-blocking TCP socket is used if set.
- I or id : Defines exporter ID.
- t or template : Defines template refresh rate in seconds.
- d or dir : (NUM) Dir bit field value.
- v or verbose : Enable verbose mode.
- c or lz4-compression : Enable lz4 compression.
- s or lz4-buffer-size : Lz4 compression buffer size (default (minimum): mtu*3).
Send exported data to the localhost using UDP as an exporter with ID 3.
ipfixprobe -o 'ipfix;h=127.0.0.1;u;I=3'-i 'pcap;file=...;' -s 'cache'
Send exported data to the localhost:4739 using non-blocking TCP as an exporter 3 with a maximal transfer unit set to 2000.
ipfixprobe -o 'ipfix;h=127.0.0.1;p=4739;n;mtu=2000' -i 'pcap;file=...;' -s 'cache'
output_plugin:
ipfix:
collector:
host: localhost # IPFIX collector address [required]
port: (number) # IPFIX collector port [required]
mtu: (number) # Maximum size of IPFIX packets
exporter:
id: (number) # Exporter ID for IPFIX [required]
dir: (number) # Direction bit field value (0 = incoming, 1 = outgoing)
protocol:
# IMPORTANT: Only one protocol can be selected: udp or tcp.
udp:
template_refresh: (number) # Template refresh rate for UDP protocol (in seconds)
tcp:
non_blocking: (bool) # Whether to use non-blocking TCP sockets (true/false)
compression:
lz4:
enabled: (bool) # Enable LZ4 compression for IPFIX (true/false)
buffer_size: (number) # Buffer size for LZ4 compression (default: mtu * 3)
This plugin exports data in our UNIREC format. It is mainly used to send data directly to NEMEA modules. It is defined by a fixed list of key-value pairs. To see a list of UNIREC fields, check out the specific process plugins you are currently using.
sudo dnf install ipfixprobe-nemea ipfixprobe-nemea-output-unirec
dnf install libtrap libtrap-devel unirec-devel unirec
cd build
cmake -DENABLE_NEMEA=ON -DENABLE_OUTPUT_UNIREC=ON ..
cd ..
make -j
sudo make install
- i or ifc : Defines unirec interface to use. guide here
- p or plugins : Defines plugin-interface mapping. Plugins can be grouped like '(p1,p2,p3),p4,(p5,p6).
- o or odid : Boolean flag. If set, exports the ODID field.
- e or eof : Boolean flag. If set, sends EOF message on exit.
- I or id : Defines exporter id.
- h or help : Prints libtrap help.
- d or dir : (NUM) Dir bit field value.
- v or verbose : Increase verbosity.
Send exported data to the Unix socket ipfixprobe.
ipfixprobe -o 'unirec;i=u:ipfixprobe' -i 'pcap;file=...' -s 'cache'
Same as previous, but should be used with small pcap files to avoid data not being sent.
ipfixprobe -o 'unirec;i=u:ipfixprobe:timeout=WAIT:buffer=off' -i 'pcap;file=...' -s 'cache'
Save the exported data to the data.trapcap.
ipfixprobe -o 'unirec;i=f:data.trapcap' -i 'pcap;file=...' -s 'cache'
Read packets from pcap file, enable 4 processing plugins, sends L7 HTTP extended biflows to unirec interface named http and data from 3 other plugins to the stats interface
ipfixprobe -i 'pcap;file=pcaps/http.pcap' -p http -p pstats -p idpcontent -p phists -o 'unirec;i=u:http:timeout=WAIT,u:stats:timeout=WAIT;p=http,(pstats,phists,idpcontent)'
Unirec is not yet available for use as a systemd.
ℹ️ You can run 'ipfixprobe -h output' to see installed output plugins.