# How to use

## Startup

There are two ways you can start up ipfixprobe. 

<accordion>
    <panel>
        <label><heading h4>As a systems service (recommended)</heading></label>
         
      
    
       ## Configuring the service 
       Ipfixprobe can be set up to be used as a **daemon to continuously process** incoming packets.
       
       All you need to do is **create NAME.conf** (e.g. ''instance.conf'') file in ''/etc/ipfixprobe/''
      
     <note>ℹ️ **To see how to write your ''.conf'', check out our [[https://github.com/CESNET/ipfixprobe/blob/master/init/link0.conf.example|example file]] or for plugins specific option see [[en:plugins|plugins guide]]**</note>
     
       ## Starting the service 
       <note warning>⚠️ Exchange NAME for name of your ''.conf'' file </note>
       
       #### Start manually
       
       <code bash>
       sudo systemctl start ipfixprobe@NAME.service
       </code>
       
       <code bash>
       sudo systemctl stop ipfixprobe@NAME.service
       </code>
       

----

       #### Enable at system startup 
      
       <code bash>
       sudo systemctl enable ipfixprobe@NAME.service
       </code>
  
       <code bash>
       sudo systemctl disable ipfixprobe@NAME.service
       </code>
       
       
       

----
       
       <note warning>⚠️ If you are having **trouble setting up ipfixprobe** as systemd service **make sure that**:
  *''.conf'' file is in ''/etc/ipfixprobe/''
  *You are using the right ''.conf'' file
  *You are launching the service with **administrative privileges** 
  *Try running ''systemctl status ipfixprobe@NAME.service''
  *Check for errors with ''sudo journalctl -xeu ipfixprobe@NAME.service''
  *Make sure ipfixprobe is installed in ''/usr/bin/ipfixprobe/''
  *Check out ''ipfixprobed'' in the ''init'' directory and make sure the ''config2args.py'' and ''scheme.json'' are installed in ''IPFIXPROBE_LIB_DIR'' (recompile after change)
        </note>
        
       #### Check running processes 
       
       Once ipfixprobe is running, you can check out the process by 
       <code bash>
       pgrep -af ipfixprobe
       </code>
       Or kill the process by 
       <code bash> 
       kill PROCESS_NUMBER
       </code>
       
    
   You can have **more than one ipfixprobe** instance running at the same time 
   <note warning>⚠️ Some **command line parameters** for ipfixprobe and its plugins are **NOT supported** as systemd parameters. (most are) See example file above for all available options, or run **ipfixprobe** as a daemon from the **command line** (without the need for a ''.conf'' file) by adding ''-d'' flag. </note>
    
    

    </panel>

    <panel>
        <label><heading h4>Launching manually from the  command line</heading></label>
        Ipfixprobe supports manual launching from a terminal by specifying parameters.
        <note warning> ⚠️ **Remember:** Ipfixprobe MUST be run with: **1**  ''input'' - **1**  ''storage'' - **0-N**  ''process'' - **0-1**  ''output'' plugins at the same time</note>
        
        ##  Arguments 
  ***-i** or **--input** set input plugin (-h input for help)
  ***-s** or **--storage** (default:cache) set storage plugin (-h storage for help)
  ***-p** or **--process** set process plugin/s (-h process for help)
  ***-o** or **--output** (default:ipfix) set output plugin(-h output for help)


       
        ### Example usage
        
        Lunch **ipfixprobe** from the terminal with ''raw socket'', ''cache'', and ''text'' output.
        <code bash>
        ipfixprobe -i 'raw;i=eth0' -s cache -o text
        </code>
        
        <note>ℹ️ For more information about plugins check out our [[en:plugins|plugin guide]].</note>
        
        ### Other arguments
        
  ***-t** or **--telemetry** (default:none) Path where to mount the telemetry directory to [[https://appfs.rkeene.org/web/index|AppFs (Application File system)]]. This exposes telemetry data to userspace and other applications.
  ***-q** or **--iqueue** (default:64) The size of the queue (in records) between ''input'' and ''storage''
  ***-Q** or **--oqueue** (default:16536) The size of the queue (in records) between ''storage'' and ''output''
  ***-f** or **--fps** (default:unlimited) Limit the number of exported flows to N per second or 0 for unlimited
  ***-c** or **--count** (default:unlimited) Quit after N number of packets processed on each interface
  ***-P** or **--pid**(default:false) Create .pid file (text file with process number)
  ***-L** or **--plugins-path**(default:''CMAKE_INSTALL_FULL_LIBDIR/ipfixprobe/'' e.g. ''/usr/local/lib64/ipfixprobe'') Define the new path to the plugins directory
  ***-d** or **--daemon**(default:false) Run as a standalone process
  ***-V** or **--version** Show ipfixprobe version
  ***-C** or **--cpus** Specifies what CPU cores should be used by IPFIXPROBE. Cores are specified as numbers (e.g. 0,2,3)
        
    </panel>
 
</accordion>



## Data storage

Get data from **packet files** or **probes output** stored in different formats for **later usage**.
<accordion>
    <panel>
      <label><heading h4>PCAP file (or flow data) to CSV</heading></label>
      You can use **ipfixprobe** with a combination of other tools from **NEMEA** to convert ''.pcap'' files to ''.csv'' (Comma Separated Values) format.
      To do this, you can choose from **two options**. Install **ipfixprobe and dependencies locally** or use a **Docker image**.
      
      <note> ℹ️ You can **easily adjust** this procedure to convert any **Ipfixprobe input** to **CVS files.** </note>\\
      
      
### 1️⃣ **Using ipfixprobe manually from the command line**

      
      <accordion>
      <panel>
      <label><heading h4>Install as binery package</heading></label>
      1️⃣**Install ipfixprobe (Check [[en:get_started|installation guide]])**
      
      2️⃣**Insatll ipfixprobe plugins** 
      
      <code bash> 
          sudo dnf copr -y enable @CESNET/ipfixprobe 
          sudo dnf install ipfixprobe-nemea ipfixprobe-input-pcap ipfixprobe-nemea-process-experimental ipfixprobe-nemea-output-unirec
      </code>
      
      3️⃣**Install Nemea** 
      <code bash> 
         sudo dnf copr -y enable @CESNET/NEMEA 
         sudo dnf install nemea 
      </code>
      
      4️⃣**Install Extra Packages for Enterprise Linux**
      <code bash> 
          dnf install -y epel-release
      </code>
      </panel>
      
      <panel>
       <label><heading h4>Build from source </heading></label>
       
      1️⃣**Install ipfixprobe (Check [[en:get_started|installation guide]])**
      
      2️⃣**Insatll ipfixprobe plugins**
      <code bash> 
      cd build
      cmake -DENABLE_INPUT_PCAP=ON -DENABLE_OUTPUT_UNIREC=ON -DENABLE_PROCESS_EXPERIMENTAL=ON -DENABLE_NEMEA=ON ..
      cd ..
      make -j
      make install
      </code>
      
      3️⃣**Install [[https://github.com/CESNET/Nemea|Nemea]]**
      
      4️⃣**Install Extra Packages for Enterprise Linux**
       <code bash> 
          dnf install -y epel-release
      </code>
      </panel>
      </accordion>
     \\

      
      <accordion>
      
      <panel>
          <label><heading h4>Lunch from command line</heading></label>
           1️⃣ **Create a ''.trapcap'' file with ipfixprobe out of your ''.pcap'' file.** (replace NAME with path to pcap file)
      <code bash>
      ipfixprobe -i "pcap;file=NAME.pcap" -p "pstats" -p "nettisa" -o "unirec;i=f:tmpTrap.trapcap:timeout=WAIT;p=(pstats,nettisa)"
      </code>
      
      2️⃣ **Conver ''.trapcap'' file to ''.csv'' file with **NEMEA module** ''logger''.** (replace NAME with path to csv file)
      <code bash>
      /usr/bin/nemea/logger -t -i "f:tmpTrap.trapcap"  -w "NAME.csv"
      </code>
      
      <note warning>There could be **more tools** named ''logger'' on your system. Use **NEMEA module**. </note>

      3️⃣ **Remove the temporary ''.trapcap'' file.**
      <code bash>
      rm tmpTrap.trapcap
      </code>

      </panel>
      
      </accordion>

----
     \\
      ### 2️⃣ **Using Docker image** 

      <accordion> 
      
      <panel>
      <label><heading h4>Docker image</heading></label>
      Use the provided Docker image to convert files from ''.pcap'' to ''.csv''.
      <note warning> Ipfixprobe **must be installed from source** to use Docker image </note>
      
      ### Requirements 
  ***Docker** or **Podman**
  ***bash**
  ***which**
  ***mktemp**

      ### Usage
      
      Use the **provided script** in the build files (in the **docker folder**) for **ipfixprobe** to convert PCAP to CSV with a Docker image. 
      <note> Provided script **builds Docker image automatically**. Make sure to be in the **same directory** as ''Dockerfile''. </note>
      



      <code bash>
      bash ./ipfixprobe_wrapper.sh <process_script.sh> <input_file.pcap> <output_file.csv>
      </code>
      
  ***process_script.sh** : Script for processing the pcap file inside the container. (Script available in docker folder)
  ***input_file.pcap** : Path to the input pcap file.
  ***output_file.csv** : Path to the output CSV file.
      
      ➡️ Example usage 
      
      <code bash>
      bash ./ipfixprobe_wrapper.sh ./process_script.sh ../pcaps/mixed.pcap ./output.csv
      </code>
      
      #### Build a Docker image by hand.
      
      <code bash>
      docker build -t docker_ipfixprobe .
      </code>
      </panel>

      
      </accordion>
 
      
    </panel>
       
      <panel>
      <label><heading h4>Ipfixprobe flow output to FSD (efficient long-term storage), JSON, IPFIX file</heading></label>

This is possible with **Ipfixcol2** collector. 

1️⃣ Install Ipfixprobe, see [[get_started|here]].

2️⃣ Enable necessary ipfixprobe output plugins (IPFIX), see the [[plugins|plugins guide]].

3️⃣ Install Ipfixcol2 from [[https://github.com/CESNET/ipfixcol2|here]]

4️⃣ Create Ipfixcol2 config file. For input, use ''UDP'' or ''TCP'' plugin. For output use ''JSON'', ''IPFIX File'', or ''FSD File''. See Ipfixcol2 documentation for information on how to do this. 

5️⃣ Lunch Ipfixprobe (lunch with **u** ipfix plugin parameter for ''UDP'').

<code bash>
ipfixprobe -i "pcap;i=eth0" -o "ipfix;host=127.0.0.1;port=4739"
</code>

6️⃣ Lunch collector (start after probe).

<code bash>
ipfixcol2 -c ipfixcol2-confign.xml
</code>

<note>ℹ️ Use verbose parameters available both for **Ipfixprobe** and **Ipfixcol2** for troubleshooting. </note>

      
      </panel>
</accordion>

## General usage

When monitoring network traffic with **ipfixprobe**, it is important to consider **where and how** to place the probe. Correct placement affects data visibility, performance, and the ability to detect security events.

<accordion>
    <panel>
        <label><heading h4>Where to monitor the traffic</heading></label>
#### On the monitored device (localhost)

  *Simple deployment without the need for additional hardware. 
  *Ability to detect encripted data **before encription** (e.g. in the application layer).
  *The overhead of running the probe affects the monitored device.

#### In front of the network

  *Good for monitoring **NETWORK <-> INTERNET** traffic.
  *Difficult to attribute traffic to a specific internal host.
  *Cannot see communication **within the** internal network.

#### Inside network

  *Good for monitoring lateral movement (device <-> device traffic).
  *Possible need for more probes to cover the whole network.


        </panel>
        <panel>
        <label><heading h4>One-way vs Two-way communication</heading></label>

  *Seeing **both directions** is generally more accurate (TCP handshake, responses, and lack of them, DoS attacks...). 
  *Some plugins **cannot** work correctly without a response.
  *However, one-way monitoring may be **sufficient** for some use cases (e.g., outbound-only detection, flow volume statistic) and consumes fewer resources.

        </panel>
        <panel>
        <label><heading h4>How to tap into the network</heading></label>

#### TAP (Test Access Point)
  *Dedicated hardware for network analysis.
  *Provides complete, reliable visibility.
  *Good for long-term network monitoring.
  *More expensive, but provides consistent results.


#### SPAN (Switched Port Analyzer)
  *Software solution for monitoring the network.
  *In high packet counts, packets can be dropped -> **may miss events**.
  *Cheaper and easier to deploy.
  *Less reliable than TAP.

        </panel>
             <panel>
        <label><heading h4>One vs More ports</heading></label>

#### One port monitoring

  *Less demanding
  *Unable to see both sides of communication

#### Two or more port monitoring 

  *Needs aggregation of data
  *Able to see both sides of communication
        </panel>
</accordion>