Monitoring of network traffic

A wide spread technique to identify QoS problems of applications in networks is scanning the network traffic in order to detect transaction-like behavior. Looking at IP-based networks, traffic occurring successively between the same pair of source and destination address is called a flow that, in combination with the application's port number, helps to find out start and end point of a client request and its response. In case of high-speed networks an evaluation of the measured data mostly cannot be done on-the-fly and therefore flows are recorded in real-time but analyzed at a later date. If network nodes (e.g., routers) do not support recording of flows, especially suited devices (so called probes) are installed in the network that provide the needed functionality. Advantages of this method are that no source code access is needed and every application that communicates over the network can be monitored.

Nevertheless, there are several disadvantages. As usually only transactions of standard applications can be detected automatically the system administrator must have in-depth knowledge about protocols of custom-designed applications to configure probes in that way that they distill individual transactions. Even worse, many protocols are used for different purposes than originally designed (e.g., SOAP uses HTTP for its RPC mechanism), which further increases complexity. In case of encrypted communication monitoring of network traffic is not suitable at all. As mentioned above, due to massive data volumes in high-speed networks the data analysis cannot be done in real-time and therefore cannot be used to react in time when a problem occurs. Additionally, it is impossible to determine the reason of a QoS degradation as long as it is not a network failure. Furthermore, time stamps are not taken from a user's point of view but at the network meter point.

Overall, solely monitoring network traffic is not suited well for application service performance monitoring as this technique was originally developed and used to detect network failures, for capacity planning and for reporting. There are several working groups within the IETF that develop standards in the field of network based (application) performance monitoring: e.g., IP Performance Metrics (IPPM) [#!rfc2330!#] and Realtime Traffic Flow Measurement (RTFM) [#!rfc2063!#]. Several companies offer probes and analyzing software using this technique: e.g., CompuWare's EcoSCOPE [#!comp99!#], and Apptitude's MeterFlow [#!apmf00!#].