Virtual Network Monitoring in FEDERICA Project

CESNET technical report 12/2010

Vojtěch Krmíček, Pavel Čeleda

CESNET, z.s.p.o.

Received 22.11.2010

Other formats: PDF, EPUB

Abstract

In this technical report, we present a framework for virtual network monitoring, which was deployed in FEDERICA project (Federated E-infrastructure Dedicated to European Researchers Innovating in Computing network Architectures) [1]. It uses a flow-based approach, which acquires NetFlow data from the FEDERICA network and processes it by the open-source collector NfSen [2]. The NfSen collector doesn't provide a special tool for the monitoring of virtual networks. Especially we have no long-term statistics about the FEDERICA network accessible in real-time, a possibility to inspect the traffic with the respect of particular virtual networks or a possibility to generate regular report statistics. Therefore we have extended the NfSen collector by the set of tools supporting virtual network monitoring.

These tools support real-time access to the long-term monitoring data and statistics, provides various types of reports and performs more detailed views of the virtual network traffic.

Keywords: FEDERICA, virtual network monitoring, NetFlow, FlowMon, VLAN, NfSen, 802.1Q.

1  Introduction

FEDERICA [1] is an EU founded project focused on implementation of an experimental network infrastructure for trialling new networking technologies. It is based on the principle of virtualization and builds virtual infrastructure over the physical layer of the network using virtual links and virtual nodes [3], see Figure 1.

[Image]

Figure 1. Scheme of the FEDERICA virtual and physical layers.

Therefore FEDERICA as a large virtual network has to have a reliable monitoring and supervising system. We need to know what types of data and which routes are transferred through this network, monitor behaviour of particular nodes, provide FEDERICA users detailed information about performed experiments etc.

There are two basic domains, which we need to monitor in the FEDERICA network. The first one is focused mainly on the monitoring and management of the FEDERICA network devices. The SNMP technology suits well for such purpose. This type of monitoring consist mainly of:

The second domain of monitoring is focused on the flow-based traffic measurement of the FEDERICA network and consists of:

In the next section, we introduce a virtual network monitoring system and a FlowMon probe, which was used for the generation of flow data (in the form of NetFlow version 9). We describe an overall architecture of the monitoring system and also we present the extension of NetFlow version 9 format to be able to monitor virtual networks. Section 3 describes NetFlow collector NfSen and our use of this tool for the monitoring of virtual networks. Section 4 focuses in detail on the plugins for NfSen collector, which we developed during our work. They provide valuable statistics and details about the traffic and behaviour of the observed network.

2  Virtual Network Monitoring System

During the initial phase of the project, we needed to choose the appropriate solution for the independent network traffic monitoring. The solution based on in-line monitoring (e.g. the router inside the monitoring network generating the NetFlow data) were not acceptable. In such case the measurement can be influenced by various situations in observed network like network tuning, testing, etc. We decided to use a dedicated monitoring solution, which would be completely independent on the various network states.

Another aspect of network traffic monitoring is that it is better to avoid to use internal NetFlow engines in network nodes (routers, switches). The states of network nodes could be unstable and we can get incomplete information. On the other hand the monitoring at the level of network links is independent of the various states in the network nodes and precisely reflects the network behaviour.

2.1  FlowMon Appliance

Therefore, we decided to choose a completely independent network monitoring appliance FlowMon [5], which is not connected to the observed network directly, but just receives the copy of transferred traffic via specialized devices like network TAPs (Test Access Port) or monitoring ports. Such device should provide reliable network statistics in the form of NetFlow data independently on the state of the network.

Used FlowMon probe was specially customized for FEDERICA project (added VLAN support). FlowMon probe provides following capabilities:

The possibility to monitor VLAN tags in the observed traffic was crucial, because the virtual links and virtual networks in the FEDERICA project are identified by the VLAN tags as defined in IEEE 802.1Q standard [6], see Figure 2. The ordinary devices (as routers) generating NetFlow data are not able to inspect and report VLAN tag information. Therefore the FlowMon appliance was enhanced to be able to report VLAN tag information in NetFlow data.

[Image]

Figure 2. Scheme of 802.1Q VLAN standard and VLAN representation in NetFlow format.

Similar issue arises at the other end of the NetFlow metering process. Although NetFlow version 9 format defines VLAN tags (see RFC 3954 [7]) and we are able to transfer VLAN tags to the NetFlow collectors, NfSen NetFlow collectors doesn't support VLAN tag handling and storing. Therefore we have overloaded DST_AS field of NetFlow version 9 record and used it for the storing the VLAN tags. Consequently, we were able to work with these VLAN tags at the side of the NfSen NetFlow collector.

[Image]

Figure 3. FlowMon appliance architecture.

Figure 3 provides the overview of the layered architecture of the FlowMon appliance. The bottom Network Layer is responsible for the generation of the network traffic copy and passing it in the form of packets to the NetFlow Generation Layer.

The NetFlow Generation Layer processes the packets and FlowMon exporters aggregate them to the NetFlow data including VLAN tags. Exported flows are stored in the Collector Layer to the NFDUMP [4] data files for further processing.

Top layer is responsible for the processing and presentation of the NetFlow data to the FEDERICA operators and users. We use customized NfSen collector extended by the set of specialized VLAN profiles, alerts and monitoring plugins, as described later.

2.2  FlowMon Deployment at Prague Node

Figure 4 describes the structure of the core FEDERICA PoP (Point of Presence) in Prague disposing with the FlowMon appliance and the connection of FlowMon probe ports to the particular lines. There are four backbone lines connecting the Prague PoP with the FEDERICA core PoPs in Athens, Erlangen, Milan and Poznan. FlowMon probe observes eight fiber links (each FEDERICA line consists of two fiber links representing both directions of the traffic).

[Image]

Figure 4. Deployment of FlowMon appliance at Prague node.

3  NfSen Customization for Virtual Network Monitoring

The NetFlow data generated by the FlowMon exporters are handled by the NfSen collector [2]. This collector is responsible for the NetFlow data acquisition, storing and basic visualisation. We have enhanced its functionality for supporting virtual network monitoring. In the following, we will describe in detail these features, provided to the FEDERICA administrators and users.

3.1  Customized NfSen Profiles

NfSen profiles are specific views on the stored NetFlow data. A profile is defined by its name, type and one or more profile filters, which are any valid filters accepted by the NfSen tool. We provide a predefined set of profiles and groups of profiles in the FlowMon web interface offering various views on the FEDERICA traffic data. The users can choose from the following profile groups:

[Image]

Figure 5. NfSen Profile representing network traffic in particular virtual networks.

Because the VLAN networks can be created dynamically in the FEDERICA network, we have developed a tool for an automatic detection of new VLAN traffic in the FEDERICA network. This tool detects a new VLAN network and creates a new profile for this VLAN automatically.

3.2  Automatic Alerting

The standard NfSen collector interface provides a tool for automatic alerting. The network administrator can define a set of alerts based on the conditions depending on the NetFlow data. The alert itself can execute a specific action based on specific conditions.

In the case of the FlowMon monitoring center for FEDERICA network, we have predefined a set of alerts useful for the links state surveillance. If there is an outage in the network traffic in the particular FEDERICA link, an alert is triggered and the network administrators receive an email informing them about the outage details.

3.3  NfSen Plugin API

NfSen may be extended with plugins to provide additional functionality. We can add two types of plugins - backend plugins and frontend plugins.

The backend plugins are loaded into the NfSen background, while is NfSen collector started and provide several functions including periodic data processing, alerting conditions and alerting actions. The frontend plugins display visually any results of the backend processing through web interface. Backend plugins are implemented as Perl modules and frontend plugins as PHP files. Both plugins may exchange relevant data over the standard UNIX socket. In the following section, we will describe plugins we developed to support virtual network monitoring in FEDERICA network.

4  Collector Plugins Supporting the Virtual Network and VMware Monitoring

The goal of our work was to extend NfSen collector by specialized plugins, which perform a regular reporting and provide an advanced monitoring interface with special focus on virtual networks. This new interface supports real-time access to the long-term monitoring data and statistics and performs more detailed views of the virtual network traffic.

We divided this work in two separate plugins. The first one is focused on the monitoring and analysis of the virtual networks. The second one performs a detailed analysis and comparison of NetFlow data and SNMP data from VMware virtual hosts. Both plugins should provide FEDERICA users desired statistics and knowledge about the current and past states of the FEDERICA network.

4.1  NfSen Plugin Supporting Virtual Network Monitoring

First plugin is focused on the monitoring of virtual networks inside the FEDERICA. Although traffic in FEDERICA network is divided into the particular virtual networks (VLANs), NfSen collector has no standard tool for detailed VLAN traffic analysis. Therefore main goals of this plugin are to process stored VLAN tags (as described in Section 2.1) and provide detailed statistics about VLAN traffic, provide graphical representation of monitored data and also provide regular reporting to the FEDERICA users.

[Image]

Figure 6. Scheme of the NfSen Plugin Supporting Virtual Network Monitoring.

The detailed architecture of the NfSen Virtual Network Monitoring Plugin is displayed at the Figure 6. The plugin consists of three frontend parts and two backend parts. VLAN traffic overview is shown at Figure 7. The frontend parts are following:

[Image]

Figure 7. VLAN's traffic displayed by the NfSen Plugin Supporting Virtual Network Monitoring.

The backend part consists of the following parts:

4.2  NfSen Plugin Supporting NetFlow and SNMP VMware Monitoring

The second plugin is focused on the monitoring of traffic generated by the VMware machines inside the FEDERICA network. This plugin correlates the SNMP data provided by the particular VMware machine together with the NetFlow traffic data acquired by the FlowMon probe. Such correlation provides the overall view to the state of the FEDERICA network and to the performed experiments in the particular VLAN networks and slices.

[Image]

Figure 8. Scheme of the NfSen Plugin Supporting NetFlow and SNMP VMware Monitoring.

The detailed architecture of the NfSen NetFlow and SNMP VMware Monitoring Plugin is displayed at the Figure 8. The plugin consists of three frontend parts and three backend parts. The frontend parts are following:

The backend part consists of the following parts:

Precise time synchronisation is very important for the correct representation of the NetFlow and SNMP data. Therefore the ntp daemon is installed and running at the FlowMon appliance and VMware machines.

An example of the overview page representing the slice scheme and the traffic overview is at Figure 9 and detailed plugin output representing both NetFlow and SNMP data is at Figure 10. We can see an experiment periodically performed in the CESNET slice. Iperf tool is used to generate and transfer a large amount of traffic between the PoPs in Prague, Milan and Poznan.

[Image]

Figure 9. Overview of the slice scheme and slice traffic represented by the NfSen Plugin Supporting NetFlow and SNMP VMware Monitoring.

[Image]

Figure 10. Detailed comparison of NetFlow traffic and VMware SNMP records in CESNET slice represented by the NfSen Plugin Supporting NetFlow and SNMP VMware Monitoring.

5  Conclusion

In this technical report, we have presented our work in FEDERICA project. Our main contributions are (i) design of the flow-based monitoring system for the FEDERICA network, (ii) implementation of the monitoring system to the FEDERICA network, (iii) extension of NfSen collector by the tools for virtual network monitoring and (iv) running this monitoring systems operationally in FEDERICA network.

In detail, we have designed NetFlow monitoring system and deployed customized FlowMon probe 8000 in Prague's PoP. Then we have customized NetFlow collector by a set of FEDERICA profiles and automatic alerts.

In the consequent part of our work, we have implemented two collector plugins. The first one provides detailed information about virtual networks in FEDERICA network and regular reporting about network statistics by representing stored NetFlow and VLAN data. The second one is focused on the NetFlow and SNMP VMware monitoring and provides parallel views to the both type of statistics.

With these tools, the administrators of the FEDERICA network have a better overview of the state of the virtual networks and could easily access information about particular VLAN traffic. The end users of FEDERICA network have the detailed view of their experiments performed inside the FEDERICA network.

6  Acknowledgements

This work was supported by FEDERICA project Seventh Framework Program of EU grant agreement 213107.

References

[1] FEDERICA - Federated E-infrastructure Dedicated to European Researchers Innovating in Computing network Architectures. 2010 [cit. 2010-11-01]. Available online.
[2] HAAG, P. NfSen - NetFlow Sensor. 2010 [cit. 2010-11-01]. Available online.
[3] FEDERICA - European Virtualisation Project is Looking for Future Internet Researchers. 2008 [cit. 2010-11-01]. Available online.
[4] HAAG, P. NFDUMP. 2010 [cit. 2010-11-01]. Available online.
[5] INVEA-TECH a.s. INVEA FlowMon Probe 8000. 2010 [cit. 2010-11-01]. Available online.
[6] IEEE. 802.1Q: IEEE Standard for Local and metropolitan area networks. Virtual Bridged Local Area Networks. 19 May 2006 [cit. 2010-11-01]. ISBN 0-7381-3662-X. Available online,
[7] CLAISE, B. Cisco Systems NetFlow Services Export Version 9. RFC 3954, IETF, October 2004.
[8] The jQuery Project. 2010 [cit. 2010-11-01]. Available online.
[9] OETIKER, T. RRDtool. 2010 [cit. 2010-11-01]. Available online.
další weby:fond rozvojemetacentrumCzechLightpřenosyvideoservereduroameduID.cz