hledat:

Deployment of CESNET2+ E2E Services in 2007

CESNET technical report number 18/2007
also available in PDF, PostScript, and XML formats.

Václav Novák, Pavel Šmrha, Josef Verich
2.12.2007

1   Abstract

This paper describes design and implementation of the E2E services in the CESNET2+ research network in the end of 2007. The current CESNET2+ backbone offers the E2E services based on various technologies (optical transmission DWDM and IP/MPLS network layers) and on the national and international level based on the cooperation with GN2+ network and other NRENs.

Keywords: DWDM, ROADM, cross-border fiber, alien wavelength, performance monitoring, forward error correction, wavelength cross-connect, virtual private LAN service.

2   Introduction

The E2E services are defined as the End-to-End connection realized using the OSI layer 1 and 2 technologies such as Ethernet. E2E link connects End-Points in organizations which may be hosted in the same or different domains and they may cross additional network domains. Here a domain may be a regional network, an NREN, a multinational network line GN2+ or any other network.

CESNET2+ network supports both, intradomain and interdomain E2E for the research projects and activities and advanced users with the cooperation with GN2+ and other NRENs.

Traditional E2E services are designed as the Point-to-Point services. However, there are some applications and projects which needs the multipoint Ethernet services. There is a workaround solution based on external L2 switches to simulate multipoint functionality. This solution is not flexible from the end user perspective, because is not quite transparent its services. The main disadvantage of this solution is missing support for transparent QinQ services. Within the research task "CESNET2+ DWDM and IP/MPLS deployment" we investigated possible implementations of the transparent multipoint Ethernet services as the pilot projects. In following paragraphs we will present several practical cases.

3   POSN

The main objective of the POSN (Private Hospital Optical Network) is to build private network between the hospital data centers to store and share medical data processed by computer tomographs, X-ray and many other medical equipments. This project is covered by our application support activity and we participated on the technical solutions.

For solution of this network we used static DWDM based on CLA amplifiers. The hospital data centers operates SAN networks with the Fiber Channel (FC) switches, see Figure.

The data transmission and data sharing in the SAN environment are very sensitive to data circuit quality andit requires very low latency. For security and other reasons, the data circuits must be designed as private and separated from normal network traffic.

The static DWDM solution on the top of optical line allows strict separation of internet and SAN traffic on Layer 1 and offers low latency for used FCIP protocol between the FC switches. Because there is requirement to connect Point-to-Point lines and establish the shared L2 infrastructure, we used as the cost effective solutions VLAN based central L2 switch. The static DWDM systems based on CLA was a reasonable and cost-effective solution for such simple network.

4   L2 over DWDM

METACentrum as a grid computing facility located in three cities came with the idea secure virtualisation of its nodes. Pilot project connects the DWDM nodes Praha, Brno and Plzeň by the 10GE Xponder cards in the ring topology (needed for the protection mechanism). As part of E2E multipoint Ethernet services we deployed and verified with the METACentrum project new Xponder card for Cisco ONS15454 MSTP which allows Layer2 operations (VLAN or QinQ) over the DWDM optical transport system (see Figure). It also offers the protection mechanism integrating G.709 messaging with the Ethernet layer, so there is sub 50 ms protection time achieved.

The 10GE XPonder card has two client 10GE ports and two trunk 10GE ports with the DWDM XFP support. It supports more operating modes (2x 10GE transponder, L2 switch or regenerating) so it's more flexible compared with traditional transponders. It supports FEC/E-FEC, but trunk ports are not tunable and it's possible to use only DWDM XFP for fixed wavelength.

For testing was used Linux based WS with the iperf testing SW in the nodes with the 802.1Q connection to the Xponders. We also have configured the jumbo frames of 9212 bytes and protection mechanism enabled (with master in Praha). On the XPonder client ports we hade to increase the Ethernet "Committed Burst Size" and "Excess Burst Size" from default of 4k to 4M and 8M to achieve good performance results.

The maximum transmission speed was about 9.8 Gbps without any packet loss.

The average RTT on the longest optical channel Praha-Brno (via Hradec Králové and Olomouc) was about 5.43 msec (which corresponds to the actual channel length of 462 km).

The XPonders can be used as a method for building virtual multipoint Ethernet services with the sub 50 ms recovery time for protection. But it is important to be aware of protection mechanism based on G.709, because th traffic is flooded around all 10GE DWDM ring.

5   VPLS over IP/MPLS

VPLS stands for Virtual Private LAN Service, and is a L2 VPN technology that enables Ethernet multipoint services (EMSs) over a packet-switched network infrastructure, e.g. IP/MPLS networks. VPN users get an emulated LAN segment that offers a Layer 2 broadcast domain. The end users perceive the service as a virtual private Ethernet switch that forwards frames to their respective destinations within the L2 VPN. VPLS as a multipoint technology allows a user to reach multiple destinations through a single physical or logical connection. This requires the network to make a forwarding decision based on the destination of the packet. Within the context of VPLS, this means that the network makes a forwarding decision based on the destination MAC address of the Ethernet frame. A multipoint service is attractive because less connections are required to achieve full connectivity between multiple points. An equivalent level of connectivity based on a point-to-point technology requires a much larger number of connections or the use of suboptimal packet forwarding. Cisco VPLS implementation is based on the IETF draft draft-ietf-pppvpn-vpls-ldp, which has wide industry support (currently there is no RFC standard, because VPLS specifications are still under development at the IETF).

The main driving force for the introduction of the VPLS services within the CESNET2+ IP/MPLS infrastructure was the demand of the METACentrum project for the standards-based multipoint L2 VPNs like Metro Ethernet Forum (MEF) E-LAN services to provide transparent LAN services (TLS) for the mutual interconnection of the three main METACentrum sites in Praha, Brno and Plzeň. There was a strong demand for providing IEEE 802.1Q tunnelling (802.1 QinQ) among the three METACentrum sites through the CESNET2+ IP/MPLS backbone to preserve VLAN significance inside the global METACentrum 802,1Q domain and to ensure its partial segregation from the outside world using a proper L2 global METACentrum VPN. This setup should provide the logical independence of the internal (dynamic) VLAN allocation inside the global national METACentrum 802,1Q domain for advanced research in the field of grid computing, especially dynamic interconnections among several virtual machines. On the other hand, some of the METACentrum internal VLANs should have been accessible from the outside world to provide global IPv4 and IPv6 connectivity in a controlled manner.

The CESNET2+ first pilot VPLS implementation represents the simplest form: the METACentrum VPLS instance consists of three METACentrum sites in Praha, Brno and Plzeň connected to the respective provider edge (PE) Cisco 7609 routers implementing the emulated LAN service at each of the three GigaPoPs. These PE routers make the forwarding decisions between sites and encapsulate the Ethernet frames across an IP/MPLS backbone network using a pseudowire. A virtual switching instance (VSI) is used at each PE to implement the forwarding decisions of this VPLS. The provider edges use a full mesh of Ethernet pseudowires to forward the Ethernet frames between themselves.

To achieve VPLS support within the CESNET2+ IP/MPLS backbone all three Cisco 7609 PE routers in Praha, Brno and Plzeň GigaPoPs had to be equipped with the new ES-20 Ethernet Services 20G Line Cards each with two 10 GE MPLS core facing interfaces. The METACentrum local L2 switches in each site can be connected.

• either to the VPLS instance through the 802.1Q trunks via ordinary PE Cisco 7609 LAN card interfaces configured as QinQ tunnels to provide L2 TLS,

• or some of the internal METACentrum VLANs can be L3 routed via ordinary PE Cisco 7609 LAN card interfaces configured as 802.1Q L3 end points for these VLANs (see Figure).

6   CESNET2+ and GN2+ E2E services

The new type of end-to-end (formerly E2E) guaranteed services is being developed within the GÉANT2+ research network activities. The GN2+ Ethernet/SDH switches for E2E services support EVPL (Ethernet Virtual Private Lines) on 10GE cards, which allow the mapping of Ethernet VLANs into VCGs (Virtual Concatenation Group). The VCG members are transported as individual VCs (Virtual Concatenation, VCAT) across the GÉANT2+ SONET/SDH network layer.

Distribution of GN2+ E2E services on CESNET2+ is served the by L2/L3 switch Foundry BigIron RX-8 which provides service aggregation based on VLANs. There is an 802.1Q trunk configured with the GN2+ Ethernet/SDH switch (see Figure). It is also possible to use 1 GE ports of GN2+ switch without the 802.1Q functionality.

The aggregation L2 switch Foundry BigIron RX-8 is also interconnected with the CzechLight facility switch Force10 E300 at 10 GE with the 802.1Q. This connection allows flexible E2E services configuration between the operational CESNET2+ backbone and CzechLight experimental facility. CzechLight operates experimental 10 GE line to StarLight, which servers E2E services as well (1GE channels for Czech Physics and others).

Today, current CESNET network equipments provides more methods and gives more possibilities for E2E services delivery to end users using the CESNET2+ backbone network service:

• via the DWDM optical transmission system based on ONS 15454 MSTP as an alien wavelength, L1 (termination on the transponders) or L2 (using the XPonder card)

• via the static DWDM system based on CL family as a wavelength

• via the IP/MPLS network as Point-to-Point EoMPLS tunnels or multipoint L2 Ethernet service (in the selected PoPs only)

• via the CzechLight facility testbed if needed and possible

However, there are the many problems with the local loops and academic metropolitan/university backbone networks to transport E2E services, because most of them are not ready for E2E services. In general, they are limited only on using VLANs to deliver E2E services to their end users or a dedicated local loop has to be used.

Each E2E service solution is different, because there are different environments in the cooperating metropolitan/university backbone networks.

Similar situation as it was described within the CESNET2+ network is in European academic environment connected via GN2+. There are multiple domains (CESNET2+ backbone network, metropolitan/university networks, etc.). Wherever end-to-end links are composed of elements provided by multiple domains, coordination is required between the network administrators of those domains. It must include the E2E service planning, operations and monitoring as well.

7   Conclusions and future development

The CESNET2+ backbone network supports and delivers the E2E services to its end users using different solutions and technologies. It also provides these services with the GN2+ and other NRENs on international level. We also cooperate on the GN2+ research activities related to E2E services in multidomain environment.