C-GMA Capability-based Grid Monitoring Architecture

CESNET technical report number 17/2006
also available in PDF, PostScript, and XML formats.

Ondřej Krajíček, Aleš Křenek, Luděk Matyska, Miroslav Ruda, Zdeněk Salvet, Jiří Sitera, Michal Voců
19.12.2006

1   Abstract

We present an ongoing research in the field of Grid Monitoring Services with the main intent to build an interoperable and scalable monitoring infrastructure which would allow an integration of various existing monitoring sources and frameworks. This report is based on merging the work published by the authors elsewhere [Cec03], and extended with further technical details.

The proposal is based on introducing another layer of metadata, component capabilities and data attributes. Defined handling of these allows co-existence of components coming from originally independent implementations in a single infrastructure.

We demonstrate the conceptual ideas on two proposals of the capability layer, and we show fitting two different monitoring architectures into the concept.

The report is concluded with notes on foreseen implementations and further extensions of the concept.

2   Motivation

A general architecture specification of monitoring services on the grid is described in the GGF document Grid Monitoring Architecture [Tie01] (GMA). GMA defines core components and their interactions: producer (source of monitoring events), consumer (event sink) and directory (catalogue of data types and sources). The downside of the generality of the GMA specification is the fact that it allows multiple implementations that are not mutually interoperable, though all are GMA-compliant.

Design of various GMA implementations is driven by diverse requirements, which may become even contradictory and impossible to satisfy at once (e. g. security vs. high-throughput, reliability vs. up-to-date information). Moreover, different data models and query capabilitites may be suitable for different purposes.

Therefore the principal goal of the C-GMA proposal is allowing coexistence and collaboration of different GMA implementations rather than proposing universal architecture and/or data model. Due to the wide range of sometimes contradictory requirements on monitoring infrastructures we do not believe that such a universal model even exists.

Given the existence of infrastructure composed of cooperating components with different features, we assume that the data sources specify the conditions which should be imposed on handling that data within such infrastructure. Symmetrically, the data sinks specify their abilities to satisfy these conditions. Inspired by the notion of quality of service (QoS) from the networking world, we see the data handling conditions as a contract between the infrastructure components.

3   Concepts

The C-GMA proposal is based on the GMA general architecture and inherits all its components and functionality, with addition of another component, a mediator. To facilitate co-existence of components with different properties, or even coming from different GMA implementations ( worlds ), we introduce the concept of data attributes and component capabilities, further described in Section. The C-GMA components and their interactions are described in Section, where we also discuss the process of matching data attributes and component capabilities to create paths for data transfer. We also introduce sessions (Section) to avoid the re-negotiation of path properties between components. The C-GMA goal is providing infrastructure specification which would alow not only creation of new compliant components and implementations, but also enable incorporation of components coming from already existing implementations; therefore our proposal is divided into two layers, as described in Section.

3.1   Data attributes and component capabilities

In order to allow components with different properties to cooperate, we need means to describe component features; similarly we need instruments to describe data handling conditions. This is done by introducing additional metadata, component capabilities and data attributes. They determine how and which components of the infrastructure may talk to each other, as well as how particular data should be handled; otherwise they are orthogonal to the traditional metadata, i. e. the schema or types of the data payload.

3.1.1   Capabilities

Components declare via their capabilities any features that may affect either the possibility of communication with other components or their ability to handle particular data. Component capabilities are e. g.:

We have to emphasize that this list contains illustrative examples only. The C-GMA proposal neither defines any fixed capabilities nor assumes their specific semantics.

3.1.2   Attributes

On the other hand, meta-description of data expressing in which way and to which components the concrete data may be handed over is expressed with data attributes. Data attributes may be e. g.:

In order to prevent confusion we emphasize that neither data attributes nor component capabilities are related to event data types or component interfaces.

3.1.3   Capability language

Capabilities and attributes are expressed in a capability language (Section). Having the common language for both these entities allows treating them in a symmetric way, namely expressing requirements on capabilities in attributes, and vice versa. Moreover it gives us the possibility to express capabilities and attributes descriptively (i. e. not using explicit requirements at all) and encapsulate the matching logic entirely into separate component; the attributes can restrict data handling by components without an a-priori knowledge of specific component capabilities.

The capability language must satisfy certain minimal requirements (Section) but no fixed language is prescribed. In Section we propose two different capability languages.

3.1.4   Attribute scope

Unlike capabilities, which are always assigned to a single component instance, attributes may occur at three levels (from the most general to the most specific):

However, none of these levels is mandatory - a C-GMA implementation may choose to support only some of them1. On the other hand, when an implementation allows more than one level of attribute specification, conflicts may arise. Hence in this case the capability language must provide the operation of attribute override - given two attribute specifications (one more general, e. g. component level, the other more specific, e. g. data record level) it yields a single consistent attribute specification at the more specific level.

3.1.5   Attribute and capability semantics

Despite C-GMA central components (namely the Mediator, see Section) process capabilities and attributes in order to match producers with consumers, C-GMA itself does neither define any concrete capabilities and attributes nor it interpretes them in any way. For C-GMA these are semantics-free objects which are manipulated in a pure syntactic way (e. g. compared for equality). Actual meaning of capabilities/attributes is known and evaluated only by concrete producers and consumers.

3.1.6   Security issues

When evaluating attributes/capabilities by producers and consumers, certain level of trust may be required. Namely when component advertises a capability, the other party has to be able to verify that the capability is not forged, i. e. that the component really treats data in the way it advertises. This issue can be addressed in a concrete infrastructure by attaching some sort of digital signature to advertised capabilities (and possibly attributes). However, for the sake of generality, we do not require any concrete security mechanism; we even don t require any such mechanism to be present. The decision is left to a concrete implementation.

3.2   Infrastructure components

Component definition and model of communication between components is based on Grid Monitoring Architecture (GMA) [Tie01]. New component, Mediator is defined and integrated into the model of communication.

C-GMA architecture then consists of these components (or, to be more precise, components may play the following roles):

Producer

makes monitoring data available in the form of events using either push (publish/subscribe, producer sends events independently) or pull model (query/response, producer performs specific monitoring action on request by consumer).

Consumer

is component receiving data from producers.

Registry (Directory Service)

is an index service, where producers and consumers are registered. In C-GMA, Registry is a passive component, just repository for the registration metadata describing producers and consumers. It is not accessed directly by producers and consumers but used only via mediator - its active counterpart.

Mediator

is new component, which is responsible for discovery of appropriate producer or consumer partner. The matching is done along two axes, using both attribute/capability layer and data definition layer (see Section); i. e. producer must have data the consumer wants and their capabilities must correspond to each other as well as to the data attributes, in order for the mediator to create a match.

The rationale for introducing the new mediator component is the following:

However, our proposal is implementation unbiased, mediator can be implemented as remote (central or distributed) service or library linked into producers and consumers. Analogically we don t constrain implementation of registry - it can be local file, central or distributed database or P2P network.

Unlike some GMA implementation (e. g. RGMA) C-GMA does not define an explicit Schema component. Data schema (event types using GMA terminology) is handled and enforced specificaly on the data-definition layer (see Section) which may include an explicit Schema component, though. Extended repository service is also possible solution.

The component model of C-GMA is illustrated in Figure.

[Figure]

Figure 1: C-GMA Component Model

3.3   Component interaction

Following original GMA proposal, monitoring data are send by producers as events within a defined event schema. The model of communication between producers and consumers follows GMA proposal, it is only extended to support sessions. During registration and discovery of corresponing component, producer and consumer interact with mediator instead of registry.

Interaction between components can be broken up into four phases (their naming is adopted from Condor matchmaking [RLS98]):

Advertising

registration of producers and consumers with the registry. The registration data include general information like component identification and endpoint address accompanied by the component capabilities and data attributes (if they are uniform for all data) at the capability metadata layer, as well as data description at the data-definition layer. Registration is soft-state, components must renew registration before expiration.

Matching

based on registered metadata, mediator is looking for matching producer/consumer pairs. When new pair is found, both parties are informed.

Claiming

- direct communication between the producer and the consumer (occurs when the component is notified about a corresponding party). Verification of mutual compatibility between components capabilities and data requirements must be done in this phase by the components.

It is expected that during claiming phase, native (non C-GMA) protocol will be used. However, C-GMA specification contains definition of optional claiming operations too. In some cases (i. e. for lightweight implementations), it may be beneficial to delegate claiming part to mediator.

Data transfer

- data (in the form of events) are sent directly between producer and consumer in native protocol. C-GMA session (see Section) can be used here.

Three types of data transfer are defined in GMA:

notification

- one stage communication initiated by producer, which sends events to (particular) consumer

query/response

- two stage communication initiated by consumer. Consumer sets transfer and producer sends all data in single response.

publish/subscribe

- three stage communication, which can be initiated both by producer and consumer. In the first stage, possibly several control messages are send to establish subscription. In the next stage, the producer sends one ore more events to the consumer. In a final stage, subscription is terminated either by the producer or the consumer.

The order of component interactions is shown in Figure.

[Figure]

Figure 2: Interaction of C-GMA Components

3.3.1   Component core operations

The GMA specification defines core operations for all components. In general the operations are paired - a communication between two components is reflected by operations on both sides.

In this section we provide the core operations that are added by C-GMA; most of them reflect introducing the new mediator component. Some of these operations are initiated by producers/consumers and some are initiated internally and involve calling out other component interface functions.

The mediator implements the following operations (arranged according to the communication phases introduced in the previous section):

Definition of all core operations for producer, consumer, and registry defined in GMA was not changed. New C-GMA specific core operations for producer and consumer were added:

Complete list of C-GMA specific core functions for producer, consumer and registry is in Appendix A.

3.4   C-GMA session

The claiming phase of communication set-up between consumer and producer can be rather time-consuming; moreover we do not expect producer and consumer capabilities to change quickly. It would then be beneficial to cache results of the claiming phase for subsequent usage with another data. The C-GMA sessions serve the purpose.

We refer to C-GMA session as a saved state of established communication channel between producer and consumer; this state contains at least results of claiming (i. e. capability verification), but may also include other channel parameters - encryption keys (SSL session), timeout settings etc.

The list of active C-GMA sessions must be maintained by all the producers and consumers which support them; the party initiating communication can refer to C-GMA session as a shortcut for establishing the necessary channel properties without the need to repeat the claiming phase.

3.5   Two-layer C-GMA specification

In previous sections we described the C-GMA concepts at an abstract level. However, our goal is providing an infrastructure which ensures interoperability (see Section for notes on interoperability) of components, even if those come from different independent original implementations. For achieving interoperability a strict definition of common protocols is required, though. A concrete specification of C-GMA-compliant monitoring infrastructure defines necessary requirements on an implementation of cooperating components, i. e. it forms one C-GMA world .

On the other hand, we still want to impose minimal requirements on the involved components, in particular we do not want to put any restrictions on the data description, representation and queries the components use.

Therefore the concrete specification of a C-GMA infrastructure is two-layered:

  1. A unique common definition of a capability layer contains the language used to express component capabilities and data attributes, as well as concrete C-GMA specific operations of all infrastructure components (i. e. mediator, producer and consumer).

  2. Multiple independent instances of data-definition layers define languages of describing, expressing, and querying data, as well as native operations specific for the given data layer instance. These definitions may come from original C-GMA non-aware implementation.

In this section we describe requirements on these layers of C-GMA specification. Section shows two concrete proposals of the capability layer definition (i. e. two independent, non-interoperable C-GMA worlds), while Section deals with two examples of data-layer definitions.

3.5.1   Capability layer

The definition of capability language is unique and common for all the components that are supposed to be part of the cooperating infrastructure. There are two mandatory operations the language must support:

Component matching.

Given capabilities of a producer and a consumer it must be possible to decide whether these components can communicate with each other, e. g. whether they implement the same protocol.

Attribute matching.

Given attributes of a piece of data and capabilities of both the producer and the consumer it must be possible to decide whether this producer may handle this data over to this consumer, e. g. whether data security requirements are satisfied.

In addition, if the infrastructure supports attribute specification at multiple levels (see Section), the following operation must be defined:

Attribute override.

Given two attribute specifications, one more general, the other more specific, merge them together and resolve possible conflicts. E. g. when one attribute is defined at both levels, it must be clear whether one of them takes precedence or both should be used to form a multivalued attribute etc.

As semantics of attributes is not known at this level, the capability language must contain e. g. directives to determine override behaviour for concrete attributes.

Once the capability language is fixed it is possible to define exactly the operations of components (Section), namely the mediator. The operations may be defined either in terms of communication protocol, e. g. WSDL s of the components, or an API.

In order to be complete the operation definitions must include parameters referring to items of data language. However, the data language is not only undefined at this layer yet, but it is also desirable not to bind the operations to a concrete data language (so that multiple data worlds may coexist within a single infrastructure, sharing the common capability language). Hence we treat the data-related arguments here as opaque, either string or binary objects. Their interpretation is left to concrete producers/consumers and the data-specific modules of mediator respectively.

3.5.2   Data-definition layer

Language used by producers to specify what data they generate and by consumers what data they request is defined at the data-definition layer of C-GMA specification. A concrete instance of data-definition language must be able to specify:

Data types

that identify uniquely what is the data entity a producer generates or a consumer requests.

E. g., in the case of relational data model, data definition language describes name of the table as well as names and types of individual columns.

In addition, in many real applications it is desirable to include also

Conditions

express a restriction on the domain given by a concrete datatype. The producer declares with these condition that it produces only data satisfying the conditions, the consumer expresses its interest only in such data and not others.

In the relational data model the conditions are WHERE clauses of the producer and consumer defining SQL statements.

3.5.3   Understanding interoperability

The C-GMA design assumes that there are multiple data definitions in various data languages all taking part in a single logical infrastructure, sharing the common capability layer definition.

Our goal is that these otherwise independent data worlds co-exist within a single infrastructure given by the common capability-layer definition. In this sense we consider two components of different data worlds to be interoperable if they are both able to register within a single infrastructure.

Quite obviously, this level of interoperability does not imply directly that those components are able to exchange data, i. e. achieve the interoperability in the more conventional sense.

However, the existence of common infrastructure allows the following:

4   Proposed capability languages

In this section we propose two languages suitable for use as the capability language in C-GMA architecture, one based on Condor s ClassAds and one on plain XML. Both of these languages fulfill the required criteria for use as capability language.

4.1   ClassAds

For description of service capabilities and data atributes, ClassAd language [RLS98] is used. Each of producer capabilities, consumer capabilities and data atributes will be represented by one ClassAd. ClassAds may contain static attributes and capability values and also explicit requirements on matching component.

For matching, standard matchmaking algorithm [RLS98] is used. To allow matching of three ClassAds, composed ClassAd which consists from ClassAds of producer, consumer and data attributes is used in the following form:

  { Producer = {
    ...
    }
    Data = {
    ...
    }
    Consumer = {
    ...
    }
  }

Requirements from all three sub-ClassAds are evaluated in context of this composed Class­Ad.

4.1.1   Simple example

 Producer = {
        Protocol = { http, https};
 }

 Data = {
        MinSecLevel = 4;
        Requirements = (.Consumer.SecurityLevel >= MinSecLevel);
 }

 Consumer = {
   Protocol = {https};
   SecurityLevel = 5;
   Requirements = member(Protocol,.Producer.Protocol);
 }

The example illustrates description of static capabilities and attributes and explicit requirements of consumer and data. References to attributes/capabilities of compared ClassAd are also demonstrated.

4.1.2   Implementation details

4.1.3   Registry implementation

In the first prototype, a simple central registry will be used. For each component, at least this information must be stored:

Complete API specification for mediator can be found in Appendix B.

4.2   XML and XPath based

The principal idea of this proposed capability language is describing both data attributes and component capabilities in terms of XML documents. Consequently XPath [CD99] expressions can be used to localize items in these documents as well as express various conditions.

The attribute/capability XML documents contain:

We considered two different approaches to the document schema: flat and structured (in terms of XML) constant values. Probably the structured approach is more general. However, the flat one is considerably easier to be handled in implementation while not restricting any of the principal C-GMA features. Therefore we stick with the flat model for the time being.

For the sake of simplicity we also assume that data attributes are specified at the component-level only. Consequently no override operation is required (see Section).

4.2.1   Constant attribute and capability values

Component capabilities are expressed as documents having <producer> and <consumer> root element, e. g.

   <producer>
        <cap name="protocol">A</cap>
        <cap name="encoding">AES</cap>
        <cap name="encoding">DES</cap>
        ...
   </producer>

where the <cap> element with the mandatory attribute name records constant capability values. Multi-valued capabilities are stored as multiple occurrences of the element.

Data attributes are expressed in a very similar form, a document with <data> root element and multiple <attr> elements inside.

The content of <cap> and <attr> elements must be CDATA in the considered flat model. There are also optional attributes common-cap and common-attr to specify simplified requirements, see Section.

4.2.2   Explicit requirements

Relationship between capabilities and attributes is expressed in terms of <req> elements, see examples in Section. For the purpose of matching the individual producer, consumer and data capability/attribute documents are assumed to be merged together, e. g. using XInclude [MOV06]:

<match>
    <producer> ... </producer>
    <consumer> ... </consumer>
    <data> ... </data>
</match>

The <req> elements, allowed in any of the <producer>, <consumer>, or <data> part, contain a mandatory XML attribute test which is an XPath expression being evaluated with the root <match> element of the above document as the starting context node.

A concrete producer, consumer, and data item match mutually if and only if all the <req> expressions are evaluated to true.

4.2.3   Implicit requirements

In addition to expressing requirements via the <req> elements described above we define shortcuts (i. e. they can be translated to <req> s in a straightforward way) for testing attribute/capability values for equality.

Either <cap> or <attr> elements may specify additional attribute common-cap and common-attr having the value of "producer", "consumer", or "data". In this way existence of <cap> or <attr> in the refered-to party is required, having the same value as the referring element.

Currently, the value of of common-cap and common-attr is redundant from the context it is always clear which party is refered to, i. e. it might have boolean values only. However, it is defined for future extensions.

4.2.4   Simple example

<producer>
  <cap name="protocol" common-cap="consumer">A</cap>
</producer>

<consumer>
  <cap name="protocol" common-cap="producer">A</cap>
  <cap name="securityLevel">5</cap>
</consumer>

<data>
  <!- without "common", i.e. private ->
  <attr name="minSecLevel">4</attr>

  <req test="consumer/cap[@name='securityLevel'] >= 
    data/attr[@name='minSecLevel']" />
</data>

The example illustrates both explicit and implicit requirements. Both producer and consumer require the other component to speak the same protocol via implicit requirement (common-cap attribute). In addition, data require the consumer to satisfy certain security condition via an explicit <req> expression.

4.2.5   Core functions mapping

As explained in Section an exact specification of component interfaces is done at the capability layer in order to allow for interoperability.

For an infrastructure based on the XML-based capability language defined in this section we assume the components implementation to be hidden behind a well-defined API. The C-GMA core functions (Appendix A) are mapped to API calls, usually pairwise, i. e. a pair of corresponding core functions is implemented with a client-side API call, while the potential communication with the server is hidden in the API implementation, and server-side callback function eventually.

Despite this approach may look less flexible (e. g. a restricted programming language binding), it allows more implementation flexibility, e. g. the mediator can be seamlessly implemented as a distributed service.

The API exports the following C-GMA-specific calls to the mediator service:

cgma_registration

implements MaintainRegistration at the producer/consumer side and Add, Update, and Remove at the mediator side.

cgma_search

implements LocateConsumer (LocateProducer) at the consumer (producer) side, and Search on mediator.

On the other hand, the following calls are used to activate producer and consumer functions:

cgma_match_notify

implements MatchNotify on mediator and AcceptMatchNotify on producer/consumer. It is called by mediator for each found pair of matching components. It is the responsibility of the API implementation to deliver appropriate messages to both the involved components.

cgma_query_cap

implements the pair QueryCap/AcceptQueryCap symmetrically for a producer to query consumer s capabilities or vice versa.

C-GMA sessions are not supported in this implementation currently, hence the corresponding core functions are not implemented.

The rest of producer and consumer core functions is related to their data communication; therefore it is left to be specified in data-layer protocols.

More exhaustive description of C binding of the API in this section is presented in Appendix C.

5   Considered data models

In this section we present examples of two different data models, one based on trivial relational language represented in XML and one taken from the Mercury monitoring system; our goal is to show how different data models can be implemented in the C-GMA architecture.

5.1   Trivial relational language

The data language presented in this section is very simple and artificial by intention. Its purpose is C-GMA proof-of-concept only, we want to demonstrate the important C-GMA features while avoiding implementation problems of complex real data languages.

The language is relational, i. e. the principal data entity is a table with a fixed number of named and typed columns. Rows of the tables are GMA data events passed from producer to consumer (cf. R-GMA, Section). Querying capabilities are limited, there is no equivalent of join operation, and filtering conditions are restricted too.

5.1.1   Data types

For a particular table XXX the following XML Schema types are defined:

  <xs:complexType name="XXXRow">
    <xs:sequence>
      <xs:element name="YYY" type="xs:TYPE" 
        nillable="true/false"
        minOccurs="1" maxOccurs="1"/>
      ...
    </xs:sequence>
  </xs:complexType>

  <xs:complexType name="XXX">
    <xs:sequence>
      <xs:element name="row" type="XXXRow"
        minOccurs="0" maxOccurs="unbounded"/> 
    </xs:sequence>
  </xs:complexType>

where TYPE is further restricted to int, string, or dateTime. An element of the resulting type XXXRow represents a single row of the table XXX (i. e. a single GMA event) while the type XXX represents a fragment of the table.

5.1.2   Filters

Producers describe the table fragments they produce as well as consumers declare the data they are interested in in terms of filters. A filter can basically express a very restricted subset of SQL SELECT statements:

  <filter table="XXX">
    <conditions column="YYY">
      <compare op="OP1">VALUE1</compare>
      <compare op="OP2">VALUE2</compare>
      ...
    </conditions>
    <conditions column="ZZZ">
      <compare op="OP3">VALUE3</compare>
      <compare op="OP4">VALUE4</compare>
      ...
    </conditions>
    ...
  </filter>

where OP s are one of <,>,=,<>. The shown filter has the same semantics as the following SQL statement:

  select * from XXX
  where (YYY OP1 VALUE1 or YYY OP2 VALUE2)
    and (ZZZ OP3 VALUE3 or ZZZ OP4 VALUE4)

i. e. always all columns of a table are selected, we allow only two-level nesting of logical expression, the conditions on the bottom level must refer to the same column and are logically or-ed, the conditions on the upper level are logically and-ed.

5.1.3   Component interaction

Both producers and consumers advertise with a table name (besides attributes and capabilities) in the simplest case, or with a filter (which may be simplified, i. e. covering a larger result set). We assume the schema is well-known, i. e. all components can map a table name to the list of its columns and their types.

Data part of matching is done by looking up the potential party based on table name first. In the more complicated case when filters are advertised it is refined further by checking the union of filters (i. e. anding them) for logical contradiction. Due to the restricted structure of the filter formula the checking is trivial.

On claiming the producer and consumer must check the table name, and optionally evaluate the filters (which are fully specified now, unlike the matching phase when empty or simplified filters may be involved) for logical contradiction to avoid keeping unnecessary connection.

Data transfers send XML documents of the type XXX (Section).

5.2   Interface to Mercury

On the contrary to the artificial data language presented in the previous section, Mercury is an existing GMA implementation that defines its data language. In this section we demonstrate the ability of the C-GMA concept to wrap an existing GMA implementation. While data types are directly adopted from Mercury, implementation of queries using regular expressions is specific to our CGMA proposal.

5.2.1   Data types

The Mercury Monitoring System [Bal01] uses a simple language for defining the type of monitored data. The data model used by Mercury is based on metrics. Every sensor in Mercury can provide one or more metrics. Every metric has a unique name, a well-defined data type and a set of parameters. Every monitoring event generated by the producer belongs to a single metric.

Mercury has no distributed registry for the data schema therefore connected producer and consumer must agree on the definition of the schema. On the other hand consumers can query the schema from the producer run-time therefore dynamic schema changes are possible. Indeed, it is even possible for a consumer to simultaneously talk to several producers having different schemes.

Mercury installations are usually multi-level: every monitored machine runs a so-called Local Monitor while the cluster s front-end runs a Main Monitor. The Main Monitor accepts requests from consumers outside of the cluster and dispatches these requests to the individual Local Monitors and then routes the responses back to the consumer. The Main Monitor (or more specifically, the routing sensor inside the Main Monitor) uses the metric parameters to decide which Local Monitor is the request targeted at.

Therefore for Mercury, data provided by Mercury provider will be described by tuple:

Mercury also supports actuators components responsible for performing monitoring actions. Similarly to sensors, actuators are managed by producers and can be accessed using the interface provided by the producer Just like metrics, the single operations that can be requested from an actuator are defined by controls. The definition of a control has the same format as metric definitions and therefore can be represented in C-GMA the same way.

5.2.2   Queries, searching

Proposed query language in our CGMA proposal follows the data definition as it is implemented again as a tuple. For simplicity we only include the metric name, parameter values and data type allowing POSIX 1003.2 regular expression matching for filtering. Indeed, the producer location can be considered technical detail of the data layer, needed for building the communication channel but not strictly necessary for defining the query s semantics.

5.2.3   Example

Examples about data sources registered by producers:

  (skirit.ics.muni.cz; pbs.queuelen; queue=skirit; int)
  (skirit.ics.muni.cz; host.loadavg;
      host=skirit1.ics.muni.cz; double[3])
  (skirit.ics.muni.cz; host.loadavg;
      host=skirit2.ics.muni.cz; double[3])
  (n0.hpcc.sztaki.hu; app.cputime; jobid=gridjob001; double)

The above example shows the registration of two producers (skirit.ics.muni.cz and n0.hpcc.sztaki.hu). Information about individual hosts in the skirit cluster can be requested from skirit.ics.muni.cz. The consumers should not be aware how the requests are routed to the individual hosts. Similarly, n0.hpcc.sztaki.hu offers the monitoring of the CPU time consumed by the job identified by gridjob001; the consumer does not need to know on which internal nodes the job is really running.

The following example shows consumer-registered queries:

  (host.loadavg; host=skirit.*.ics.muni.cz, double[3])
  (app.cputime; jobid=gridjob001, .*)

The first example shows a consumer that is interested in load average information for all hosts in the skirit cluster. Since the expected data type is fully specified, a producer exporting the same data but in a different format will not match the query. On the contrary, the consumer emitting the second query is only interested in the name of the metric and is expected to deal with whatever data type the application actually registered for it.

5.2.4   Component interaction

Producers advertise the tuples with producer location, metric name, parameter values and data type. Consumers advertise query tuples. Since the schema is not defined globally and can also change dynamically, data types are handled as strings on the C-GMA level. For claiming and data transfer the native Mercury protocol is used between the consumer and the producer.

Allowing the consumer to use regular expressions in the type component of the query tuple makes it possible to adapt for dynamic schema changes. On the other hand, using a specific type in the query registration provides schema compatibility checking in the capability layer.

6   Related Work

6.1   Grid Monitoring Architecture (GMA)

The Grid Monitoring Architecture (GMA) [Tie01] is a general abstraction of the essential characteristics needed for scalable high performance monitoring on a large distributed computational Grid.

6.1.1   Overview

GMA proposal [Tie01] defines components producer, consumer and directory service. Besides component definition, proposal for interaction between components and required operations for each components were defined. Basic requirements on monitoring architecture were defined as low latency, capable of high data rate, minimal overhead, security and scalability.

In order to separate data discovery and data transfer, service called directory service was defined. Metadata describing producers of information and consumers waiting for some information are published in this service. Producers and consumers can then use published information to locate corresponding parties.

Several communication models between producer and consumer were identified: publish/subscribe, query/response, and notification. Operations supporting all types of communication were defined for producer and consumer. Basic operations for directory service were also identified. In all communication models, monitoring data and control messages occur directly between a particular consumer/producer pair and directory service is involved only in location of corresponding sink.

In GMA, monitoring data are sent in form of (time-stamped) events. Events are send only from producer to consumer, but communication can be initiated by both parties.

Compound components, which implement both producer and consumer interfaces are also studied.

Interoperability of different monitoring systems was one of motivations for GMA. Definition of GMA inspired several implementations of monitoring systems. While GMA definition proved to be general enough to cover all these implementations, it proved to be to much general to guarantee real interoperability between proposed architectures. GMA definition does not constraint communication protocols between components nor data model for description of producers and consumers.

6.1.2   R-GMA implementation

Relational Grid Monitoring Architecture (R-GMA) [Fis01] developed in the European Datagrid project is a relational implementation of GMA. R-GMA is planned to be used not only as monitoring infrastructure, but also as generic information service. The R-GMA system uses web servlets implemented in Java. Client-side APIs are available in Java, C, C++, Perl and Python. R-GMA is using a subset of the SQL language to describe both data and queries.

The current R-GMA implementation has several drawbacks - large overhead both in runtime and compilation and instalation. From developer standpoint, rapid development and changing nature of R-GMA is also problematic. R-GMA API is also much more complex, it contains more than 200 API calls [BG04], while jGMA or pyGMA APIs have 46 or 17 API calls.

6.1.3   Lightweight implementations

The jGMA [BG04] is an lightweight implementation of the GMA in Java. It contains very reduced API, no support for web-services or SSL/GSI authentication. Registry API allows to associate with registration an XML description of features and capabilities of registered entities. The jGMA support different communication models for local and wide-area networks. The jGMA implementation is available only as binary release.

The pyGMA is an lightweight implementation of the GMA in Python. It has web-services SOAP interfaces. Implementation contains only a framework, only sample producers or consumers are provided. Distribution contains simple central registry.

CODE [Smi02] is Java based implementation of GMA. CODE defines components directory service, observer (provides information), actor (which can be asked to perform actions) and manager. Manager asks observers for information, reasons upon that information, and asks actors take actions if needed. Implementation includes support for GSI. For implementation of directory service is used LDAP server. Several grid services were implemented using CODE, however the framework is currently not supported.

The Mercury Monitoring System [Bal01] developed in the GridLab, is C-based GMA implementation which is focused to resource and application (performance) monitoring. Mercury implements only producers and consumers, no directory service is used. Mercury has a modular infrastructure where individual sensors and actuators are implemented as loadable modules thus providing easy extensibility. Mercury uses a compact binary protocol to reduce network traffic and communication overhead.

Mercury also features a hierarchical design where lower level monitoring components are aggregated under higher level components. Mercury supports GSI authentication, data encryption and access control lists to address security concerns. Mercury is implemented using the C language but the producer and the consumer API are also available in Java.

Mercury describes data types in terms of metrics.

6.2   Information services

Related to problem of resource selection, information services are used for publishing, discovering and querying information about services. Two prominent examples are Globus MDS used in grid toolkit Globus and UDDI used in web-service community. Both information services provide basic query language.

UDDI is an XML based registry that provides framework for publishing and querying information about organizations and their web services and defines set of API s to interact with UDDI data registry. The registry contains entries describing services with abstract tModels, which are used to categorize web services into taxonomies. The UDDI is aimed at business cooperation and e-commerce services.

Globus MDS (MDS4) is a suite of web services used to discover and monitor services and resources on the grid. The foundation of MDS is the WSRF based Index service which implements the registry similar to UDDI. Clients use the standard WSRF query and subscription interface to obtain information stored in the Index service.

6.3   GGF standards

Certain emerging web-services standards correspond to the proposed communication model between mediator and producers and consumers. Particularly, WS-Notification draft fits our needs. Parts of WS-Resource (WSRF, especially WS-ResourceProperties (WSRF-RP) corresponds with our definition of capabilities.

OGSA-WG in GGF has plan to define OGSA Information and Monitoring Architecture. Architecture should be based on OGSA Architecture document [Fos05] and GMA proposal.

6.4   Publish-subscribe systems

In article [Cec03], distributed mediator using Content-based Publish-subscribe systems [Car01] was proposed. We plan to investigate other approaches that leverage recent research results regarding self-organizing latency-aware overlay topologies [CJ05], distributed hash tables [Sto01] and epidemic dissemination information protocols [Eug03].

6.5   Capability languages

In the context of Grids, different languages were studied for description of component capabilities - classified advertisement (ClassAd) [RLS98] for bi-lateral matching, set based extensions of ClassAds [Liu02] for specifying and solving set-matching problem, gang matching [Ram00] allowing the request to specify a list of bi-lateral matches, constraint based language Redline [LF03]. Article [LF03] can be used as survey for this research area.

6.6   Semantic web technologies

The term semantic web is used for the concept of enabling automated processing of machine understandable data on the web, allowing data to be shared and reused between independently developed applications. The RDF language is used to represent information about (web) resources, or more generally about identifiable things, in machine readable way. The RDF-Schema and OWL Web Ontology Language can be used to share set of terms and rules describing given knowledge domain, again intended for automated processing.

There are several proposals for means to describe (web) service properties. The WSDL S extension to WSDL allows annotation of web service interfaces in WSDL with references to external semantic descriptions in language of author s choice. The OWL S is an OWL ontology consisting of basic set of classes and properties for describing services. An OWL S description of service somehow overlaps with WSDL in that it covers input and output parameters of the service; in addition it also announces effects the service invocation has and pre /post conditions for service usage.

GGF group Grid Resource Allocation Agreement Protocol (GRAAP-WG) is working of WS-Agreement protocol, which supports the establishment and management of agreement-based service relationships for both a service provider and a service consumer.

7   Foreseen implementation and future work

Expected prototype implementation will consists of

Independently, peer-to-peer implementation of mediator and registry is studied in cooperation with INFN. Foreseen capability language is XML based, with reused data languages from first implementation.

Based on this proof-of-concept implementations, possible revision of defined operations may appear. Research may continue in several areas

WSRF and WS-N implementations exists for several languages, including Java, C, Perl, Python and C#. Comparison of several implementations can be found in [Hum].

ClassAd implementation is available for C/C++ and Java languages. There is also C based version of ClassAd Catalog, which implements several features needed for registry.

8   Conclusions

We described the C-GMA architecture which is designed to overcome the generality of GMA and contradicting requirements of various use case scenarios. We have overviewed the C-GMA components, their high level operations and interactions and we suggested possible implementation based on two different capability languages.

The work presented here is a snapshot of on-going research. Therefore many proposed solutions can be still subject to change. At the time of writing this report we work in parallel on a prototype implementation in order to prove the validity of concepts presented here. The prototype is based on a ClassAd capability language, and the mediator is implemented as a service (exposing web-service interfaces).

9   Acknowledgement

This work was done with the support of Ministry of Education of the Czech Republic, research programs MSM0021622419 and MSM6383917201.

10   Appendix A. C-GMA components - complete operations

In this section we list (in italics) all C-GMA specific component operations. Many of them have their counterpart in GMA and differ only in the communicating party (i. e. mediator instead of registry). The operation usually involves two components; in that case we specify corresponding operation of the passive component (i. e. the component whose interface is called). GMA operations are listed as well for completeness (in normal font).

10.1   Mediator operations

10.2   Producer operations

Optional functions for claiming protocol:

10.3   Consumer operations

Optional functions for claiming protocol:

11   Appendix B. ClassAd capability language - API

11.1   Data structures

component ID

URI, unique component identifier

endpoint ID

Unique identifier of C-GMA producer/consumer interface. Complete endpoint reference is tuple (component ID, endpoint ID).

address

URL of particular endpoint

capability

ClassAd which describes capabilities of the endpoint.

attributes

ClassAd which describes data attributes for all data produced by producer.

DDL hint

Expression in DDL, which describes produced or required data. Each DDL hint is label by DDL name, which defines which of supported DDLs is used. If data attributes are associated with data type, ClassAd which contains these attributes is attached.

11.2   Mediator operations

Add

First registration of component, used also when registration has to be changed.

Parameters: component ID, endpoint ID, address, capability, attributes, list of (DDL name, DDL hint, attributes)

Update

Prolong soft-state registration for complete component.

Parameters: component ID

Remove

Request for unregistration of endpoint.

Parameters: component ID, endpoint ID

When endpoint ID is empty, complete component registration is discarded.

MatchNotify

Called by mediator when new matching component is found. Message has incremental semantics.

Parameters: list of [component ID, endpoint ID, address, capability, DDL]

Search

Returns complete list of matching endpoints.

Parameters: component ID, endpoint ID

Returns: list of [component ID, endpoint ID, address, capability, DDL hint].

12   Appendix C. XML-based capability language infrastructure API

This appendix contains a tentative description of a C binding of the API outlined in Section. Currently it is not intended as a complete reference of a proposed implementation, it illustrates the overall approach only.

With the exception of the context manipulation (Section) the API is completely symmetric for each of the caller-side functions (e. g. cgma_registration), which are described here, there exists a callee-side callback which must implement the operation. The callbacks are invoked via cgma_receive_op and have exactly the same prototypes as the respective caller functions. Therefore they are not described explicitely. See Section for an example.

12.1   API Context

The API uses an opaque data type cgma_context_t to store various state information passed over among the API calls.

Synopsis

cgma_init_context(
    char *my_id                  // IN  unique ID of the calling component
    cgma_context_t *ctx          // OUT context to be initialised */
)

Description

Initialise a new context object. The calling component has to supply its identifier. It is the responsibility of the component to generate it in a globally unique way, i. e. it should encode e. g. hostname or IP and process id.

Synopsis

cgma_receive_op(
    cgma_context_t ctx           // IN  context
    struct timeval *timeout      // IN  maximal timeout
    cgma_callbacks_t *ops        // IN  callback functions

Description

Wait at most timeout for an incoming call of C-GMA operations served by this component (e. g. AcceptMatchNotify). The structure ops contains pointers to functions which are executed for each operation (see bellow).

12.2   Advertising

Synopsis

cgma_registration(
    cgma_context_t ctx           // IN  context to work with
    enum cgma_registration_op op // IN  operation to perform
    char *url                    // IN  URL (endpoint) of the component
                                 //     registered
    cgma_capattr_t *caps         // IN  capabilities of the component
    cgma_capattr_t *attrs        // IN  component-level attributes
                                 //     (producer only)
    char *data                   // IN  data description
    time_t *expire               // OUT registration expiration
)

Description

Registers, unregisters, or updates registration of this component with the mediator, depending on value of op. Capabilities and attributes are suitably represented XML documents as described in Section, e. g. using libxml types. Data description data are treated as an opaque object at this level, encoded in a printable string.

12.3   Matching

Synopsis

cgma_match_notify(
    cgma_context_t ctx           // IN  context to work with
    char *dest                   // IN  URL of the notified component
    char *party                  // IN  URL of the matching party
    cgma_capattr_t *caps         // IN  its capabilities
    cgma_capattr_t *attrs        // IN  its component-level attributes
    char *data                   // IN  its data description
)

Description

The function is called by the mediator whenever a potential match between a producer and a consumer is found. Typically, it is called twice, with producer and consumer in the role of the notified component, and registered capabilities, component-level data attributes, and data description of the other one.

Synopsis

cgma_search(
    cgma_context_t ctx           // IN  context to work with
    char ***parties              // OUT URLs matching parties
    cgma_capattr_t **caps        // OUT their capabilities
    cgma_capattr_t **attrs       // OUT their component-level attributes
    char ***data                 // OUT their data descriptions
)

Description

A synchronous variant of the matching. This function is called by a component (producer or consumer) and it queries the mediator for matching parties. List of their URLs is returned in the array parties, together with their corresponding capabilities, data attributes , and data descriptions.

12.4   Claiming

Synopsis

cgma_query_cap(
    cgma_context_t ctx           // IN  context to work with
    char *dest                   // IN  URL of the component to query
    cgma_capattr_t **caps        // OUT its capabilities
    cgma_capattr_t **attrs       // OUT its component-level attributes
    char **data                  // OUT its data description

Description

Queries a concrete component for its capabilities, data attributes, and data description.

12.5   Simple example

12.5.1   Producer

/* local table of consumers to which this producer is sending data */
struct { ... } *known_consumers;

main()
{
  cgma_context_t  ctx;
  cgma_init_context(&ctx);
  char *my_url;
  cgma_callbacks_t *my_caps;
  char *my_data;
  time_t expire;

  ... /* fill in my_XX with concrete values */

  pthread_create(...,callback_thread,ctx);

  /* attrs = NULL - we don't provide component-level attrs */
  cgma_registration(ctx,CGMA_REGISTER_PRODUCER,
                    my_url,my_caps,NULL,my_data,&expire);

  while (1) {
    /* do the real work, i.e. when there are data available,
       send them to consumers stored in known_consumers 
       using the data communication chanel specific for this
       kind of components
     */
    ...
  }
}

void callback_thread(void *arg)
{
  cgma_context_t  ctx = arg;
  cgma_callbacks_t ops;

  ops.match_notify = producer_match_notify_callback;
  ... /* other operations */

  while (1) {
    cgma_receive_op(ctx,NULL /* wait indefinitely */,&ops);
    ... /* handle returned errors */
  }
}

int producer_match_notify_callback(
  cgma_context_t ctx,
  char *dest,
  char *party,
  cgma_capattr_t caps,
  cgma_capattr_t attrs, /* unused, consumers have no attrs */
  char *data
)
{
/* Check whether this producer realy matches with the consumer suggested
   by mediator. We are a real producer implementation, therefore we 
   understand and can check the data description too.
*/
  if (check_match(ctx,caps,data)) {
    /* insert `party' (the consumer) into known_consumers,
       so that the main thread starts sending data to her
     */
  }

  return 0; /* or an error */
}

/* implementation is specific to this consumer */
int check_match(cgma_context_t ctx,cgma_capattr_t caps,char *data)
{
  ...
}

12.5.2   Consumer

The example is really trivial. It does not call cgma_receive_op at all, effectively thowing away mediator s notifications.

main()
{
  ... /* the same my_XX's, initialised appropriately */

  cgma_registration(ctx,CGMA_REGISTER_CONSUMER,
                    my_url,my_caps,NULL,my_data,&expire);

  while (1) {
     /* accept incoming data, via the specific data channel,
        from any producer and process them */
    ...
  }
}

12.5.3   Mediator

Unlike producer and consumer, this mediator implementation, despite being very trivial, is generic w. r. t. data layer, i. e. it satisfies the requirement of interoperability of various data-layer implementations.


/* trivial registry - lists of registered components */

struct party { 
  char *url;
  cgma_capattr_t *caps;
  cgma_capattr_t *attrs;
  char *data;
} *producers, *consumers;

main()
{
  cgma_context_t ctx;
  cgma_callbacks_t ops; 

  ops.registration = mediator_registration_callback;

  while (1) {
    /* everything is done in the callback */
    cgma_receive_op(ctx,NULL /* wait indefinitely */,&ops);
    ... /* handle returned errors */
  }
}

int mediator_registration_callback(
  cgma_context_t ctx,
  enum cgma_registration_op op,
  char *url,
  cgma_capattr_t *caps,
  cgma_capattr_t *attrs,
  char *data,
  time_t *expire
) 
{
  switch (op) {
    case CGMA_REGISTER_PRODUCER:
      ... /* insert url,caps,attrs,data into producers[] */

      for (i=0; consumers[i].url; i++) {
        if (check_match(ctx,caps,attrs,consumers[i].caps)) {
          cgma_match_notify(ctx,consumers[i].url,url,caps,attrs,data);
      }
      break;

    case CGMA_REGISTER_CONSUMER:
      ... /* insert url,caps,attrs,data into consumers[] */

      for (i=0; producers[i].url; i++) {
        /* consumer's don't advertise attributes either */
        if (check_match(ctx,producers[i].caps,NULL,caps)) {
          cgma_match_notify(ctx,producers[i].url,url,caps,NULL,data);
      }
      break;
    
  }

  *expire = 0xffffffff; /* Feb 7, 2106 :-) */
  return 0;
}

int check_match(
  cgma_context_t ctx,
  cgma_capattr_t *producer_caps,
  cgma_capattr_t *producer_attrs,
  cgma_capattr_t *consumer_caps
)
{
  /* 
     Form the <match> merged document, transform implicit requirements
     into explicit, and evaluate all <req> elements using XPath.

     We do not care about data in this example.
     Real (=effective) implementations should provide data-type specific
     plugins to be called here and do the data matching too.
   */
}

References

[Tie01] Tierney B. et al.: A Grid Monitoring Service Architecture. Global Grid Forum Performance Working Group, 2001.
[Smi02] Smith W.: A System for Monitoring and Management of Computational Grids. Proceedings of 2002 International Conference on Parallel Processing, 2002.
[Fis01] Fisher S.: Relational Model for Information and Monitoring. Technical Report GWD-Perf-7-1, Global Grid Forum, 2001.
[Bal01] Balaton Z. et al.: From Cluster Monitoring to Grid Monitoring Based on GRM. Proceedings 7th EuroPar2001 Parallel Processings, Manchester, UK. pp. 874-881. 2001.
[BG04] Baker M. A. and Grove M.: jGMA: A lightweight implementation of the Grid Monitoring Architecture. In UKUUG LISA/Winter Conference, February 2004.
[RLS98] Raman R., Livny M. and Solomon M.: Matchmaking: Distributed Resource Management for High Throughput Computing. Proceedings of Seventh IEEE International Symposium on High Performance Distributed Computing, July 28-31, Chicago, 1998.
[Hum] Humphrey M. et al.: State and Events for Web Services: A Comparison of Five WS-Resource Framework and WS-Notification Implementations. Presentation.
[Fos05] Foster I. et al.: The Open Grid Services Architecture, Version 1.0. GGF OGSA Working Group (OGSA-WG), 2005. Available online.
[Liu02] Liu C. et al. Design and Evaluation of a Resource Selection Framework. Proceedings Eleventh IEEE International Symposium on High Performance Distributed Computing, Edinburgh, Scotland, 2002.
[Ram00] Raman R.: Matchmaking Frameworks for Distributed Resource Management. Computer Science, University of Wisconsin, Madison, 2000.
[LF03] Liu C. and Foster I.: A Constraint Language Approach to Grid Resource Selection. Technical report TR-2003-07, University of Chicago, Department of Computer Science, Chicago, 2003.
[TDK03] Tangmunarunkit H., Decker S. and Kesselman C.: Ontology-Based Resource Matching in the Grid - The Grid Meets the Semantic Web. In International Semantic Web Conference 2003, pp. 706-721.
[CD99] Clark J. and DeRose S. (Editors): XML Path Language (XPath), Version 1.0. W3C Recommendation, 1999. Available online.
[MOV06] Marsh J., Orchard D. and Veillard D. (Editors): *XML Inclusions (XInclude) Version 1.0*, Second Edition. W3C Recommendation, 2006. Available online.
[Car01] Carzaniga A. et al., Design and evaluation of a wide-area event notification service. ACM Transactions on Computer Systems 19(3): 332-383, 2001.
[CJ05] Ceccanti A. and Jesi G.P.: Building latency-aware overlay topologies with QuickPeer. In Proc. of IEEE ICNS 2005.
[Sto01] Stoica I. et al., Chord: a scalable, peer-to-peer lookup protocol for Internet applications. In Proc. of ACM SIGCOMM 01, 2001.
[Eug03] Eugster P. T. et al.: Lightweight Probabilistic Broadcast. ACM Transactions on Computer Systems 21, 2003.
[Cec03] Ceccanti A. et al. Towards Scalable and Interoperable Grid Monitoring Infrastructure. Submited to CoreGrid integration workshop.
[Sit05] Sitera J. et al.: Capability and Attribute Based GRID Monitoring Architecture. Proceedings of Cracow Grid Workshop, Academic Computer Centre CYFRONET AGH, 2005, pp. 176 183.
[Kra06] Krajíček O. et al.: Capability languages in C-GMA. Proceedings of Cracow Grid Workshop 2006, Academic Computer Center CYFRONET AGH, 2006, pp. 131 138.

 

Footnotes:

  1. But supporting at least component-level attributes is encouraged to benefit from all C-GMA features.

další weby:fond rozvojemetacentrumCzechLightpřenosyvideoservereduroameduID.cz