Courseware Implementation in Open Source Portal

CESNET technical report 2/2011

Josef Krupička

University of West Bohemia

Received

Other formats: PDF, EPUB

Abstract

This report describes the implementation of our custom Java portal, scripting interface for managing portal structure and integration of Courseware.

Keywords: JSR 168, Gatein, portal, Courseware, XMLAccess, IBM WebSphere Portal, Vaadin

1  Courseware at the University of West Bohemia

In the past it was very difficult to find information about studied courses. Teachers used personal websites, departmental websites or other solutions. Web pages had different look and feel and structure of provided information. The main objective of Courseware is to provide a uniform interface for all courses at the university. Main features of Courseware at UWB:

Main goals of our project are:

Our Courseware is highly coupled with the IBM WebSphere portal, namely with its XMLAccess scripting interface. XMLAccess is a tool which enables export (import) everything from portal installation to (from) XML file. We use it for creating (setup of predefined portal pages and port lets) and publishing (move from private space to public) courses, batch changes of permissions and portletů (migration to new academic year). We use some proprietary APIs in our applications too, but it is not so crucial as XMLAccess. Desired open source portal must have some kind of scripting interface.

2  Searching for suitable open source portal

After six years of experience with portals (IBM WebSphere portal, Apache Jetspeed, our simple portal based on JBoss Portlet Container) we have a clear idea of “dream-like” portal, which we would like to operate and recommend to our customers (universities without decent J2EE administrators). Our desired portal should have following features:

We wanted to use some existing open source portal because we thought that after more than eight years of Java portal business, they should provide same features as WebSphere. We chose and analyzed three most spreaded open source portals.

This is currently most used open source portal with lot of nice features. User interface and available portlets surpasses IBM’s WebSphere portal. In addition, there is a huge community around Liferay that is also formed by commercial firms. Unfortunately, there is no scripting interface and core team isn’t planning to implement this feature in near future. Liferay in current state is big portal solution like Webpshere and it isn’t easy to install, configure and maintain. We want lightweight solution with simple administration and customization. Liferay is a “beast” with lot of features but our customers would use only small percent of them.

We used this portal in version 2.0 for our outsourcing portals. We were not too satisfied with this release because it contained lots of errors in the administration interface. It also had cumbersome deployment of new applications. Version 2.1 contains many enhancements, but it also contains same errors as the previous version. Also it had no scripting interface, and therefore we rejected this portal.

Gatein is a combination of JBoss Portal and eXo Portal. This version of the portal is already mature platform. But unfortunately its community is small, and even the core development team is small and not very active. Scripting interface is not available and the authors are not planning anything like that in future releases. Therefore, this portal has been also rejected.

After quick analysis of these portals we had no winner. Our desired portal must have scripting interface. Management of courses in Courseware will be impossible without this feature. We have considered the implementation of a scripting interface for Liferay, but it would be very time consuming for the scope of the project. We didn’t have enough experience with Liferay architecture and inner mechanisms, and it would take lot of time to master it. Finally we decided to create our own lightweight portal with desired features.

3  Creating custom portal

Every portal is based on portlet container which is runtime for portlet applications. Our portal uses Gatein portlet container, which is compliant with JSR 168 and 286. This product is distributed under LGPL license. It is reliable solution (Gatein portal uses it too). We considered Apache Pluto, because it is reference implementation of portlet standards, but Gatein is easier to integrate and provides better API. On top of this we developed portal layer which provides common features like:

3.1  Core components

[Image]

Figure 1. Block diagram of core components.

3.1.1  Portal database

Our portal needs repository for storing:

We use Oracle Express edition, but it is possible to use other decent RDBMS solutions (Postgresql, DB2, MSSQL,…).

3.1.2  Portal caches

Database is used only as persistent storage. Everything is stored in thread safe portal caches (we use components from java.util.concurrent). This ensures very good performance and throughput of our portal. Also memory consumption of these caches is very low and they are loaded during portal startup.

3.1.3  Authentication and authorization

Our portal provides pluggable authentication providers. Out of box it can authenticate against portal database, which contains table with user’s login and optional passwords. There is authentication interface so it is possible to implement own module. We provide IS/STAG and LDAP (usable with Active directory) authentication module.

All users, groups and memberships must be stored in portal database. There is registry interface which is responsible for periodic synchronization of portal database and external user registry. There are two default registries for synchronization with IS/STAG and LDAP. There are two default groups for guest users and authenticated users.

3.1.4  Permissions

There are three types of permissions which can be set for node:

Every node has attributes to:

Permissions are cached and it is very fast (quick lookups in cache) to compute them for current node and user. If user can’t view desired node, he is redirected to portal home page. Administrator can also set which principals can view and edit portlets. There is XML configuration file for this which is loaded during portal startup.

3.1.5  GUI interface

We want to implement easy to use GUI. After bad experience with slow and not very user friendly GUI of WebSphere. GUI is created in Vaadin framework. It is a Java framework for building rich internet applications which is built on top of Google Web Toolkit. We can recommend it as a very productive tool for creating web apps. You don’t need to mess with HTML, CSS and JavaScript. Everything is in plain Java and it is similar to development of desktop applications with Java Swing. I can compare it with Adobe Flex, which is another solution for creating rich internet applications, and Vaadin is much better for Java programmers.

[Image]

Figure 2. Screenshot of the graphical user interface.

Available features:

We plan to enable managing of portlet containers and portlets without entering admin GUI. It is more convenient for normal users (editors) to directly drag & drop portlets while they are browsing portal. This feature is available in other portals and with good JavaScript library it won’t be hard work.

3.1.6  Scripting interface

This is the most important component for us. We like XMLAccess and it is very powerful tool. We need only subset of its features (managing nodes, portlets and permissions). We have lot of Courseware administrator scripts for manipulating courses in portal:

These scripts generate XML files and run XMLAccess tool. It is very desirable to reuse their output and run it in our portal. So we developed following two components.

This is J2EE servlet which takes XML from HTTP POST request and makes desired changes in portal. XML provides only subset (only for managing nodes, no applications and user management) of XMLAccess features. Currently it supports following functions:

We create converter which takes XMLAccess file and transform it to our simpler XML format. It removes some useless attributes and simplifies container model. Our portal doesn’t have nested portlet containers like WebSphere. 99% percent of pages have one or two column layout. Also possibility to create nested containers results in some very complicated layouts and our users are confused with this feature. So we decide to implement simple container model and converter which is able to map nested containers in this model.

<request>
  <portal>
    <content-node action="locate"
                  objectid="6_0G06RA3CVD061JCD5G40000000"
                  uniquename="PREDMETY-KIV"/>
    <content-node action="update"
                  objectid="6_0G06RA3CVD061JCD54L0000000"
                  name="PPA1" uniquename="PREDMETY-KIV/PPA1"
                  ordinal="0" type="folder"
                  content-parentref="6_0G06RA3CVD061JCD5G40000000"
                  export-descendants="false">
      <localedata>
        <title>PPA1</title>
      </localedata>
      <localedata>
        <title>Počítače a programování 1</title>
      </localedata>
      <access-control>
        <role actionset="User" update="set">
          <mapping subjectid="all authenticated portal users"
                   subjecttype="group" update="set"/>
        </role>
        <role actionset="Editor" update="set">
          <mapping subjectid="herout"
                   subjecttype="user" update="set"/>
        </role>
        <role actionset="Administrator" update="set">
          <mapping subjectid="brada"
                   subjecttype="user" update="set"/>
        </role>
      </access-control>
    </content-node>
    <content-node action="update"
                  objectid="6_0G06RA3CVD061JCD64L0000000"
                  name="O předmětu"
                  uniquename="PREDMETY-KIV/PPA1/O předmětu"
                  ordinal="1" type="folder"
                  content-parentref="6_0G06RA3CVD061JCD54L0000000"
                  export-descendants="false">
      <localedata>
        <title>O předmětu</title>
      </localedata>
      <access-control>
        <role actionset="User" update="set">
          <mapping subjectid="anonymous portal user"
                   subjecttype="user" update="set"/>
        </role>
      </access-control>
    </content-node>
    <content-node action="update"
                  objectid="6_0G06RA3CVD061JCD84L0000000"
                  name="Přednášky"
                  uniquename="PREDMETY-KIV/PPA1/Přednášky"
                  ordinal="3" type="ppage"
                  content-parentref="6_0G06RA3CVD061JCD54L0000000"
                  export-descendants="false">
      <localedata>
        <title>Přednášky</title>
      </localedata>
      <access-control/>
      <component action="update"
                 objectid="7_0G06RA3CVD061JCDV0H2000000"
                 ordinal="100" orientation="V"
                 type="layout">
        <portletinstance action="update"
                         objectid="5_0G06RA3CVD061JCD2NB1000000"
                         portletref="Courseware;;ESFPortlet">
          <preferences update="set" name="ESFKatdraPref">
            <value>KIV</value>
          </preferences>
        </portletinstance>
        <portletinstance action="update"
                         objectid="5_7OQG3PD208O1602L4K3N6S10S3"
                         portletref="Courseware;;PredmetInfoPortlet">
          <preferences update="set" name="PredmetInfo_PPRokVarianty">
            <value>2011</value>
          </preferences>
        </portletinstance>
      </component>
      <component action="update"
                 objectid="7_0G06RA3CVD061JCD10Q3000000"
                 ordinal="200" orientation="V"
                 type="layout">
        <portletinstance action="update"
                         objectid="5_KLE8OC84085010IQG44SN30430"
                         portletref="Courseware;;UcitelVizitkaPortlet">
          <preferences update="set" name="UcitelVizitka_PP_Login">
            <value>netrvalo</value>
          </preferences>
        </portletinstance>
        <portletinstance action="update"
                         objectid="5_0G06RA3CVD061JCDCSU0000000"
                         portletref="Courseware;;TextPortlet">
          <preferences update="set" name="static_text_id">
            <value>6224</value>
          </preferences>
        </portletinstance>
        <portletinstance action="update"
                         objectid="5_KLE8OC840OK9F0IALKE46U3000"
                         portletref="Courseware;;TextPortlet">
          <preferences update="set" name="static_text_id">
            <value>46551</value>
          </preferences>
        </portletinstance>
      </component>
    </content-node>
  </portal>
</request>

3.1.7  Templating system

Templates are responsible for creating page header, footer, navigation menus and inserting generated markup from portlets in appropriate containers. Every node in portal can have own template as its node property. Children inherits template setting. Core portal servlet reads this property and calls template default JSP file.

Template is directory in portal server with following structure:

3.1.8  Spring framework

Our portal layer uses Spring dependency injection container. It glues all core components together and enables easy customization for current portal deployment. For example it enables choosing desired authentication and authorization providers. Everything is configured in Spring XML files. Portlet applications running in portal can access core services and use them.

4  Testing custom portal

We had time to make only simple load testing. For testing purposes we created portal database with 20k nodes, 100k portlets, 20k users, 20 groups, 30k memberships and 5k permissions. Portal caches took only 30 megabytes of memory. We used the Apache benchmarking toolab for load generation. There was a complex portlet page with several portlet containers and portlets from Courseware and IS/STAG . These portlets are using caching and they don’t generate database load. These are results from ab:

Concurrency Level:      50
Time taken for tests:   72.151 seconds
Complete requests:      10000
Failed requests:        0
Write errors:           0
Total transferred:      65644010 bytes
HTML transferred:       63231600 bytes
Requests per second:    138.60 [#/sec] (mean)
Time per request:       360.754 [ms] (mean)
Time per request:       7.215 [ms] (mean, across all concurrent requests)
Transfer rate:          888.49 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       28  262 227.6    278    3494
Processing:     1   98  88.5     65     387
Waiting:        1   66  63.4     36     366
Total:         29  360 220.4    344    3510

Percentage of the requests served within a certain time (ms)
  50%    344
  66%    352
  75%    361
  80%    367
  90%    399
  95%    485
  98%    612
  99%    677
 100%   3510 (longest request)

These are not bad numbers. Most of CPU was used by Apache server (maybe some problem in configuration). Activity of garbage collector was normal and Java memory heap didn’t grow.

There is plan to create simple workload tool which will login in portal and randomly visits pages and wait for random time before next “click”. This test should be more accurate than simple ab use.

Portal startup speed depends on two factors:

Scripting interface was tested on some real word XML files (import of selected big courses) from our portal and it worked quite well. It will be possible to use this scripting interface with some minor tweaks and we think about migration from WebSphere to this solution in next years.

5  Conclusion

Practical result of this project is our own portal based on reliable Gatein portlet container. We are going to distribute this portal to our customers (VFU, UJEP, JCU, UPCE and 4 smaller private colleges). It can be customized for different customer’s requests. This solution is also ready to host our Courseware and customers can ask for installation and configuration of this system. There is also possibility to integrate Courseware with Grouper and Moodle system, which is widespread among universities. It can also be used as content management system. Our customers will get simple to use portal which is easy to administer and customize. We want to publish portal to GitHub or SourceForge after some testing period (about half a year) and we also need to find some good name for it. We think that this portal can be lightweight alternative to other complex solutions from open source world.

další weby:fond rozvojemetacentrumCzechLightpřenosyvideoservereduroameduID.cz