hledat:

Network Collaborative Environment for Human Tissues 3D Modeling

CESNET technical report number 30/2007
also available in PDF, PostScript, and XML formats.

Přemysl Kršek, Michal Španěl, Miroslav Švub, Vít Štancl, Ondřej Šiler and Vítězslav Sára
7.12.2007

1   Abstract

This report deals with problematic of network based virtual collaborative environment to support clinical applications of 3D models of human tissues, created from CT/MR data. Designed system allows clinical realizations of 3D applications as a service to clinical workplaces, provided by specialized 3D laboratory, even over great distances. Necessary consultations, corrections and verifications, which have to be realised distantly, are the source of problems. This is supported by our system in the form of Virtual Collaborative Environment (VCE). This system is built upon three-layer client-server architecture. Our application is focused on 3D tissue modeling. Generally it can be used as a basis for other similar applications.

Keywords: network collaborative environment, 3D modeling, medicine, computer tomography, magnetic resonance

2   Introduction

Recent trend in medicine has been individual approach to patient treatment. One of possible ways to achieve this trend, is 3D geometry modeling of human tissues. Created models are used for the needs of individual planing, simulations and navigation of performed surgeries.

Input data for 3D tissue models are images obtained mostly by computer tomography (CT) or magnetic resonance (MR). CT/MR data are commonly available in standard digital format DICOM (Digital Imaging and Communications in Medicine). The data are commonly stored in PACS systems (Picture Archiving and Communication System). These systems offer long-term archivation of stored data. When data are needed, these systems allow distant access to stored data and their transmission to another workplace via computer network.

Many various computer systems for 3D tissue modeling and support of clinical applications exist. Most of them are running locally at one particular workplace. Process of preparing clinical applications with support of 3D tissue modeling consists of following steps :

• obtaining CT/MR data from PACS system

• tissue segmentation and 3D modeling

• preparation of clinical applications (planning, simulation etc.)

• consultation, correction and verification of results

Everything needed for realization of particular clinical application is in one place: CT/MR data in PACS system, all concerned specialists (radiologist, surgeon, technician), patient etc. Therefore, there are no technical or organisation difficulties in communication, data transmission, people, etc.

However, all of conditions for realization of 3D applications are not to be found in one place, frequently. Only couple of big hospitals can afford to operate their own workplace that is technically and personally capable of handling whole process of 3D application preparation. In rest of clinical workplaces, 3D applications are not used due to technical complexity, financial or personal difficulties.

One of possible solution to this problem is preparation of 3D applications by independent specialized workplace (3D lab). This lab would offer services to other clinical workplaces based on supplied CT/MR data. Problem arises when it is needed to consult, correct and verify prepared models. If 3D lab, clinical workplace and consultants are distant from each other, this concept of independent 3D lab is very complicated.

This report deals with problematic of Virtual Collaborative Environment (VCE) suited for 3D modeling of human tissues based on CT/MR data. Basically it is an tool that would allow consultations, corrections and verifications in virtual environment integrated by computer network. Participating specialists (radiologist, surgeon, technician etc.) would be able to cooperate even if in different buildings, cities or regions. This tool would make the concept of 3D lab realizing clinical application feasible. Current PACS systems can not offer this kind of services.

3   Related Work

Recent five years we have been intensively participating on research and development in the field of 3D human tissue modeling and clinical applications. Recently we have been cooperating with several hospitals in Czech Republic (mostly Saint Ann Faculty Hospital of Brno, Faculty hospital of Olomouc, Faculty hospital of Ostrava) [Cer05] [Kru04]. We have come across particular problems with distant communication whenever we need consultations and corrections of prepared 3D applications. When our cooperations were limited to local hospitals, it was no problem for us to gather at one computer desk. That is of course not possible for longer distance cooperation. Up to now, we have solved this communication problem offline. Data were passed through network via SFTP servers or PACS systems. Resulting iterative process of consultations, corrections and verifications was lengthy and not intuitive.

For 3D modeling of tissues based on CT/MR data we have developed our own graphical user environment MediTools (Figure). This system works locally and provides full 3D visualization of CT/MR data. (volume rendering, multiplanar cross sections, etc.) and user support of tissue 3D modeling. 3D visualization is based on C++ library OpenInventor. Data core and algorithms for volume data processing is provided by MDSTk library [SK06].

To solve communication problems with consultations and corrections of prepared 3D applications between distant workplaces we have implemented and tested two architectures, Peer-to-Peer and Client-Server.

3.1   Peer-to-Peer Architecture

3D visualization in MediTools system is based on C++ library OpenInventor. This we have used together with OpenVCE library [Pec06] and it allows us to automatically synchronize state of whole OpenInventor based 3D scene. The OpenVCE library implements network Peer-to-Peer communication based on TCP/IP via SSL sockets. Extending our current MediTools system with communications in VCE framework is relatively simple. It comprises linking OpenVCE library and ensuring synchronization of additional data, which are not part of 3D OpenInventor scene.

VCE realized this way on local network was running with no serious problems up to five concurrently connected users. When more users were connected or communication was realized outside local network, network delays were noticeable (synchronisation is broken).

Due to character of peer-to-peer communication [Pec06], initial CT/MR data synchronization in VCE was very problematical. This was because the system does not comprise central data storage and providing access to data to everybody. Individual changes (offline) of users were local and had to be initially distributed to all participants. This scheme was vulnerable to conflicts in different versions of data being processed.

The most serious problem was however communication between 3D lab and clinical workplaces, usually part of hospitals. IT security policy of these hospitals is generally "paranoid" so it maximally tries to limit whatever nonstandard communication outside these. Our effort to run Peer-to-Peer communication outside hospitals over SSL sockets was strictly rejected. Only acceptable way was standard HTTP/HTTPS communication with secure and certified server outside hospital.

It is for these reasons we have stop development and tests of the Peer-to-Peer architecture and we have switched on Client-Server architecture.

4   Client-Server Architecture

Next step of out work was three-layer Client-Server architecture for further VCE design and development (Figure). VCE clients will communicate through central server using HTTPS protocol running on default port 80. This will take out the main problem of IT security policy of hospitals. It is also not a problem to detach VCE communication by redirecting the connection to private reserved port, via reserved channel or VPN connection, etc.

4.1   Concept of Reduced VCE

The biggest general problem of collaborative systems is delay of computer network. Due to the fact, that we need to communicate over generic internet network, no connection parameters are guaranteed. It could affect the fluency of interactive work with the VCE system. This problem we have solved by weakening the requirements on VCE.

Fortunately our application is very specific. We do not need to explore virtual world that is shared by number of clients. We therefore do not need to solve conflicts of time continuity of generated changes, so that clients could "argue" over a particular object in 3D scene (changing of view, changing the position of objects, changing the data, etc.). For that reason we have introduced a rule, which would ensure, that only one client at time (master) can generate changes to VCE. Other clients (slaves) are being notified after last synchronization. Master/Slave role is to be passed by token request system and is changed in time by users needs.

4.2   Central Server

Central server is built on standard three-layer architecture. HTTPS communication is provided by standard WWW server. Application layer consists of group of small specialized modules, implemented according to the needs of performance in PHP (database, storage) or C++/FastCGI (VCE). Development of whole system can be therefore focused on VCE solution itself. For remaining parts of server solution (database, storage, network communication, HTTPS protocol etc.), commonly used, debugged and secure products (Apache, MySQL, PHP, Linux, HTTPS encrypting etc.) are going to be used. In case where there is a demand to deploy the whole system in security and reliability critical conditions, it is no problem to switch to certified solutions of reputable firms (Sun, IBM, Oracle etc.).

Thanks to the integration of patient database and CT/MR data repository on central server, the problem of synchronization, archivation and version administration is automatically solved. With central server, it is possible to work locally. Client will be able to download current data of selected patient, do the whole process of 3D application preparation off-line and upload results afterwards onto server.

4.3   VCE Server

All of the source CT/MR data and prepared 3D applications are stored on server. At the moment when it is needed to consult or verify them, it is enough to create a VCE instance (start VCE session) on server and upload the data of selected patient (most significant synchronised copy of the data). Involved clients can then connect to the running VCE instance and work with the data together: viewing and modifying, showing areas of interest to each other, etc.

VCE on central server is represented by several running processes, which administrate their own copy of data currently being processed and queues of incoming and outcoming requests. All valid incoming requests for change in current VCE data are immediately replicated into working copy on server, which represent the current state of VCE. All connected clients are synchronized according to this VCE state. Normally, clients themselves synchronize the state of their local data copies according to requests of changes and ordered according to how they are stored in queues on the server. If for any reason a delay of one client occur, for example server timeout, this client downloads current state of data from VCE working copy on the server.

Thanks to multitasking character of working with WWW server, VCE itself multitasks. With one VCE, more clients can safely communicate with. There can be also several VCEs running concurrently. As requests number and number of connected clients rise, or number of concurrently running VCEs, it is not a problem to scale-up the capacity or performance, thanks to its modular architecture.

Larger volumes of transmitted data occur only at the beginning, when clients are logging to VCE and synchronizing initial data (100-500 MB). Largest dataflow is due to CT/MR data transmission, which are for our applications read-only. Data peak involved in working with VCE can be effectively lowered by caching data already read to a local hard-drive with identification with their hash code. When working with VCE itself, amount of transmitted data is theoretically relatively small in order of 1-50 kB/s, depend on number of clients, because only changes are synchronised.

4.4   VCE Client

Our MediTools system is to be used as graphic client (Figure). It is basically a "fat" client, which is in possession of its copy of processed data. Clients themselves ensure visualization, interaction, changes in data etc. The only thing server ensures is synchronization of data and parameters of their visualization. Connection to server is invoked by client, which passes its requests and awaits answer by HTTPS protocol. Clients periodically post requests on whether there are new changes in data state. If so, they perform synchronization and data replication into their own copy. According to requests, multiple versions of VCE clients can exist. Simpler versions will go as far as visualizing current data state, higher versions will allow generating of changes in data.

Concurrently to working with VCE, it is useful to utilize other means of communication as VOIP, NetMeeting etc. Rights to access the system are administered by WWW server. Access to stored patient data and their CT/MR data is administered on SQL database level.

4.5   Data Coding

For data transmission between WWW server and clients, standard HTTPS protocol is used. Encoding of data is based on our own system, which we call GELD (Group Element Length Data). This encoding method is inspired by standard communication format DICOM, in implicit version.

GELD encoding allows us to transmit binary or text data. Thus, we are able to send data and instructions divided into logical blocks. Each data or instruction block is uniquely identified by group ID (Group, 2 Byte) and element ID (Element, 2 Byte). Then comes length of transmitted block (Length, 4 Byte) followed by transmitted block itself.

4.6   Implementation

Actual version of graphical clients of MediTools system has been developed in C++ and for visualization, it utilizes graphical C++ library OpenSceneGraph. Communication server is based on Apache WWW server, patients database is MySQL. Application server is assembled according to performance requirements from PHP or C++/FastCGI scripts. On client side is used cURL library for HTTP/HTTPS communication.

Currently we have the main infrastructure of whole system implemented (Figure and Figure): HTTP/HTTPS communication library with C++ interface for clients, VCE application modules including unified binary and text data coding; PHP application modules for communication with client database and their CT/MR data; basic VCE functionality involving parameter of 3D scene visualization synchronization (camera movement, multiplanar slices movement, density parameters of visualization of CT/MR data etc.)

4.7   Testing

Development and debugging-purpose testing was done over local network at FIT VUT in Brno. In order to test system properties, communication between FIT VUT in Brno and Faculty hospital of st. Anna in Brno (imaging methods clinic), Faculty hospital of Olomouc ( platic and aesthetic surgery clinic) and Faculty hospital of Ostrava (clinic of neurosurgery) was tested.

During the tests, 1 to 25 clients were connected simultaneously. Single VCE server instance was running and it served all clients that were connected.

Clients were periodically asking for server's internal data status with minimum response time of 100 ms (reflecting network response time). One of clients was acting as master (who was periodically updating its status on server) and the rest of clients were in slave mode (requesting updates from server according to their last change).

Our server was running on following configuration: 2x Intel Xeon 2.1 GHz (dual core), altogether with 4 GB RAM, Raid 5 disc array 4,3 TB. Network connection is to a Gigabit backbone. It is located at FIT BUT.

4.8   Results

Client update request interval was set to (100 ms), which was sufficient for fully interactive communication on reduced VCE. Overall response times show, that depending on used connection medium, network's response time does not significantly affect functionality of our system (Figure).

Internal response of VCE server itself was also low (Figure and Figure). Increase of response time when 20 clients were connected,was caused by server configuration (number of service threads). There is still space for server performance improvement, whether it is software based improvement (configuration, optimization), or hardware based improvement (virtualization, grids).

Common client data flows were around 100 B/s - 1 kB/s. On server side, dataflows were between 1-50 kB/s, according to number of connected clients (1-25 clients). Considering todays network capacities, these values are virtually insignificant.

Dataflow issues, whether on server or client side, begin to emerge during initial CT/MR data synchronization. Average sizes of this data is in hundreds of MB. In order to minimize this effect, we are planing to include progressive compression (gradual loading of increasing level of detail data) and temporal data caching on client side.

5   Conclusions

Based on wide practical experiences with clinical applications of 3D modeling in medicine, we have designed computer system for Virtual collaborative environment. This system allows to do consultations, corrections and verifications of prepared 3D clinical applications through generic computer network. Its main concept is maximum usage of current technologies (HTTPS, PHP, FastCGI, MySQL, etc.) in the form of three-layer architecture.

Designed system is generic enough to be used in many other fields, where it is necessary to work within VCE. Up to today, we have done the implementation of basic functionality of designed system (data transfer, synchronisation and collaboration) and of it application in area of 3D geometrical modeling of human tissues (CT/MR data archiving, visualization, consultation and synchronization).

To get rid of the problem of network delay, we have limited our VCE system to allow data change from only one client at time, which is suitable for this type of cooperation. Even when communicating with couple of clients across whole Czech Republic, the delay has not had significant affect on the fluency of work with VCE. Data traffic is also at very low rate, with one exception: the moment when initial data synchronization is being done.

5.1   Future Work

• Extend VCE functionality (interactive 3D tissues modeling etc.)

• Digital signature on source CT/MR data

• Correction changes workflow including digital signature

• Progressive compression of source CT/MR data

• Local temporary saving of source CT/MR data on client

6   Acknowledgments

The work is supported by grant project number MSM6383917201 CZ and MSM0021630528 CZ.

References

[Cer05]	Černochová P. et al.: Application of geometric biomodels for autotransplantation of impacted canines, In: World Journal of Orthodontics 2005, Paris, FR, 2005, p. 1, ISBN 1530-5678
[Kru04]	Krupa P. et al: 3-D real modelling and CT biomodels pplication in facial surgery, In: Neuroradiology, Berlin, DE, Springer, 2004, p. 1, ISBN 0028-3940
[SK06]	Španěl M., Kršek P.: Vector-based Medical Image Segmentation using Adaptive Delaunay Triangulation, In: Proceedings of the Sixth IASTED International Conference on Visualization, Imaging, and Image Procesing, Palma de Mallorca, ES, ACTA Press, 2006, p. 6, ISBN 0-88986-600-7
[Pec06]	Pečiva J.: Active Transaction Approach for Collaborative Virtual Environments, In: ACM International Conference on Virtual Reality Continuum and its Applications (VRCIA), Chinese University of Hong Kong, HK, ACM, 2006, p. 171-178, ISBN 1-59593-324-7