Saturday , June 23 2018

A Large Scale Distributed Virtual
Environment Architecture

Elfizar1,2, Mohd Sapiyan BABA3, Tutut HERAWAN1
1 Faculty of Computer Science and Information Technology,
University of Malaya,
Pantai Valley, Kuala Lumpur, 50603, Malaysia;
2 Information System Department, University of Riau,
Kampus Binawidya, Pekanbaru, 28293, Indonesia
3 Gulf University of Science and Technology,
Kuwait City, Kuwait, Kuwait

Abstract: Virtual Environment (VE) is a simulation application that is widely used for the development of computer generated synthetic environments. A Distributed VE (DVE) allows many users to access a VE concurrently from different locations. Most current DVEs are still using simulator-centric architecture that views VE operations as a set of homogenous simulators, each aggregating data structure and all the actors operating on the data structure. This architecture limits the number of users involved in the DVE. It reduces users’ experiences because the area of VE is restricted. Also, when the number of objects increases, the VE runs more slowly. Although other architectures such as Distributed Scene Graph and Sirikata, have become available, the simulator still manages many objects in the simulation. It also restricts the number of objects and users involved in the VE. This paper proposes a new architecture to enable large scale distributed virtual environment. A simulator separation method is developed based on objects consisting of one process for one object (1P1O). The 1P1O architecture has a core component that comprises several simulators. In order to maintain the object, each simulator has two engines: physics engine and scripts engine. To maintain the consistency of the simulation, we introduce Universe that stores all objects state generated by simulators. Universe is responsible to store the state updates and disseminate them to interested simulators. There are two aspects used to evaluate the scalability of the 1P1O model, i.e. the number of objects and the number of concurrent users involved. Parameters such as CPU usages and memory allocation are used to analyze and evaluate the performance of the model. Experiments are conducted with varying number of objects and users. Compared with current architecture, the 1P1O model scalability and performance are better than current existing models in P2P network. The experiment results also comply with the mathematical model of the simulator and universe.

Keywords: Distributed virtual environment, 1P1O model, Large scale DVE, Architecture.

>Full text
Elfizar1, Mohd Sapiyan BABA, Tutut HERAWAN, A Large Scale Distributed Virtual Environment Architecture, Studies in Informatics and Control, ISSN 1220-1766, vol. 24 (2), pp. 159-170, 2015.

  1. Introduction

A Virtual Environment (VE) imitates a certain real environment. It should make user feel as residing in the real environment. Hence, VE should meet some requirements occurred in the real world. To involve many users in a VE, Distributed Virtual Environment (DVE) is required. Many users in separated places can come together to collaborate in a VE. DVEs have many applications used in games, education, war simulation, medical simulation, etc.

Virtual world is one of the most popular applications of DVE. For instance, Second Life [1] is state of the art of virtual worlds. On May 2012, the world of Second Life was made up of thousands of regions, which if they are linked together will spread over 1,962.93 km2 of virtual lands [2]. The world consists of avatars, terrains, trees, buildings, and other objects.

DVEs may have a very large number of objects and users at a time and that can easily overload a fast network, and impose huge processing requirements at the server and client computers. As computing resources are limited, there are obvious problems that arise once the number of objects and users in a simulation reach a certain limit. If no special mechanisms are provided, one may expect a DVE to produce undesirable effects such as choppy rendering, and loss of interactivity.

This paper focuses on this scalability issue. Scaling a DVE depends on two aspects, i.e. scaling the number of concurrent users interacting with each other, or scaling the scene complexity (number of objects and the complexity of their behaviours and appearances).

Several methods have been generated to scale DVEs such as dividing simulation workload [3][4], using dynamic load balancing among servers [5], and creating alternative architectures [6][7][8][9]. Scaling the DVEs can be done at the server’s side (using cluster or cloud computing) or the client’s side (using peer-to-peer model). Unfortunately, those techniques are not enough for DVEs with huge number of objects and thousands of concurrent users. Some limitations still occur in the current DVEs. Increasing the number of objects and users decreases the performance of DVE.

This paper proposes a novel DVE architecture, called 1P1O model, to scale up the present DVEs. In the proposed architecture, each object in DVE is treated as a separated process: one process for one object (1P1O). This concept is inspired by the real world in which there are many objects composing the world.

They may be static objects and dynamic objects, and there are interactions among them. Each object has control to itself to determine what kind of properties and behaviours it should appear in the world.

This paper makes two research contributions. The first is 1P1O model, a novel DVE architecture that provides a large scale DVE. This model is unlike existing DVE architectures, where a simulator manages many objects. 1P1O breaks the DVE into three components: object simulators, universe, and Content Delivery Network (CDN). An object simulator just simulates an object. Since object simulator is an independent process, the workload-balancing problem can be solved. DVE researchers and developers can use this architecture to scale up their applications in order to accommodate many objects and users in the environment.

The second contribution is the mathematical model of 1P1O. It is able to simplify the architecture. By means of the mathematical model, we are able to determine the characteristics of the architecture including the model complexity so that we can compare the 1P1O model with others.

The rest of the paper is organized as follows. Section 2 presents the current approaches used by researches in scaling up the DVE. Section 3 explores the 1P1O model as a proposed method along with its mathematics model. The experimental results and discussion are described in Section 4. Finally, Section 5 gives some conclusive remarks and future work.


  1. SECOND LIFE, Second Life, from, 2014, accessed on 27-09-2014.
  2. SHEPHERD, T., Second Life Grid Survey – Region Database, from, 2012, accessed on 05-10-2012.
  3. FAROOQ, U., GLAUERT, J., Scalable and Consistent Virtual Worlds: An Extension to the Architecture of OpenSimulator, Proceedings of 2011 International Conference on Computer Networks and Information Technology, 2011, pp. 29-34.
  4. CARLINI, E., RICCI, L., COPPOLA, M., Flexible Load Distribution for Hybrid Distributed Virtual Environments, Future Generation Computer Systems, vol. 29, no. 6, 2013, pp. 1561-1572.
  5. LIU, H., BOWMAN, M., Scale Virtual Worlds Through Dynamic Load Balancing, Proceedings of IEEE/ACM Symposium on Distributed Simulation and Real Time Applications, 2010, pp. 43-52.
  6. WALDO, J., Scaling in Games and Virtual Worlds, Communications of ACM, vol. 51, no. 8, 2008, pp. 38-44.
  7. BYELOZYOROV, S., JOCHEM, R., PEGORARO, V., SLUSALLEK, P., From Real Cities to Virtual Worlds using an Open Modular Architecture, The Visual Computer, vol. 28, no. 1, 2012, pp. 1-13.
  8. LAKE, D., BOWMAN, M., LIU, H., Distributed Scene Graph to Enable Thousands of Interacting Users in a Virtual Environment, Proceedings of Annual Workshop on Network and System Support for Games, 2010, pp. 140-148.
  9. HORN, D., CHESLACK-POSTAVA, E., MISTREE, B.F.T., AZIM, T., TERRACE, J., FREEDMAN, M.J., LEVIS, P., To Infinity and Not Beyond: Scaling Communication in Virtual Worlds with Meru, Stanford Computer Science Technical Report, CSTR 2010-01, 2010.
  10. OPENSIMULATOR, Open Simulator, from, 2014, accessed on 15-07-2014.
  11. DEBEAUVAIS, T., VALADARES, A., LOPES, C.V., RCAT: A Scalable Architecture for Massively Multiuser Online Environments, from, 2013, accessed on 12-05-2013.
  12. WORLD OF WARCRAFT., World of Warcraft, from, 2014, accessed on 12-05-2014.
  13. NAJARAN, M.T., HU, S.Y., HUTCHINSON, N.C., SPEX: Scalable Spatial Publish/Subscribe for Distributed Virtual Worlds Without Borders, Proceedings of 5th ACM Multimedia Systems Conference, 2014, pp. 127-138.
  14. RICCI, L., CARLINI, E., Distributed Virtual Environments: From Client Server to Cloud and P2P Architectures, Proceedings of International Conference on High Performance Computing and Simulation (HPCS), 2012, pp. 8-7.
  15. OLANDA, R., PEREZ, M., ORDUNA, J.M., Hybrid P2P Schemes for Remote Terrain Interactive Visualization System, Future Generation Computer Systems, vol. 29, no.6, 2013, pp. 1522-1532.
  16. KIM, K.C., YEOM, I., LEE, J., HYMS: A Hybrid MMOG Server Architecture, IEICE Transactions on Information and Systems, vol. E87, 2004, pp. 2706-2713.
  17. ALMASHOR, M., KHALIL, I., TARI, Z., ZOMAYA, A.Y., Automatic and Autonomous Load Management in Peer-To-Peer Virtual Environments, IEEE Journal on Selected Areas in Communications, vol. 31, no. 9, 2013, pp. 310-324.
  18. CARLINI, E., COPPOLA, M., RICCI, L., Evaluating Compass Routing Based AOI-Cast by MOGs Mobility Models, Proceedings of SIMUTools’11, 2011, pp. 328-335.
  19. VAN DEN BOSSCHE, B., DE VLEESCHAUWER, B., VERDICKT, T., DE TURCK, F., DHOEDT, B., DEMEESTER, P., Autonomic Microcell Assignment in Massively Distributed Online Virtual Environments, Journal of Network and Computer Applications, vol. 32, no. 6, 2009, pp. 1242-1256.