Storytelling and Content Presentation with Virtual and Augmented Environments in a Museum Context

Storytelling and Content Presentation with Virtual and Augmented Environments in a Museum Context
of 10
All materials on our website are shared by users. If you have any questions about copyright issues, please report us to resolve them. We are always happy to assist you.
Similar Documents
  Storytelling and Content presentation with the Virtual Showcase in a museum context Steffi Beckhaus 1 , Florian Ledermann 2 , Oliver Bimber  3 1   Fraunhofer IMK, Germany, 2 Technische Universität Wien, Austria, 3 Bauhaus Universität Weimar, Germany Abstract This paper gives an overview of the Virtual Showcase as an augmented reality display system for museums. It explains about different hardware prototypes, interaction tools as well as several software techniques to utilize the features of the Virtual Showcase. It also presents two case studies, one from paleontology and one from archeology. Keywords:  Virtual Showcase, Digital Storytelling, Augmented Reality 1 Introduction Virtual and augmented environments can be utilized to enhance the experience of museum visitors. With Augmented Reality (AR), the extension of real objects with virtual content, additional layers of information and experiences can be added to otherwise static objects. As virtual environments (VEs), augmented environments (AEs) can address all human senses. The presented content can be synthetically complemented with visual, auditory, haptic, and even olfactory (smell) information. We believe that these new forms of presentation hold the prospect of enhancing the experience of museum visitors. Computer-generated content can be either predefined to tell a story, or it can be fully interactive, allowing the visitors to explore the content by themselves. This paper will give an overview of the usage of the Virtual Showcase (VS) in the museum context. We describe the Virtual Showcase as an innovative spatial Augmented Reality display that offers an imaginative and exciting way of accessing, presenting, and interacting with scientific and cultural content. We give an overview of different existing prototypes, their technical components, and software features in Section 2. In Section 3, we briefly describe storytelling possibilities with the VS and explain about technical methods to interact with the VS. We will then describe the recent applications of the Virtual Showcase to the field of digital story-telling for education and scientific visualization in Section 4. In particular we will describe two case-studies. The first presents the scientific findings of leading paleontolologists to a non-scientific audience. The second presents an archeological model of a gate ruin. The acceptance of this new presentation form by potential museum visitors, as well as the quality of the visual appearance of such mixed content was evaluated in a user study [Bimber et al., 2003]. Section 5 concludes the paper. 2 Virtual Showcase Technology The Virtual Showcase, first presented in [Bimber et al. 2001], is a new Augmented Reality (AR) display system. This museum display provides an imaginative method for accessing, presenting, and interacting with scientific and cultural content. Conceptually, the Virtual Showcase is compatible with conventional showcases. Real scientific and cultural artifacts are placed inside the Virtual Showcase allowing their augmentation with computer- generated 3D graphics and animations within the same 3D space.  As virtual representations and real artifacts share the same space, this provides new ways of merging and exploring real and virtual content. Solely virtual exhibits may also be displayed. The VS supports  up to four users looking at the Virtual Showcase from different sides watching the same content from different sides. This feature allows the collaborative exploration of artifacts shown in the Virtual Showcase. Three different kinds of prototypes yet exist: a pyramid shaped projection based setup, a cone shaped projection based setup, and a cost effective CRT setup. 2.1 The Pyramid and Cone Shaped Projection Based Virtual Showcase The Virtual Showcase consists of two main parts (the numbers in parentheses correspond to the numbers in Figure 1): a convex assembly of half silvered mirrors (1) and a graphics display (2). Two different mirror configurations can be used. The first prototype consists of four half silvered mirrors assembled as a truncated pyramid (see Figure 1a). The second prototype uses a single mirror sheet to form a truncated cone (see Figure 1b). We placed these mirror assemblies on top of a projection screen (2). Users can see real objects inside the showcase through the half-silvered mirrors merged with the graphics displayed on the projection screen. The showcases’ content is illuminated with a controllable light source (3) while presenting view-dependent stereoscopic graphics to the observer. These prototypes use standard shutter glasses (5) controlled by infrared emitters (4). Head tracking which lets the system render the augmented graphics from the correct perspective is accomplished using an electromagnetic tracking device (6). It is also possible to use a wireless infrared tracking device. Figure 1: Pyramid Shaped (a) and Cone Shaped (b) Virtual Showcase The pyramid-shaped prototype supports up to four viewers simultaneously looking at the showcase from four different sides. The cone-shaped prototype provides a seamless surround view onto the displayed artifact. 2.2 CRT based Virtual Showcase In contrast to the previous prototypes, the CRT variation of the VS utilizes up to four CRT monitors, instead of a single CRT projector [Bimber et al., 2003]. This reduces the cost of the entire display to a fraction of the cost of a suitable projector. In the projection based approaches, multiple users were supported by partitioning the screen area (as well as the image resolution) into five sections: one for each of the four possible users, and one for the image area that is located underneath the mirror optics. The monitor-based setup provides an UXGA resolution per screen and consequently increases the image resolution per user by a factor of five. The display panel of the monitors has been coated with a light directing foil. This foil directs the light exclusively from the monitor towards the mirror optics, which makes the stereoscopic images appear only inside the mirror optics – not on the monitor. Thus, observers are not distracted by the source images on their own, or on the opposite display anymore. Since the monitors can be tilted towards the mirrors by an arbitrary angle, it is possible to cover the entire height of the mirror assembly by graphical overlays. All technology, such as PCs, tracking system, monitors, video beamers and sound system have been integrated seamlessly into the frame of the display. They are not visible from the outside anymore.   Figure 2: CRT based Virtual Showcase Figure 2 shows the CRT based version of the Virtual Showcase with again the numbers in parentheses corresponding to the numbers in Figure 2: a pyramid shaped assembly of half silvered mirrors (1), one of up to four CRT monitors (2) whose image forms the virtual object as can be seen in the mirror (9), a light projector (3), standard shutter glasses with a tracker sensor (5), an electromagnetic tracking device (6). This prototype features the IR stylus described below with a camera (7) and the stylus in the user’s hand (8). Inside the prototype is storage for PCs (10). 2.3 Software and Computer Hardware The VS is driven by off-the-shelf PCs with conventional 3D graphics cards. Three different software solutions are currently available to drive the Virtual Showcase display. The MS Windows based VS player was explicitly designed for the use with the VS. It features all necessary techniques for the setup and rendering in the VS context, some of them described in this chapter. This software is able to drive two monitors simultaneously with an UXGA resolution. Consequently, only two PCs are required to drive a four-user configuration. Depending on the requirements, additional PCs may be needed to support a projector-based illumination. More complex, interactive presentations are created using our APRIL authoring framework [APRIL].  APRIL provides concepts for hardware-configuration, story authoring, animation and interaction in an  XML-based file format, easy to use also by non-experts in computer graphics or programming. APRIL uses the concept of state-engine driven stories to overcome the apparent contradiction of realizing complex, interactive presentations with simple, robust interaction tools in a museum setting. In addition, it is possible to use different interaction mappings, depending on the target audience and the setting of a presentation - stories can be run in a fully interactive mode, or a "demo mode" that runs without any user interaction.  Avango™ is a virtual environment development platform and a software framework for complex and distributed virtual environments. It was recently extended to include tools for non-linear storytelling in virtual environments and it offers support for many VR interaction tools like the IR stylus. To use its wide range of features, it is currently adapted to drive the Virtual Showcase. Avango is available for SGI Irix and Linux. 2.4 Presentation Features of the Virtual Showcase 2.4.1 Overlay of Real and Virtual Objects For the integration of real and virtual objects in the Virtual Showcase, the scanned geometric representation must be registered against its physical counterpart first. Doing so lets the Virtual Showcase compute the illumination and occlusion effects directly onto the real object’s surface. A simple mouse-based interface allows to interactively place the pre-modeled virtual objects on or inside the real object at their appropriate position. 2.4.2 Projector-based Illumination Simulating realistic occlusion and illumination effects between the real objects, e.g. physical bones in our Raptor example, and the virtual objects, e.g. soft tissues, is essential for the presentation (see Figure 4 (c,d,f)). A drawback of such displays is that achieving realistic occlusion effects between real and virtual components is difficult. To overcome this difficulty, the Virtual Showcase uses controllable video projectors instead of simple light bulbs [Bimber and Fröhlich, 2002, Bimber et al., 2003 b]. The video projectors create view-dependent lighting effects on the real objects surface. Generating shadows on the physical object exactly beneath the overlaid graphics, for example, lets virtual parts mutually occlude the underlying real surfaces. Having the real object’s depth information, on the other hand, makes it possible to cull the occluded graphics before they display. Thus, the real objects can occlude virtual components and vice versa as best visible in Figure 4 (f). This strongly enhances the interactive presentation’s realism. 2.4.3 Projector-based Augmentation By replacing a physical object—with its inherent color, texture, and material properties—with a neutral object – same in shape but lacking these properties - and projected imagery, projector-based augmentation can directly reproduce either the object’s srcinal or altered appearance. This approach effectively lifts the object’s visual properties into the video projector. Projector-based augmentation is ideal when the physical object is available for visualization, even if it forms a complex geometric shape. Multiple users can view the augmented object simultaneously, without using stereo glasses or head-mounted tracking devices. For a high-quality augmentation, the system computes an image of the srcinal object’s textured 3D graphics model from the projector’s viewpoint. When projected, the rendered image appears smoothly registered with the neutrally colored object, which changes its appearance accordingly [Raskar et al. 2001]. 3 Digital Storytelling and Interaction with the Virtual Showcase Presentations showing non static content can potentially explain more about an exhibit and are more interesting to the visitor than static exhibits. Interactive installations can make the experience of museums even more vivid and interesting. By engaging users into the presentation process they are more involved, their attention is fully caught, and they can learn more.  A presentation or story can for example be influenced on the timeline. This enables users to move forward and backward in the otherwise linear presentation to revise certain parts or modify the duration of parts of the presentation for a more thorough experience. Much more complex story structures are possible. For example in the alVRed Project [alVRed, 2002], non-linear stories for  virtual environments are generated. Event-based, hierarchical stories allow for any degree of complexity of the story. The drawback may be that for a clear presentation of museum content the story has to have a certain dramaturgical and content structure. To combine both, fully interactive parts of the story can be integrated into an otherwise linear or branching storyline. To influence the story, ways to interact with the story have to be provided. Developing interaction techniques and devices for the museum context that can be integrated into digital stories is a challenging task. This is even more difficult, if multiple users are involved. For multiple user scenarios, additionally it can be differentiated between a guided, individual and cooperative interaction [Bimber et al., 2003]. The interaction with a story can be guided by a dedicated user, while the other users observe the same outcome and continuation. An individual interaction allows each user to interact with its own variation of the same story. In this case, each user is completely independent of the others. A cooperative interaction allows each user to influence the continuation of the same story that is observed simultaneously by all users. In contrast to the other approaches, mechanisms that resolve conflicting interaction and continuation situations are required for this case. Note that the VS player software currently supports only linear timeline based stories. For this, only a mouse-based, story independent and guided interaction is offered to transform single virtual components. Avango™ provides a full VR framework which is capable of implementing hierarchical non-linear stories. This is used in large projection-based theatres for presenting stories to a large audience. 3.1 Interaction Tools To control an interactive story or installation, interaction tools have to be available to the visitor of a museum. These have to be adapted to the visitor structure. As they are open to the public and often are visited by children, young adults and school groups, displays and their interaction tools have to be easy to understand and robust enough to survive rough treatment. This means that the interaction with the presentation or story has to be either touch less or is done with simple, robust tools. Touch-less interaction is for example possible by using cameras and hand gesture recognition. For the latter, Arcade game controllers offer a wide range of interaction tools like trackballs, button,  joysticks and such (see Figure 3). It is easy to integrate those into the VS, as all of the prototypes feature a table high setting. This allows for the integration of such tools in user hand height. With tracked props – simple replicas of objects in the presentation which manipulate their virtual counterpart in position and orientation – intuitive interaction with objects can be provided. The meaning is gasped by recognizing the similarity between the virtual and the real object, providing that the virtual object acts according to the real objects (the props) behavior. This requires tracking (knowing the position and orientation) of the prop. Tracking of a prop or the user’s head can be done by using optical or electromagnetic tracking. Optical trackers do not require a connection of the prop with the VS. On the other hand they need cameras and visible markers to work properly. Electromagnetic trackers generate a strong electromagnetic field. The prop itself has to incorporate a sensor which is attached to the computer. Many VR systems work with tracked tools like the Stylus, a pen like device. For object selection, e.g. a ray can be shot into the virtual space. Or an object can be directly manipulated by moving the Stylus. If only rotational information (the orientation of the prop) is required, there are trackers available which work with sensors using gravitation. They do not need optical marks or an electromagnetic field to work. The IR stylus, shown in Figure 2, is an optical tracker which needs a camera but no markers. It is a laser pointer technique, where a laser beam emitted by the stylus is splitted and produces two infrared spots on the semi transparent mirror of the Virtual Showcase. These are evaluated to retrieve the position and orientation of the stylus [ Matveyev et al, 2003 ].
Similar documents
View more...
Related Search
We Need Your Support
Thank you for visiting our website and your interest in our free products and services. We are nonprofit website to share and download documents. To the running of this website, we need your help to support us.

Thanks to everyone for your continued support.

No, Thanks

We need your sign to support Project to invent "SMART AND CONTROLLABLE REFLECTIVE BALLOONS" to cover the Sun and Save Our Earth.

More details...

Sign Now!

We are very appreciated for your Prompt Action!