Navigate to homepage

E-learning: Beyond the Graphical User Interface

Back

E-learning: Beyond the Graphical User Interface

Research proposal for a mixed reality approach to E-learning

Context

From January to April 2018, I worked on a research proposal that aims to cover a future interface topic or concept. My choice of focus was on how mixed reality might come into play to support the learning experience by exploring learning theories (i.e., social cognitive theory) and application of the mixed reality continuum. The proposal combines both qualitative and quantitative approaches. This project was part of my MSc Human-Computer Interaction, Future Interfaces module.

This article summarises a University assignment paper. You can check the original paper with in depth analysis and references as PDF.

Abstract

Ubiquitous technology delivers education new possibilities, shifting from the two-dimensional environment to technology-enhanced simulations which permits students and teachers to go beyond the traditional didactic methods. In this paper we present an e-learning concept that leverages the power of mixed reality. By taking advantage of learning theories and mixed reality, we propose a system that aims to achieve seamless interaction in order to facilitate the learning outcomes, allowing the user to adapt the level of immersion to their needs. We further propose an evaluation plan which aims to assert whether enhancing e-learning with mixed reality is effective and achieves seamlessness.

Why is e-learning important?

As technology becomes ubiquitous, there has been a shift in higher education and pedagogy from traditional education practices towards more self-driven online education. E-learning is the concept and implementation of an online education approach, becoming well established and adapted by multiple educational systems, as modern technologies emerge.

As an integral part of the educational system, e-learning is an effective tool as it offers flexibility through a wide range of opportunities within academic institutions and not only, for learners or academic qualification seekers to fulfil their educational objectives. On the opposite side, it provides teachers, instructors or professors with novel and flexible ways to teach their students.

Graphical vs Tangible user interfaces

In contrast to graphical user interfaces (GUIs) established by Xerox, which represented a "desktop metaphor" that simulates the physical desktop onto the screen, tangible user interfaces (TUIs) augment the physical world by associating digital information to physical objects from the real world, used in the everyday life. Their basis is on the notion of "ubiquitous computing", first coined by Mark Weiser in 1991 - a computing and human-computer interaction (HCI) paradigm which attempts to make computers invisible.

Weiser’s vision of ubiquitous (or pervasive) computing was "invisibility". They described the concept of seamless technology as a tool that does not intrude the user's consciousness, allowing them to focus solely on the task they perform, rather than on the technology they interact with. Technology such as AR, MR and VR sustain seamlessness, bridging the gap between the virtual and physical world. Their focus is to extend the human-computer interface into the physical space through computer-generated 3D objects.

Taking e-learning beyond the GUI

Beyond the graphical user interface (GUI), there has been work around the impact of virtual reality environments (VREs) on learning. There is compelling evidence which shows that simulation can improve learner competence and skills compared to traditional approaches. Immersion can vary from low (i.e., illustrated textbooks to low immersion, such

as educational games on a computer screen) to high immersion (i.e., educational games presented using a head-mounted display). A study comparing low to high immersion environments in the context of learning suggests that higher immersion imposes deeper understanding of concepts and thus more efficient learning.

According to learning theories, immersive environments enhance presence, thus causing the learner to process information more deeply. Through immersion, the virtual environment excludes real world stimuli from the user providing panoramic surroundings that completely put the user in a different place, not just visually. Higher immersion promotes higher presence, which represents the degree to which users feel that they are in an environment that is different from where they actually are.

Additionally, seamless interaction promotes a stronger sense of presence due to the more natural interaction which might result in cognitive load reduction. Hence, it was found that students learnt more deeply when the immersion was high, as it promotes engagement and presence.

Finding the place on the Reality-Virtuality Spectrum

Mixed reality (MR) in the e-learning context can provide students a learning environment that clearly shows the disparity between theory and practice as simulation can also represent a prerequisite for learning psychomotor skills whilst providing an in-depth understanding of the theory. By emerging virtual and real worlds, MR can add to e-learning haptic feedback, 3D modelling, simulations, just to name a few, creating a seamless interaction. Furthermore, narration was found more effective than displayed text in terms of learning outcomes, and MR can improve collaborative work substantially.

The proposed system

In the paper, I describe a system that leverages the mixed reality concept in order to enhance the potential of e-learning.

The system is composed of physical objects: a book and a head-mounted display (HMD) with headphones and microphone. The physical book serves as the main user interface. Users can use the book without technology, still being able to read text and view images. Through the HMD, users can see 3D computer-generated objects on top of the pages in AR.

The role of these animated objects is to enable further exploration of the concept presented on the physical book as text or image. The virtual objects come with audio sounds that provide a deeper explanation for the content, in the form of a narration. The user can interact with the system through voice and gestures.

Users are free to explore the book by turning the pages and from multiple perspectives by moving themselves, the book or manipulating the objects through gestures and/or voice - this does not constrain users to remain still or in the same place. In addition to this, VR view enables them to fully immerse into the scene and interact with the objects. By offering the full Reality-Virtuality spectrum, users can adjust the level of immersion and exploration to their needs.

Please refer to the paper if you’d like to see more details and how I propose this system is evaluated.