Spinning Up in VR — Part 5: Human-Computer Interaction in Virtual Reality
Hi, welcome to Spinning Up in VR! I am Manorama and this is the fifth of a 9-part tutorial on Virtual Reality (VR) for beginners.
In the previous parts I have talked about the VR hardware and VR software and in this chapter, I will introduce the Human-Computer Interaction in Virtual Reality. If you are interested in reading the previous or the next chapters, the links are at the bottom of this article.
Introduction
HCI (human-computer interaction) is the study of how people interact with computers and to what extent computers are or are not developed for successful interaction with human beings.
As its name implies, HCI consists of three parts: the user, the computer itself, and the ways they work together.
User
By “user”, we may mean an individual user or a group of users working together. An appreciation of the way people’s sensory systems (sight, hearing, touch) relay information is vital. Different users have different ways of perceiving and retaining knowledge and form different conceptions or mental models about their interactions with computers. Cultural and national differences play an important part too.
Computer
When we talk about the computer, we’re referring to any technology ranging from desktop computers to large-scale computer systems. For example, if we were discussing the design of a Website, then the Website itself would be referred to as “the computer”. Devices such as mobile phones or VCRs can also be considered to be “computers”.
Interaction
There are obvious differences between humans and machines. In spite of these, HCI attempts to ensure that they both get on with each other and interact successfully. In order to achieve a usable system, you need to apply what you know about humans and computers and consult with likely users throughout the design process. In real systems, the schedule and the budget are important, and it is vital to find a balance between what would be ideal for the users and what is feasible in reality.
The Goals of HCI
The goals of HCI are to produce usable and safe systems, as well as functional systems. In order to produce computer systems with good usability, developers must attempt to:
- understand the factors that determine how people use technology
- develop tools and techniques to enable building suitable systems
- achieve efficient, effective, and safe interaction
- put people first
Human Perception
As I emphasized in the earlier chapters, designing an appealing VR experience is all about understanding how humans perceive and adapt in an artificial environment and what gets them comfortable with the virtual world quickly. Here are two videos explaining the role of understanding human perception in the context of VR experience design:
User Interactions in Virtual Reality
First of all, What is Interaction? Well, when it comes to interaction, VR really is different. It’s a total paradigm shift in terms of how we interact with software.
Why is that? Well, the reasons are quite basic and straightforward. User interaction in VR is very different from that with a phone or a laptop. The fundamental different stems from the fact that you have a head mounted display covering your eyes that shows you the virtual world, with the real world devices completely blocked off. It’s pretty hard to type on a keyboard if you can’t see the keys. That means a lot of the interfaces like mouse and keyboard don’t work in VR because you can’t really use them. Another reason is that a lot of interfaces are designed for 2D. If you think of a web page it’s designed to 2D playing with content on it. A mouse and a touch screen are both fundamentally designed for 2D interaction. They allow you to move around on the surface and select things but they don’t allow you to interact with depth.
Virtual reality on the other hand is fundamentally 3D. You’re interacting all around you in a 3D space. A mouse just won’t work. So we definitely need new types of interaction devices to make me VR usable at all, but there are other deeper psychological reasons why virtual reality interaction is different. Virtual reality is about transporting you to a virtual world. So the job of virtual reality interaction isn’t just to make our environment easy to use, It is about making us feel immersed in the virtual world. Virtual reality is all about creating the three illusions that make up presence.
- Place illusion is the feeling that we’re in a different world. Place illusion is about your movement and your perceptions matching. It’s mostly handled by the fundamental interactions of virtual reality like when your head moves and your view the direction you’re looking in changes. These are built into any virtual reality headset and so pretty much come for free.
- Plausibility illusion is the feeling that the world is real and that is responding to us. Plausibility is probably the most important illusion when it comes to interaction. It is about the world responding to you in a way that’s credible. What credible means is that things respond to you as they would in the real world. So it’s really important that the VR world responds to you when you try to interact with it. But in a sense that isn’t anything new as most 2D games and apps do respond to your actions.
- Embodiment illusion is the feeling that we have a body in the virtual world. Embodiment is about linking your body to the interaction. Feeling that you are interacting with your body rather than a mouse or game controller. It comes from having a visual representation of your body, or at least your arms as you interact, but it also comes from really moving as we interact not just press buttons. You will feel more embodied if you actually reach out to pick something up with your hands, rather than just pressing a button.
Natural Interaction
- Before virtual reality became mainstream there was already a lot of interest in new ways of interacting, particularly in gaming. The Nintendo Wii, the Sony Move and the Microsoft Connect all tried to create new ways of interacting with games and computers. They were less about fingers on buttons and more about moving our whole bodies. They allowed us to interact by making gestures and to interact with the kinds of movements that we would use in the real world.
- A great example of this is the Nintendo Tennis game that used a Wii controller and you play tennis simply by swinging your racket. This kind of real world interaction feels more real and it allows us to make use of the everyday skills that we have learned simply by existing in the world all our lives. You don’t have to know about gaming to play Wii Tennis. You just have to know how to swing a tennis racket, and not even that well. So real-world interaction like this seems perfect for virtual reality because all these technologies lets us interact with things as we would in the real world.
- That is why so many VR headsets come with similar interaction devices. Sony PlayStation VR comes with the Sony Move. The Oculus has Oculus Touch and the HTC Vive has its controllers. But the so-called natural interaction techniques aren’t necessarily better. The great interaction designer Donald Norman wrote an article critiquing these new interfaces or at least some of them. Making a traditional user interface easy to use requires thinking about a lot of things. The actions you can do need to be discoverable. You need to be able to see what you can do in an interface.
- Gestural interfaces often do none of these things. It’s hard to know what gestures you can do. It’s hard to know unambiguously which gesture does what. And they often don’t give you feedback on whether you’re doing a gesture correctly or not. Why is this? If natural gestures are so natural, why are they hard to use? Well, sometimes they are subtle and difficult to discern in a VR system. Sometimes there just isn’t a natural action in the real world, that corresponds to an intended effect in the virtual world.
Magic Interaction
- We’ve been saying so far that virtual reality interaction should be as realistic as possible. It should be as close as we can make it to interacting with the real world. That will create plausibility illusion and make it feel like a real environment. On the other hand, traditional graphical user interfaces are more abstract. They’re not based on the real world. That means using a graphical user interface what feels real and is grounded. That will allow us to do things we might not be able to do with a realistic interface or in the real world.
- But in VR there is also another option. Because Virtual Reality is not the real world where you can have interaction that works as if it was in the real world but you can do things that are not possible in the real world. You can do magic. You can have superpowers. This is what we call magical interaction. Suppose you need to reach out and grab something on the other side of the room. Because it isn’t the real world, why not just grab it by looking at it? This is an example of magical interaction. Part of the point of virtual reality is that although it looks and feels real, it isn’t the real world. That’s the virtual bit. So, we are supposed to be able to do things that we can’t do in real life. That’s why we use virtual reality. So, why not apply that to interaction? Why limit ourselves to strictly real-world interactions? Magical interaction is different from the type of abstract interaction you do with a graphical user interface because it does use interactions that you could imagine in the real world. It is grounded in real world interaction. It just extends them to make them more powerful so you can extend reaching to grabbing something by being able to stretch your arm. You can extend jumping to being able to fly.
- Magical interactions are interactions you could do in the real world if only you’re a superhero. So, designing a magical interaction is about taking on something you do in the real world and extending its range, extending its power. Sometimes you need magical interaction because you can’t reproduce the real world interaction in VR. For example, most mobile virtual reality doesn’t include hand tracking, so you can’t just grab something. That means you have to select something by looking at it, which is pretty magical when you think about it. At other times, you want magical interaction because you don’t want to be limited by the real world.
Active and Passive Interaction
Active interaction is what we normally think about when we’re interacting with a computer. It is consciously deciding to interact. I decide to click a button on my phone and I click it. But we also have passive interactions with technology. For example, my mi-fit band will detect when I move anywhere and alerts me when I reach the target number of steps for the day. At no point am I consciously interacting with it as a piece of technology. I’m not clicking buttons. I’m not doing anything to it. It’s just detecting what I’m doing in the background. It’s doing its thing based on the actions and activities that I’m doing in my life.
Virtual reality has a lot of active interactions — like, picking up objects, moving stuff around. You are choosing to do things in the world, but there are also lots of possibilities for passive interaction. The most basic virtual reality interaction of all turning your head to look around is basically a passive interaction.
There are lots of other possibilities for passive interaction. For example, you might want the leaves of a tree to move as you brush past. That’s the kind of small response from the world that can really make you feel like you’re part of it. It’s exactly that kind of little response that creates the plausibility illusion. So as well as thinking about the active interactions that your user is going to be deciding to do and doing, also think about possible passive interactions. The way the world subtly responds to where your user is and what your user is doing. But there are a couple of things you should be careful about with passive interaction design. I will leave those to you to decide :)
In the later chapters, we will look closely into how the user navigates in VR and how the teleportation actually works in virtual reality. Stay tuned!
I hope that anyone who reads the entire series in the given order you will have a clear visualization of VR technology. Please feel free to leave your comments below for feedback. You can find me on Twitter(@mnrmja007), Facebook(@mnrmja007) and Medium(Manorama Jha).
