Mixed Reality Studio Environment Project

Introduction

In this design project, I came up with a way to use mixed reality in order to improve upon the studio experience for CMU design students. To talk about the problem space, Covid-19 and the global pandemic has drastically shifted the way education if offered, and especially for the students whom are not able to attend in-person, there is a longing for social communication, when learning from each others work. It is hard being remote and sharing 3D work, and having people around you is part of the college experience that people really look forward to. In this project, I was interested in the concept of making critique sessions better for remote students, and thought about how to integrate others into the analysis of your own work through the powers of technology.

Learning About Personas

In class, we started this project by learning about personas, or in other words a representation of a user type who would interact with the proposed design.

1. Goal-directed Personas : What does my typical user want to do with my product?

2. Engaging Personas: designers actively involving themselves in the lives of the personas.

3. Role-Based Personas: massively data-driven and incorporate data from both qualitative and quantitative sources. They focus on the user’s role in the organization.

4. Fictional Personas: requires the UX design team to make assumptions based upon past interactions with the user base to deliver a picture of how, perhaps, typical users think and behave.

Asking Peers Questions

In order to form my personas, I talked to my fellow classmates and interviewed them about their studio experience as well as their remote learning experience so far. Here are some questions I asked:

1) how do you like to learn from other people and their work? (Did you enjoy in person wall crits, or by reading people’s medium posts, or talking to people individually at their desk etc.)

2) what aspect of being in studio is the reason why you enjoy being there? (Is it the people, the convos, the independence, room ambiance etc)

3) Do you feel like you are a person who works best when the studio is silent/ less people there, or with more people with bustling energy?

Here are some of the notes and responses I gathered from my research:

CAITLYN

1. I think those are all helpful learning methods in their own ways but personal interactions/convos with people in studio talking about their work is p valuable. It’s nice just to get up every once in a while and walk around to see what ppl are working on and maybe talk to them about a current project (helps you learn about their process/approach and ask questions).

2. I think one of the best aspects of studio is having a designated workspace that is also flexible/allows for the option to either socialize or work independently. I think I work best in quieter environments, but just being surrounded by other people working was helpful in staying motivated.

3. It depends on the workload/projects at the time I guess. Fewer people or a quieter environment is sometimes nice for focusing on work, but a fuller studio can also be nice if I don’t have a ton of work and have time

CATHERINE

1. unfortunately in Covid times sometimes the most efficient method is looking at medium though sometimes people don’t update them regularly. but my favorite is definitely talking with the person face to face and just explaining our reasonings and discoveries after class while we are working in studio

2. I think it’s definitely the people and then the space itself as a close second. It’s sort of nice knowing that everyone is suffering together and seeing how their work evolve. I also like the space because of the big tables, lighting, and just having a somewhat private place on campus all to myself.

3. I definitely work best with less people but not too little? sometimes working by myself is chill, but that can be lonely and defeat the reason for going to studio in the first place. bustling studios are good places to destress/have casual chats and thankfully none of my mini studios were too noisy for me to work in.

GARY

1. I mostly just gauge how much work to invest into an assignment by looking at what other people, besides that I don’t really learn from others
2. I enjoy having a separate space from my living area where I can be more productive
3. When the studio is empty

NEELY

1. I personally like wall crits and individually talking to people at their desk cause i was able to physically see the product and hear quick responses, everything is so slow and laggy in doing it remote.
2. The convo, people and ambiance. I liked to see people work in front of me, it made me feel comfortable
3. I think it depends on the assignment or project, i like to be in my own headspace with earphones on and concentrate, but still be able to see people working

After these interviews, I mapped out some basic information and formed these persona templates per person, to better categorize everyone’s information.

Finalizing Personas

From these basic persona templates I made for my classmates, I then was able to synthesize the key information and create these mockup character personas for the types of people who would interact with my mixed reality design. I noted down a key way each of these users (Emily, Jake, Maddie, and Ben) like to learn and find their best abilities.

Here is one of the final slides from my presentation that clearly map out each persona, their key points, and the idea I considered for my mixed reality design.

Gathering More Ideas and Research

I decided to research more and gather knowledge about real design solutions that use mixed reality and AR to improve people’s work as well as home lifestyles.

Microsoft Hololens:

• blending digital life with physical life
• hologram is basically light- using and studying the physics of light to create it

examples:

• people fixing a bike and video points to parts which annotate in holograph for you (twisting and turns)
• 3D modeling- pinching and raising and scaling hologram
• using real life as a video game where character jumps on sofa and goes through obstacles
• taking assets and moving/rearranging them

Microsoft Productivity Vision examples:

• tapping on coral reef and learning about plant when scuba diving
• foldable interactive book/notepad

The Five Product Video:

• wear the goggles and gestures initiate motions of figures on game board
• mind controls actions?

How AR works

• maps environment, tracks location
• places digital content into the physical world

VR Interface Design Methods

• more screen space- allows more productivity
• 360 degrees
• spatially organizing tasks means more productivity — understanding perspective ** stacked files

12/1/20 Self Reflection

As the prevalence of digital media in our physical environments increases daily, what is the role and/or responsibility of designers in shaping our environments?

One of the role’s and responsibilities of designers in shaping environments which are increasing becoming digitized, is being able to make interactions intuitive. When people use smart devices or Apple products, or if we think into the future of holograms, if people need to use them, the nature of the design needs to be simple enough to understand, and satisfying to use.

Pressing buttons that trigger an action, or other movements like swiping, pinching, scaling, and drawing all have the system of the user doing something that creates the response the want. With digital media in physical environments, designers must consider that when the digital media is presented, it coordinates with the action that the user’s does. That way, learning is easier, and the design of any hybrid experience can accommodate for many people to understand.

Progress 12/2/20

What programs do I use to work, communicate, socialize?

• Mac- typing, Adobe programs, Zoom
• iPad- Procreate drawing
• iPhone- messaging, calling, social media, camera

How do I get work done?

• write down tasks and prioritize, complete the work one by one
• on laptop- screenshots and other files/images/documents, phone has photos and videos, iPad has drawings

After reflecting on my own way of working and how i get work done, I found an idea that I could incorporate into my project: A way to pull everything in the air and pinch and rearrange on screen — so when people view medium, they can themselves pull out images of models and look at them, then insert it back.

Daphne’s Critique

This idea seems interesting that there can be a way to organize files and pull out peoples work from medium and put it “back into the cubby” or spot on the screen. However, this is treating the mixed reality experience as more of a 2D surface, and there can be more I realized could be explored. Incorporating sound or other gestures that inform a space that is 3D can be more communicative and more impactful to help the hybrid learning experience.

I took Daphne’s ideas into consideration, and thought about how instead of using gestures to move around flat things, it could be applied to 3D forms. Specifically, I chose to think about the Semantics of 3D Form project from freshman year, and how the starfish form has so many subtleties and components that need to be paid attention to.

Storyboarding and Idea Generation

To visualize my desk space and think about the mixed reality experience, I took photos of my room and desk, and annotated on top. This technique allowed me to make connections between devices and how I like to annotate my work, and how pinching and zooming were motions I did to understand the 3D starfish when I did that project.

Taking this idea of marking up, highlighting, and commenting, I furthered my idea to think about how I can create a 3D scan of the starfish, and be able to place those highlights, comments, and annotations within space, not just on a photo.

Another idea is that there could be the option to encourage collaboration and social interaction by being able to see people’s faces talking and make synchronous markups on any 3D work. As one person points or highlights a part of a model, the technology would allow for the “invited” person to the critique to also see those actions. This would create greater ability for people to learn and understand exactly what is being seen and pointed to. It would give clearer direction, and help students make more deeper connections to their work with the feedback.

Below, I created a quick storyboard outline for the steps in the process of the mixed reality experience. First, the user would take their starfish for example, and use their phone and scan it. Once the scan registers all the data and records the 3D form and structure, it is ready to be copied into the mixed reality space. This can be seen through wearing a hololens, or with a hologram technology that stores information on a cloud.

Next, the user would pinch and drag the scanned copy of the starfish from the phone and into the air where they can modify, pinch, minimize, enlarge, and move.

To edit and make annotations, they can tap to have the menu option appear with the choice to highlight. They can highlight areas like legs, or make color coded components to make comparisons and differences. They can categories the 3D form into blocks, and understand the form better by visually marking up different 3D parts, almost like building blocks.

Next they can insert a comment annotation, pinpointed to a particular area. This would allow for specified comments indicating exactly where and what the feedback is pointing to or talking about in the 3D space, and not just on a 2D surface.

These are the two main features, and additional ones would be recording a message into a comment, or just simply making a message for a friend. The option to invite a critiquer allows for the collaborative aspect of critique that will be synchronous.

Here is the slide I created for my presentation that gives an overview of the 4 features for the experience.

Video Process

To start making my video, I mapped steps I wanted to consider. The two main features would be highlighting and commenting on parts of the 3D starfish model. I first took a photograph of my starfish, and used Procreate to outline a skeleton drawing and make axis lines to show the 3D form. I made it blue to indicate that it was the “copied” version of the scan, which can be brought out into space using mixed reality.

I then used After effects to create animations and overlay icons, highlight dots, and comment assets, and played with the functions to make them appear as one taps either the starfish scan or the menu bar.

some notes on videomaking and creating an outline for the presentation

Here is a storyboard snapshot of the video, and captions under each frame explaining the gestures involved.

Self Reflection:

How were the skills you developed in the first project similar and/or different from the second project? What is your understanding of the role of an Environments designer?

One of the skills I learned in the first project that was similar to this one was being able to think and brainstorm ideas better by gaining a sense of space. Because I took many screenshots of my SketchUp model and annotated different pathways, I learnt and got new ideas for how to designate parts of the space to different content, and what the visitor’s path would be. Similarly, with this project I learned by documenting and taking photos of the space around me and marking and labeling. Even if it could have been just a label or name of an object, it helped me realize what was around me. Something I learned that was different from the last project was that with mixed reality, drawing on photos of the space helps making icons in illustrator easier, and helps me plan out storyboards clearer.

Through this class I learned that the role of an environments designer isn’t just about creating an experience that is cool or interesting, but something that bridges the real physical world with the technological world. There are so many ways and affordances that need to be thought about like gestures and motions, and studying humans in their spaces is crucial to design for the best environmental experience. That is why the initial stages of brainstorming, researching, and gathering interview data from real people, and spending time to make personas helps a lot to think about a design that could work.

I enjoyed this project, and feel like I overall could push myself more if I had a little more time, but I got a glimpse at the infinite possibilities for what E track and environmental design could teach me.