Experiential Design｜Task1

-

Feng weijie | 0349058

Experiential Design | B' in Creative Media

028.04.2024 -28.7.2024 (Week 14 -Week 1)

---

Week 1

Experiential Design Lecture #1: Module Introduction

This week, our module coordinator, Mr. Razif, provided us with an introduction to the module, outlining the themes and expectations. We were shown samples of advanced AR projects created by previous students, which we will use as resources and inspiration for generating our own design project ideas.

In summary, the "Experiential Design" module is a multidisciplinary design approach that prioritizes human-centered environments to support positive experiences and enhance the well-being of users and visitors. For example, in this module, we will focus on the user experience of augmented reality (AR) applications.

Following the module introduction and overview, we were instructed to download Unity, a cross-platform game engine developed by Unity Technologies. We were advised to download version 2022.3.25f.

Figure 1.1 Unity application. April 24th

# Experiential Design Class Exercise: Library Reading Process

For this exercise, we temporarily formed a collaborative team consisting of Chai Hong Hong, Guan Shanlu, Atika, Ahmed, and myself. Our task was to use augmented reality (AR) solutions to create a user journey map for the experience of visiting a specific place. We chose to focus on the library at Taylor's Lakeside. In our user journey map, we covered all possible experiences from the user's arrival at the library to their departure, considering both positive aspects (gain points) and areas that might cause difficulties or frustrations (pain points). Additionally, we explored potential solutions that the library could implement to enhance the overall user experience. The complete results are documented as follows:

Figure 1.2 Library borrowing process. April 24th

Figure 1.3 Library borrowing process miro. April 24th

I discreetly captured some photos of my group members during their presentations. They are all incredibly friendly, and I find this temporary group to be fascinating. Working with them has been a delightful experience, and I am thoroughly enjoying every moment. The camaraderie and collaborative spirit within the group make our sessions both productive and enjoyable.

Figure 1.4 impromptu group presentation. April 24th.

Week 2 -  # Public holidays 

no class `

Week 3

Experiential Design Tutorial #2: Introduction to Unity and Vuforia

In this session, we learned how to use Unity Hub. After downloading the engine, we need to go to [Installs > Add Modules] and install the necessary platform modules. For Android phones, we need to install the Android Build Support module, and for iOS phones, we need to install the iOS Build Support module. Once the platform modules are installed, we can start creating new projects! Before starting, we need to log in to our Unity account.

To set up Unity and Vuforia, Mr. Razif provided video tutorials on mytimes.

Figure 1.2 Set up Unity and Vuforia tutorials. May 8th

Download the URL of vuforia ：https://developer.vuforia.com

Figure 1.3 Download the URL of vuforia.  May 8th

I don't know why I can't download it, so I chose to download it from Google Driver shared by Mr. Razif.

After downloading Vuforia, we need to import it into our Unity project and add our own app license key. This step is simple, but it is crucial for the success of the subsequent steps in the project. Following the tutorial steps, we first need to apply for and obtain an app license key from the Vuforia developer portal. Then, import the Vuforia SDK into the Unity project and add the obtained app license key in the project settings. Once these steps are completed, we can start using Vuforia in Unity to develop augmented reality applications. I thoroughly enjoy this learning process and look forward to further exploration and practice in the upcoming lessons.

After completing the Vuforia plugin setup, we moved on to the next phase. Today's lesson began with Mr. Razif giving us a lecture on how to create our own Target Manager in Vuforia. We then imported the image assets we found and completed the star rating. After that, we downloaded the Target and imported it into our Unity project.

Next, we created a 3D cube model in Unity. This way, when we use the AR camera, a 3D cube will appear on the image. This step was not difficult as long as we followed Mr. Razif's instructions.

After completing this phase, I began practicing this step in my own project.

After importing the Target asset into Unity, I began following the steps to complete the process. I added a 3D model and used the AR camera to check if it was successfully displayed.

After importing the Target asset into Unity, I began following the steps to complete the process. First, I added a 3D model and used the AR camera to check if it was successfully displayed. Then, following the next set of instructions, we needed to add an Image Target and place our downloaded video file (MP4) on it. This way, the video would play when the AR camera scans our image.

To ensure the best display effect, we needed to adjust the size of the Image Target to match the size of our image, so the video would fit perfectly on the phone screen. By following these steps, I gradually completed the process and practiced it in my own project.

After class, Mr. Razif mentioned that we could freely change the video to whatever we wanted, encouraging us to be creative. This immediately sparked a whirlwind of ideas and possibilities in my mind. I started brainstorming furiously about the kind of video I wanted to add. To ensure the final effect of Task 1 was excellent and thematically consistent, I decided to use a character from my favorite Japanese anime, "Chiikawa"—Usagi.

The thought of Usagi's adorable image appearing in my project made me incredibly happy and excited. I envisioned various scenarios, contemplating how to seamlessly integrate this character into my project to enhance both its fun and professional aspects. Ultimately, I decided to use an animation clip of Usagi as my video material. This not only makes my project more engaging but also reflects my fondness for the character and my enthusiasm for the project. Incorporating this beloved anime character into my Task 1 filled me with immense satisfaction and anticipation.

I then quickly completed the project, successfully integrating Usagi's video into my Task 1.

Week 4

In Week 4, we continued our lessons based on the project from Week 3. Mr. Razif began by explaining how to add interactive features, which required more control. Since we were working with real screens, we needed to design a user interface (UI). First, we had to adjust the screen resolution according to the phone we were using. I have an iPhone 13 Pro, but unfortunately, Unity doesn't have this option, so I opted for the screen settings of the iPhone 12 Pro, which has a similar screen size.

To control video playback and pausing, we need to set up appropriate buttons. In my project, I designed several key buttons to enable these interactive features: video playback and pause, the appearance and disappearance of the 3D model, and the start and pause of animations.

These buttons not only enhance the user interaction experience but also make the entire project more flexible and user-friendly.

Speaking of the 3D model, I wanted my project to be more aligned with the theme. So, I spent the night learning how to model Usagi and finally managed to create it in my Nomad application. After that, I exported it as an OBJ file and imported it into Unity. Unfortunately, I found that my model in Unity remained white without any color, which was very frustrating. I tried various methods but still couldn't find a solution. This experience made me realize that 3D modeling involves not just creating shapes, but also understanding more about materials and textures.

Then, following Mr. Razif's tutorial, I started adding simple animation effects to my 3D Usagi to make it more dynamic and allow it to be controlled by my buttons. I set up some basic movement and rotation animations to make Usagi look more lively and engaging. Additionally, I ensured that these animations could be triggered and paused by user interaction buttons, enhancing the project's interactivity.

Then, I conducted tests and found that all the buttons were functioning correctly, and the video played smoothly. This was very gratifying as it validated all the hard work I had put in. Next, we needed to build and export the project to the iPhone (iOS). Mr. Razif promptly updated a detailed tutorial on YouTube, which was very thorough and easy to follow.

Following the steps in the video, I imported my Unity project into Xcode, managed the signing, and successfully exported it to our iPhone. Seeing the project run perfectly on the phone, with all interactive features working seamlessly, filled me with immense satisfaction. This not only validated my efforts but also gave me confidence for the upcoming stages of the project.

After successfully importing the project to my phone, I used screen recording to capture the entire final interactive process. The screen recording allowed me to showcase the project's various features more intuitively, including video playback and pause, the appearance and disappearance of the 3D model, and the start and pause of animations.

# marker-less AR exercise

This Week's Goal:

Mr. Razif asked us to record our markerless AR practice and post it on our blog. Specifically, instead of using 3D cubes, this practice should focus on items like furniture, chairs, tables, etc., as if we were using an IKEA app. In this way, we can demonstrate how to place virtual furniture in a real environment, simulating actual usage scenarios. Mr. Razif has uploaded a detailed tutorial on YouTube, and we just need to follow his step-by-step instructions to complete the task. This process not only helps us master markerless AR technology but also enhances our ability to apply it in real-world projects.

First, I searched for free 3D furniture models. After finding a suitable 3D model of a chair, I downloaded and imported it into our Unity application. With these models, we can realistically simulate placing furniture in a real environment, just like using the IKEA app. Next, I will follow Mr. Razif's step-by-step tutorial on YouTube to ensure the task is completed successfully.

3d sofa link to：https://3dwarehouse.sketchup.com/model/cf18f600ac49d5d032039d5a855f4770/sofa

Once imported into Unity, we need to ensure our model is on the Ground Plane Stage and adjust the model size accordingly. This step is crucial as it ensures that the virtual furniture aligns accurately with the real environment, creating a more realistic augmented reality effect. We need to carefully check the position and scale of the model to make it look as if the real furniture is placed in the room. Once the adjustments are complete, we can proceed to the next steps to ensure the AR experience runs smoothly.

Next, we need to create a folder named `iosbuild` and place it in our Unity project directory. After creating this folder, we need to export the Unity project to this folder for further processing. Then, we open Xcode and import the exported project files into Xcode. Once the import is complete, we need to perform some necessary configurations, then compile the entire project and load it onto our iOS device for testing. This way, we can see the actual effects of our AR project on the phone.

As the final result, I successfully completed the entire project. The demonstration video is shown below. I used the built-in screen recording feature on my Apple device to capture the process.

---

Reflection

In this project, I learned to use Vuforia for AR development and mastered the basics of Unity and 3D modeling. Although I encountered issues with the color display when importing the Usagi model, this process made me realize that 3D modeling involves not only creating shapes but also applying materials and textures. With Mr. Razif's guidance, I learned how to add interactive features to make the project more dynamic and user-friendly.

Throughout the experiential design process, I faced many challenges that left me feeling unprepared and confused. Small issues in each lesson sometimes had a significant impact on my overall progress, causing frustration. For example, in Week 4, I couldn't install the necessary software because my computer system wasn't updated in time, which hindered my ability to keep up with the course. Most of the class was using Windows systems while I was on macOS, which caused considerable inconvenience and anxiety. Fortunately, Mr. Razif recorded detailed video tutorials, allowing me to catch up at home.

Seeing the project run perfectly on the phone, with all interactive features working seamlessly, brought me immense satisfaction. Each challenge and problem-solving process helped me grow both technically and mentally. Overall, this project taught me a lot, from technical skills to problem-solving abilities. I learned that persistence and flexibility are key to success, even when facing difficulties and setbacks. This experience not only boosted my confidence in my abilities but also filled me with anticipation for future projects.