[11/13/22] Kicking off
This week we focused on exploring different ideas in different areas of XR. All in completely different domains. It's really hard to think of a solid idea that is innovative, fun, and doable within a restrictive three-week development period. Keep reading to see what we thought of and what we ultimately decided.
TLDR
AR Research
Researched WebXR
works natively with JS but hard to use with Unity
Doesn’t easily support Apple devices depending on implementation
Mozilla Unity WebXR Exporter
fairly outdated, will take time to understand
Research Niantic Lighthouse
doesn’t natively support WebXR
Research 8thWall
costs money, uses JS
Idea #1: VR Photogrammetry Platform
Taking cues from Google’s Earth VR, this idea sought to provide a high-resolution street-level VR experience. Users would be able to upload photogrammetry scans of their streets, buildings, museums, libraries, schools, and other shared spaces to our platform. On upload, a user would be given the ability to annotate their scan with any text or audio assets they saw fit to enhance the experience of traveling their uploaded scene in VR.
This platform was conceived to provide designers who are not 3D artists with a wealth of high-fidelity 3D models. Where a team of engineers may be able to hire a 3D artist, an individual may struggle to get their products off the ground without the help of free assets and 3D models. Thus, this platform would enable designers to enter the ecosystem of 3D modeling through the backdoor.
We sought to prototype this idea by first assessing the ease with which we could create our own photogrammetry scans. Our first attempts came from the mobile app Polycam, which allowed us to take photos of an object or interior and upload them to Polycam’s servers for processing into an exportable 3D model. While quick and frictionless, this app and others like it offer a limited number of free scans, so we decided to make use of our campus resources to photogrammetrize the interior of room 1018 in the DOW building on North Campus. We did so by borrowing a DSLR camera and tripod from the College of Literature, Science, and the Arts, and processing the photos using Agisoft’s Metashape software which is available on the computers in the Visualization Studio. Take a look at the output below.
Looks pretty bad!
After seeing this result of our photogrammetry efforts, and after considering the fact that we were not interested in doing much 3D modeling for this project, we decided to scrap this project and focus on our other options.
Idea #2: AR Collaborative Social Idea Project
This idea was inspired by the different physical message boards that we see around campus, where different organizations might put up posters to promote their ideas. Our take on this, was to create an AR experience where multiple people can add to a shared board, that is represented as an Augmented Reality city that might sit on a table.
Essentially users would scan a QR code (that rests on a table), which would launch a web version of the app on their phones. With this web app, they would be able to write a post (in the form of text or images) on their phone and be able to “swipe” it onto the AR scene. In the AR scene users would be able to view their post, as well as other people’s posts, which would be represented by buildings on the table. The size of each building (post) is determined by the amount of interactions that post received (comments, likes, views, etc).
The buildings are interactable by the user hovering over the building with an AR cursor, where information about the post creator, likes, number of comments, etc will appear. If the user wants a more detailed view, they can click a button and a detailed view will appear on their phone screen (they can see all the comments, and other details).
We will potentially add source control staff — that is, ARgit. In the collaborative experiment, people can trace back the development process, creating their own branch, merge with other people’s branch. They can enjoy all the benefits git provided to the programmers.
The implementation details of this are murky. One main thing that we thought about was that we need to decrease the barrier of entry of this app significantly. Similar to a cork-board outside of class, this should be something where a potential user sees the QR code, gets intrigued, scans it, and is transported into this experience seamlessly. If they need to download an app or do something like that, the potential user will move on and we lose engagement. Due to this fact, we poured significant time researching WebXR (a web-based XR technology) and how it might interface with Unity. We also researched Niantic’s Lightship ARDK which promises very awesome AR capabilities that promote shared AR experiences amongst multiple users simultaneously.
Social Impact:
This app encourages people to collaborate on building this city by sharing their ideas and thoughts. People are rewarded for sharing their ideas by getting more interactions and seeing the physical size of their ideas grow in the AR City. This app encourages people to leave the fully digital world and physically come together without necessarily requiring people to be at a place at the same time.
Novelty
There is nothing out there like this from what we could tell. There are other collaborative art AR projects, but no big collaborative “idea” project. And there are no AR collaborative projects that enable source control.
Technicals:
Will require significant work connecting with backend services and databases in order to allow this collaborative environment to work. Also, we would need to work out how to create a strong WebXR app that makes use of Unity technologies.
Financials:
We would like to incorporate ads into the platform. People might see these ads as other buildings in the city with billboards on them (Coca-Cola logo, for example) and when these ads are interacted with (via a distinct user-driven action), a detailed ad view will appear on their phone.
XR Emphasis:
This is probably the weakest part of this project. The collaborative art project part of this can be argued to have XR emphasis, but at the end of the day, AR is not an integral part of this process. This could also be done via some 3D models displayed on the phone screen itself. It is hard to tell if this AR City idea is simply a gimmick, or something people would actually enjoy.
Idea #3 [FINAL]: Infinite immersive Exercise Game
This game is designed to give users an immersive game experience while exercising. It’s known that people nowadays lack exercise, and VR can solve it by bringing real physical movement into the game experience.
The big problem with current VR exercise games is, they are typically designed for one scenario and one kind of exercise, like playing tennis, skiing, and so on. People can easily get bored with this after being very familiar with the game because you are always interacting with one scene and only doing a very limited set of movements. We are attempting to solve this by building an exploration-based immersive game, in which people will navigate a city, interacting with different scenarios when they move forward and are requested to make different kinds of movement.
We haven’t got very clear about the exact implementation of this idea. But the basic scene is, people will be hoverboarding in the city (in order to move forward they should do hoverboarding movement, which is regarded as the basic movement of our game). Through the journey, they will meet various kinds of experiences, like some people/animals throwing things at them and you must avoid by moving left and right continuously, you will go through some playgrounds in which you will get good rewards if you play football/badminton with people and win, and river you need to swim across, some small hill you need to climb …… We are going to combine many exercises into a navigation-based game experience. And through clever reward design, people will not lose interest.
Social Impact:
Through immersive game experiences, we encourage people to do enough exercise so as to benefit their health. Also, through infinite immersive navigation, we want to make people exercise continuously.
Novelty
Although there do exist VR exercise games, they are typically designed for one specific kind of sport or movement and people will easily get bored. We are attempting to solve this with an exploration-based game experience and award system so that users will continuously play it.
Technicals:
Will require significant work in researching how to track people’s movements in VR, and what sorts of movements are trackable.
Will require a clever design of navigation experience and a reward system to maintain people’s interest.
Financials:
We can sell the game on Steam and charge money to unlock new levels
XR Emphasis:
Very strong, because computer games can provide a cool game experience but can’t encourage people to exercise, TV-based games can encourage exercise but the game experience is very limited.
P3 Milestone Video: https://youtu.be/NKhBeqo6h_o
Assets
https://sketchfab.com/3d-models/space-loop-city-76882b787e6c4e0e9dd52db4f90c2622
https://sketchfab.com/3d-models/grid-city-7875ebc750ca4ab2971c2e0d38b80223