This is the first in our 'Behind the Scenes' series exploring what goes into building the Museum of Other Realities (MOR). Last month, we introduced three new exhibits, each with exciting challenges and requirements. With six new wearable dresses in Tilt Couture, virtual flora in Lignes de Fleurs, and the futuristic cityscape of Alex's Sci-Fi World, here's a peek behind the curtain of last month's update.
An animation designer and fashion illustrator, Anand Duncan's dresses have been a part of the MOR for a long time. For the August update, we wanted to place her fashion work on the center stage (literally), especially with her new Neon Metropolis Gown on display. Anand works in the VR illustration software, Tilt Brush, so there were a few things we had to address during implementation.
To begin with, we ran an optimization pass on all the dresses, swapping out a Tiltbrush shader with a custom MoR shader that allows us to add highlights and a bit more materiality to the dresses. Since they can be worn by visitors, the dresses needed to be easily distinguishable throughout the gallery, against both light or dark backgrounds, in a similar way to how we render our avatars.
The newest dress, Neon Metropolis Gown called for a cloth-like effect. Regarding implementation, Lindsay Jorgensen, Technical Artist at the MOR, says:
"Running cloth simulation is likely to be too computationally cumbersome in respect to all we have going on, so we went with a simple effect where we split the model into top and bottom elements, then 'skinned' the bottom of the dress. By adding a script that uses a 'SmoothDamp' function to lag the position and rotation of that control bone, we are able to give the dress a swishy, twirly effect. We are also able to get the rotational velocity out of this which we then use to control the audio."
In addition to optimizing the dresses, another important part of the process was staging; figuring how best to showcase these dresses. We went for the showroom experience, installing a jumbotron (or mirror) and a runway for visitors to strut across.
A major consideration with the room design was ensuring it elevates, rather than detracts from, the dresses and the experience. So we focused on using a simple design, leveraging lights and bloom without being overpowering.
This selection of pieces comes from an ongoing daily project by Sabby Lighf called 365 Days of Tilt Brush, in which she is determined to master VR illustration. For Sabby, flora has always been a source of intrigue, not only as an influence on her art but also in terms of the feelings they evoke in her. The patterns in Lignes de Fleurs were inspired by the striations and textures in plants on a microscopic level, so an important part of staging the room was ensuring that these details were apparent to the visitor in the MOR.
We extracted some key elements from the plant pieces and placed them in ‘planters’ while letting them spread and take root. Em Halberstadt, from A Shell in the Pit, then implemented audio in relation to each plant biome. This has the benefit of grounding the pieces, making them more integrated within the museum, as opposed to having them simply float in space. Like the dresses in Tilt Couture, the pieces in Lignes de Fleurs required some script-level optimizations. We ran a decimation operation on their geometry while making sure to not lose mapping information or details such as the stroke types used in the original artwork.
The butterfly piece was where we found some scope to play. The long wavy lines were adapted with a special shader that added some transparency and motion. This created an airy feeling, allowing users to pass through less jarringly. We extracted the elements from the wings and animated them to flap when triggered randomly. With this, we could then set up a special volume with the script to temporarily attach a copy of the wings to visitors for a brief amount of time when they move through the butterflies. Along with the dresses, this type of avatar customization is something we'd love to explore more in the future.
The inclusion of Product Designer, Matt Shaefer's Alex's Sci-Fi World coincided with a pretty big update to Quill, the VR illustration and animation tool used to create the piece. This update added a bunch of animation features to Quill, and in turn, Matt updated the piece to take advantage of these features. As a result, this phase of Alex's Sci-Fi World features new micro-stories and feels much more alive.
Though we were intent on including the animated elements from the piece into the museum, the mere size of the data required made it impractical to fully adapt the piece in all its glory. We even experimented with an Alembic export, since the file format supports animated meshes, but we ran into similar issues. Our solution was to export a single frame from the piece and group elements within it into separate, more manageable, chunks. With the artist sending us a more simplified version of the scene (thanks, Matt!), we ran decimation on the models and made material adjustments. Fortunately, we were able to implement some simple animations with a script that iterates over a series of objects, enabling each one in sequence to recreate a similar animation from the original piece. In the future, we hope to recreate more animations, but since Quill doesn't export data of that nature, this is a manual process.
In terms of staging, we wondered "What if we also brought parts of the piece from within the portal into the museum?". By changing the rendering 'layer' setting in Unity we were able to take some elements and render them, both in the museum and beyond the portal. Having elements from the piece in the room feels more inviting, almost guiding the visitor towards a portal they might have otherwise missed.
You may have noticed a subtle update to the teleportation mechanic in the MOR. But if you didn't (because subtle), we added a bit of momentum to the pointer. Now it feels slightly more like a fishing line on a rod than a laser-pointer. Since teleportation is the primary mode of locomotion for most visitors in the MOR, this is one step towards making the mechanic feel more natural, fun, or at the very least comfortable.
We'd love to hear what you think about it! And if there's anything in particular about our process you'd like to hear more about, let us know on Discord, Twitter or Instagram.
