Type In Space for HoloLens (2018)

Type In Space — Explore Spatial Typography In Mixed Reality with HoloLens

Your physical environment becomes a new canvas for typography

Background

Microsoft HoloLens

As a graphic designer who loves type design and typography, I wanted to see beautiful type in space. We have been seeing many examples of type overlay in motion graphics and TV commercials but they were stuck in flat 2D screens. Since I met HoloLens in 2016, I have been exploring type in mixed reality space.

My previous projects on type in mixed reality

Typography Insight for HoloLens v1.0 (2016)

The holographic type shows magical tactile quality, feels like a neon which you cannot experience in the captured videos. The video below is closer to the actual holographic type seen through the lenses. (Captured through a phone camera)

Video recording with a phone camera through the lens on HoloLens

I expanded my spatial type experiments to a large scale three-dimensional type composition in Holographic Type Sculpture(2016). In this project, I created a geometric 3D composition and placed it in a museum. Just like a physical architecture, literally, you can walk into the shower of your favorite type.

Holographic Type Sculpture (2016), News Space (2017)

News Space for HoloLens(2017) was a spatial type experiment with text-heavy content, the news article. News headlines fill up your entire room with the spherical layout. With HoloLens’ high-resolution display, small text for the headline and summary text rendered beautifully while securing the readability.

In Museum of Type(2017), I continued my journey in virtual reality space. I wanted to create a virtual environment where you can learn about historically important typefaces with more direct interactions. Using the motion controller, you can grab and observe the beautiful shape of the type. If you love type design and typography, this will be one of the most satisfying experiences interacting with type.

With these experiments, I got great responses from the design community and augmented reality enthusiasts. I also had the honor to demonstrate my app Typography Insight for HoloLens to Matthew Carter, a legendary master type designer.

Toolkit for creating mixed reality experiences

With these new building blocks in MRTK, I was able to evolve the type lay out experience in my app Typography Insight. Eventually, I have extracted ‘type playground’ component and created as a new app ‘Type in Space(2018)’.

Type In Space (2018)

Type In Space (2018)

Interacting with text using two-handed gesture

Two-handed manipulation

For the rotation, you can specify axis constraints using the toolbar. Using this constraint, you can make the text rotate around a specific axis only.

Floating toolbar

The toolbar uses the tag-along and body-lock technique to avoid interfering your viewport. It is moving and always stays within a certain range, with a specific offset from your waist. It provides adjustable positioning buttons so that you can easily move up/down to favorite offset position from your body. Pin button makes it stay in a fixed location. You can also show or hide the toolbar with a voice command. “Show Toolbar/Hide Toolbar”

Text properties

Text properties on the toolbar

Text object menu

Snap text to a surface with spatial mapping

Typesetting on the physical surface with spatial mapping

Air-tapping the ‘Snap to Surface’ button on the floating toolbar initializes scanning experience. When you look around your environment, HoloLens will gather information about the surfaces around you. You can understand the progress with blue visual meshes.

Once it gathers enough information about the surfaces, the blue mesh will disappear. Now you can air-tap any text object and gaze on the surfaces, the text will snap to surfaces, following your gaze cursor. You can make the text align with various surface types such as walls, floors. and ceilings. Air-tap again to place the text object.

Gravity with physics

Text input

Text input with HoloLens system keyboard input

Save and Load the scene

[Update: Sep. 2018] Outline Text

Where can we use it?

Architectural Signage Design

Real-time signage design sketch on the physical surface

Object Labeling / Annotation

Labeling the physical object

Wayfinding

Using holographic text as wayfinding elements

Product Package / Book Cover Design

Data Visualization with dynamic data binding

Holographic Type Sculpture with live data

Typography in Mixed Reality

Type size in 3D space

Display resolution

Improving type rendering quality

TextMesh Pro is a great solution to secure the text rendering quality. Since it uses SDF technology, it renders crisp outlines of the text regardless of the distance. From Unity 2018.2, TextMesh Pro is included in Unity editor. If you use it with a large scale font, it shows rounded stroke edges. You can improve this quality by increasing the texture size in TextMesh Pro’s texture generation process.

Scaling and positioning text in mixed reality

Volumetric text

Thanks for reading!
I hope you can experience beautiful type in space with HoloLens!

Holographic type layout at Berlin Central Station

*I work at Microsoft as a User Experience Designer. Type In Space is my personal project. All experiments and related opinions are my own.

Get Type In Space on Microsoft Store

Read my other stories

Designer & Creative Technologist. UX Designer @Microsoft HoloLens. Creator of http://typeinsight.org Opinions are my own. http://dongyoonpark.com