Augmenting an industry-standard creative tool to better suit the needs of our students, instructors, researchers, and other campus creators in a one-of-a-kind Libraries space.
Overview
The Visualization Spaces Development Kit (VisSDK) for the Unity Game Engine project was led by Elliott Schultz and Jayden Sansom, the 2022–23 Susan Ellen Everett Interns. During this project's development, Elliott was a senior in Computer Science Engineering with a Game Development concentration, and Jayden was a junior with a Game Development concentration and a minor in Art & Design.
Objectives and Aims
The aim of this project was to design and document a media development pipeline for the Libraries' Cyma Rubin Visualization Gallery which utilizes the Unity game engine, a consensus industry tool and so-called "off-the-shelf" game engine used widely on our campus and in the software development industry generally to create interactive experiences and games. This toolkit was created to increase the accessibility of creation within the Visualization Gallery for creatives, researchers, instructors, and their communities by creating and integrating a well-supported media development tool. Additionally, the VisSDK is built to support the porting of existing projects to the Visualization Gallery's specific dimensions and capabilities. It offers broad version and tool support for the Unity game engine, focusing on core paradigms within Unity to avoid obsolescence with near-future versions of Unity.
Technical Considerations
The VisSDK for Unity Game Engine is designed to offer reverse-compatibility with existing Unity 3D and 2D games and other interactive experiences built via the Universal Render Pipeline. The core elements of this development toolkit are pre-fab camera elements for 3D and 2D environments, which allow Unity scenes to be translated to the Visualization Gallery's very distinct 15360 x 1080 aspect ratio. The project also describes how to create continuous or discontinuous interactions at the desktop's ending "seam."
Results
The VisSDK is available for download and includes a pre-built first-person camera that can be used in Unity 3D or 2D projects intended for the Visualization Gallery. Libraries staff are available to offer technology consultations or to present existing examples of compelling work that utilizes this toolkit. Instructions for setting up the camera and using it in a Unity project are provided on the project's documentation website.
How We Did It
Made Possible by the Susan Ellen Everett Internship Endowment
R. O. Everett (‘47) and his wife Edith endowed the Susan Ellen Everett Library Internship in 1999 with a gift through their estate. The internship is named after their daughter, who is also an NC State graduate. Everett served during World War II and was a prisoner of war. He later joined Wachovia Bank and ascended to executive vice president, retiring to Salisbury, NC in 1986. Mrs. Everett served on the Board of Trustees of the Rowan County Public Library and was a member of the Historic Salisbury Foundation. R.O. Everett passed away in 2000 and Edith passed away in 2010. They are survived by their two children, Simon Justus and Susan (both NC State graduates). This undergraduate internship provides extraordinary opportunities for undergraduate students in Electrical and Computer Engineering as interns in the Libraries.
Environmental Scan
An early focus of this team's work was placed on discovering existing avenues for new users of Unity game engine to educate themselves on the software's development paradigms and interfaces. Building a comprehensive educational program for these new operators was beyond the possible scope of our work at the Libraries. The most successful deployment of VisSDK would lean on first-party resources developed by the talented engineers at Unity who maintain this ecosystem and software. By identifying the most current and oft-utilized courses that Unity offers to educate users on the software's dynamics, our team was able to anticipate the core skills that users would possess at a novice, intermediate, or expert skill level.
Behind the Camera
Armed with a more full understanding of Unity's internal teaching tools and the points-of-emphasis that Unity itself prioritizes for new users, our team moved forward into the design and early development portion of this work. The central element of the VisSDK is a pair of virtual camera pre-fab elements that users can drop into a new or existing Unity project to immediately modify the project's specifications to appropriately output onto the Visualization Gallery's hyper-wide screen 360-degree screen. This approach is mirrored in both 2D and 3D formats and allows intuitive reverse-compatibility for existing projects built via the Unity URP while also allowing new 2D or 3D projects to instantly become formatted for the Visualization Gallery.
Develop From Home
When dealing with a uniquely formatted immersive media space such as the Visualization Gallery, developers are accustomed to a familiar problem: in order to test their most recent development, the developer must visit the theater itself and deploy their current build. When a space is popular or frequently in use, this can cause backlogs of developers and creators jockeying for time on a space's calendar.
VisSDK attempts to overcome this limitation through the utilization of a comprehensive in-editor emulator which simulates the behavior of the Visualization Gallery's projector array and gives a live playback of what a 3D project will look like upon deployment in the space. This is a notable accomplishment, and our team was inspired by similar tools developed by extended reality hardware manufacturers like Magic Leap, Microsoft, and Meta. To accomplish this feat, the VisSDK team made clever use of a rendering technology known as Render Textures. You may be familiar with render textures from video games that feature rear-view mirrors, security cameras, or in-game screens like jumbotrons; render texturing allows a defined viewpoint to be applied to a mesh in real time, responding to changes in the scene.
The VisSDK emulator is only applicable to 3D projects in Unity, since 2D projects intuitively display on traditional screens at a scaled-down size. For 3D scenes that wish to include the emulator, users are delivered all of the necessary tools via Unity's package import functionality, a paradigm that nearly all Unity developers frequently utilize.
Deploy and Support
VisSDK is equal parts tools and documentation of those tools. Each feature and component described on this page is heavily documented on the VisSDK's online home. There are also several available demonstrations of these tools created by the aforementioned Jayden Sansom and Elliott Schultz, including a 360-degree deep sea simulation called The Aquarium (complete with procedurally guided schools of fish and charismatic megafauna) and a suite of 2D games focused on a feathered pigeon protagonist named Hank. These demos are important testing grounds for VisSDK improvements and adjustments but also allow the Libraries to equip developers and creatives with known-to-be-working software that features VisSDK and its constituent tools.
Additionally, the VisSDK team worked to convert other Unity projects (that had previously been inaccessible to Visualization Gallery audiences) to formats that could suit the space using the tools within the VisSDK. One such project is the beautiful game created by RJ Washington entitled Mason and the Elegy of Time.
Results
- Visualization Spaces Development Kit for Unity
Team
- Hannah RaineyAssociate Head, Research Engagement
- Colin KeenanExperiential Learning Services Librarian
- Shaun BennettResearch Librarian for Business, Education, and Immersive Pedagogy
