Sheet Music
Senior Technical Designer, Feature Owner
Ubisoft, Rocksmith+ (2024), Snowdrop Engine
In order to meet our authenticity goal for the release of piano, it was essential to create a sheet music view.
This feature presented an extremely challenging mix of technical problems and an ability to guide engineering, as well as a litany of design problems, which is why I was asked to own it. My work on Sheet Music particularly highlights my ability to thrive in spaces I have limited background in (I did not know anything about sheet music going in), as well as my leadership skills.
Sheet music is a highly subjective language that varies widely across cultures and music philosophies, even within our team of 25+ professional musicians it often brought disagreement on which direction we should take. A large part of my role as Feature Owner was resolving these conflicts by collating the insight of all parties and guiding production towards our design/product goals.
Uniquely challenging
An ancient language…
with infinite ways to write the same thing…
that must be legible across all screen sizes…
Rocksmith data was not designed to be visualized in a sheet music format. The Rocksmith Noteway visualizes notes in 3D space (time == space), while in authentic sheet music, spacing between elements has no relation to time played.
For example, each measure seen here is played for the same amount of time, despite widely varying in width.
Pre-production
— Leading a task force of 2 engineers —
Early experimentation
Some experiments I made exploring how we could make different elements scalable to accommodate the wide range of device sizes.
Sheet view needed to be generated from the existing data procedurally in order to support our library of thousands of tracks. We essentially had to write a new noteway system that could translate existing data.
Guiding a team of 2 engineers I was able to prop up architecture for a system that would translate notes into individual elements, abstract coordinates for placement, and draw them using our UI node system.
Production
— Onboarding 45+ devs to the team —
As Feature Owner, it was my responsibility to onboard the rest of the dev team - conveying the vision and goals for the final polished feature.
I headed up an intensive performance pass; the rendering needed to be moved out of the UI System. UI nodes were the most logical tool to allow for rapid iteration, but they were tanking performance on mobile. Rendering had to be moved to shaders to reduce draw calls.
In addition to working with the art team on custom shaders, I worked with Art/UIUX to develop a final visual treatment and achieve the ‘premium feel’ we strive for as a product.
We also needed to account for the many menu changes and HUD elements that needed to be adjusted in order to accommodate this new noteway view.