Scope
We have decided to focus our scope on developing tools that will engage the community of music performers and their audience. We seek to augment practice and performance in solo and/or group contexts to better facilitate dialog between performers and their audience, be it in practice rooms, concert halls or streets. While it is challenging to understand the music community as whole, we aim to observe and perform further study on students from the UM School of Music, Theatre, and Dance, especially those in the Performance Arts and Technology program. Insights gained from additional observation and interviews will help us narrow down our scope further during the next milestone.
Preliminary Research and Observations
We selected our target audience and context by conducting a 30 minute brainstorming session. Here, each of us took turns to say one audience or context we were interested in, while one of us noted it down on the whiteboard. This process sparked sporadic conversations, and at the end, it was clear that there were a few topics we had discussed more than others. We further discussed feasibility of access to the audience or place, access to mentors familiar with the topic, and personal interests in or past experience with each topic. Eventually, we narrowed down our context to “people working out at the gym”, and “musicians practicing or playing an instrument”.
Next, we each conducted hour long observations at the Central Campus Recreation Building and the UM School of Music, Theatre & Dance. We conducted several informal interviews with individuals at each site to understand their motivations and the challenges they faced while working out or practicing their instruments. We also looked at YouTube videos to observe group workout sessions like Zumba, Body-weight strength, and Yoga, and conducted a preliminary literature review to better understand the challenges associated with music practice spaces and fitness centers.
From our preliminary research and observations, we discovered the following key insights within each context:
|
Gym
|
|
|
Music Practice Rooms
|
|
We discovered that many of our insights from both contexts overlapped significantly, which led us to develop several key design insights that would guide our ideation process:
- In both contexts, individuals participate in repetitive movements that require accurate posture to prevent injury. How can we augment these spaces to support proper technique when trainers and coaches are not present?
- Both activities require significant physical effort and stamina. How can we increase access for those who might not physically be able to participate in conventional expressions of these contexts?
- Connecting with a group or like-minded community is a key motivator for engaging both populations. How can we connect individuals- who would otherwise be exercising or playing music alone- with others to increase motivation and develop a sense of community?
- Both musicians and athletes perform for audiences, whose reactions can significantly impact the performers themselves. How can we foster dialog between performers and audiences in a productive, engaging way for both parties?
Ideation Process
Based on our individual research and observations, each of us individually listed down our ideas on a shared document. We described the idea, how it will be used, the applicable sensing systems, and the impact of the idea. We then met as a group, took turns to share our ideas, and critiqued each one of them. Using our measure of interest and feasibility, we came to a consensus on 4 preliminary ideas that we sketched, and presented to the class:
|
Smart gym clothing
Sensors embedded in clothes track form, progress, and manage break time. The data obtained could be used in gamification, where users pay with points to customize the appearance of their clothes. These clothes will contain LED lights or similar color changing attachments throughout the fabric, which can be used for self expression, like displaying patterns, expressing opinions through custom text, or visualizing their workout data. |
|
Gym of the future
A room that senses form, enthusiasm, pace of the workout, and progress. It then reacts to these inputs by showing visualizations, changing the lights and sound, and connecting people through shared progress. |
|
Musician–Audience Collaboration
A multiplayer instrument used by street performers that encourages passersby to join in the performance. This system could be networked to connect street performers and passersby in multiple locations. |
|
Virtual music instructor
This tool can serve musicians and healthcare professionals alike to better understand overuse injuries by tracking the biomechanics of fingers, wrists, and shoulders. The system could use computer vision to study the user’s skeletal outline, or utilize inertial measurement units or other sensors to track x, y, z coordinates of target joints or muscle groups. The system could provide verbal feedback on their technique and posture, or take the shape of an exoskeleton-like device that could be used to teach proper technique, correct bad form, or enable individuals of various body shapes and sizes to play instruments they might not otherwise be physically able to play. |
After further discussing these initial ideas among the team, we decided to focus our scope on “musicians practicing, playing or performing with an instrument”. We continued to brainstorm with this scope in mind, and narrowed it down to the following 3 concepts.
Concept Proposals
Concept 1: Instrument that invites contribution from audience
When street musicians play at the corner of the street or at the entrance of a restaurant, people often walk by indifferently. Some individuals may stop and listen for a while, and some may dance with the music, but the interaction between these musicians and their audience can be weak. This kind of relationship not only exists between street musician and passersby, but also exists in many other musical performances. What if we create an instrument which can invite contribution from the audience? This concept aims to enhance an audience’s engagement within a musical performance. It helps create a more interesting and interactive experience for the audience by transferring them from music listeners to music players. We can detect the audience’s motion and transfer the data into new musical elements added to the original music. For example, when a street musician is playing guitar and a pedestrian walks by, his motion gets detected by this smart instrument and an additional drum beating sound is generated. These added sounds are auxiliary so they won't weaken the original music. When the passerby makes different types of movements, the sounds will change accordingly.
When street musicians play at the corner of the street or at the entrance of a restaurant, people often walk by indifferently. Some individuals may stop and listen for a while, and some may dance with the music, but the interaction between these musicians and their audience can be weak. This kind of relationship not only exists between street musician and passersby, but also exists in many other musical performances. What if we create an instrument which can invite contribution from the audience? This concept aims to enhance an audience’s engagement within a musical performance. It helps create a more interesting and interactive experience for the audience by transferring them from music listeners to music players. We can detect the audience’s motion and transfer the data into new musical elements added to the original music. For example, when a street musician is playing guitar and a pedestrian walks by, his motion gets detected by this smart instrument and an additional drum beating sound is generated. These added sounds are auxiliary so they won't weaken the original music. When the passerby makes different types of movements, the sounds will change accordingly.
Concept 2: Smart clothing that enhances the expressions of the performer
Live music performance is not only about the music, but also the presentation and atmosphere. Some music forms, such as classical music, traditionally do not allow performers to enhance the presentation by things other than music, unlike rock concerts where performers jump around and dance. People oftentimes associate classical music concerts as “boring”. We propose a piece of smart clothing that lights up based on performer’s movement and heartbeat, and help illustrate the passion and emotion of the performer. The lighting patterns can be adjusted and defined by performers. When you play solo, it can make your performance more interesting. When used in a classical music concert setting, it can present a light show that breathes and changes with the music. Audience can experience the rhythm of the music and the art of the performers in a new, more robust way.
Concept 3: A music stand that helps with form and feedback to a musician that is practicing alone
Based on our observations, a lot of music school students practice by themselves. Our main insight from one of the interviews was about injuries resulting from poor form during solo practice. How can we provide feedback on form before it is too late? Additionally, how can students get feedback about their progress during solo practice? This concept aims to solve this issue, through a common accessory in every practice room: the music stand.
This music stand consists of a display that shows music notations in large size. The owl bot at the top of the stand consists of a visual and auditory sensing system that can detect the form of the student. The owl also has a small speaker inside it to provide auditory feedback. We think that interacting with and receiving feedback from persona could make for an interesting engagement, perhaps similar to a wise music instructor from a fantasy world. The student will likely be required to stand back so that the visual sensing system could detect the whole body. The display senses what the student plays on their instrument and automatically scrolls the sheet music notation. This allows the screen to display the notation in large enough size that the student can stand back, and also not worry about page turning. Additionally, it could replay the practice session with feedback by the “owl” about form, as well as feedback about the music performance and mistakes marked on the sheet music. The student can then choose to practice specific parts or attempt from the beginning.
