Entertainment
The Future of Music
In this episode of 'The Future of Music,' Steve Mencher interviews Todd Macover, a pioneering figure in music technology and education. They explore the intersection of music, health, and te...
The Future of Music
Entertainment •
0:00 / 0:00
Interactive Transcript
spk_0
tek
spk_0
It's a touch, it's a touch.
spk_0
Welcome to another one of our music and the brain podcasts from the Library of Congress.
spk_0
I'm Steve Mencher.
spk_0
Today we're joined by Todd Macover, one of the most original thinkers in the world of music
spk_0
over the past several decades.
spk_0
And especially noted for his ability to lead the way in creating musical opportunities
spk_0
for non-musicians to experience the joys of making music.
spk_0
His mentors include the most important composers of the 20th century, his collaborators
spk_0
or musicians like Yo-Yo Ma and Prince, and he operates from a base at MIT, where he's
spk_0
currently professor of music and media at the MIT Media Lab, and director of the lab's
spk_0
hyperinstruments and opera of the future groups.
spk_0
I could go on, but then we wouldn't have any time to talk.
spk_0
So welcome, Todd.
spk_0
Thank you, Steve.
spk_0
Now I was going to do the first interview with you in recent times that didn't even mention
spk_0
guitar hero, but I'm sorry I couldn't pull it off.
spk_0
So perhaps you can use that game and its invention to tell me a little bit more about hyperinstruments
spk_0
and the media lab.
spk_0
And then we're going to go quickly to where we're going to concentrate, which is music
spk_0
and health, music in the brain, music and Alzheimer's, and the other ideas we've been talking
spk_0
about, with a lot of smart people here at the library for about two years.
spk_0
But let's start briefly with guitar hero.
spk_0
Good, so quickly, quickly to get there.
spk_0
I was realizing today that I was talking to Ann McLean, and it turns out it's been 27
spk_0
years since I've been in this building.
spk_0
The last time I was here was to premiere a piece for cello and electronics that I wrote
spk_0
for Joel Krasnik, that was commissioned by the Library of Congress.
spk_0
The reason I think of that is that I'm a cellist myself, grew up playing classical music
spk_0
and rock music, my dad's a computer scientist, my mom's a pianist.
spk_0
And I got interested very early in extending what you could do with traditional instruments,
spk_0
especially something like the cello.
spk_0
And I was thinking that this piece I premiered here 27 years ago was a piece for cello
spk_0
and electronics.
spk_0
And I used my imagination to extend the sounds of the cello, do things that the cello acoustically
spk_0
couldn't do, bring these two worlds together.
spk_0
But at that time, the only way to make the electronic part was to go into a studio I was
spk_0
working at Eirkham Pier, Boulez's Institute in Paris at that point, and put all of this
spk_0
music pre-recorded so that I could set it off at certain times to go along with the cello.
spk_0
And that's the best you could do at that point.
spk_0
And I remembered that when I was a teenager, in many ways the reason that I ended up doing
spk_0
strange instruments and ending up with guitar hero was, it all goes back to Sergeant
spk_0
Peppers when it came out.
spk_0
And if you think of how amazing that was musically, it's incredibly like technical or all of
spk_0
a sudden, amazing music, complex, rich.
spk_0
And it was also the first album that the Beatles made that couldn't be played live.
spk_0
It was made with all the wonders of technology in the studio.
spk_0
And not only that, they'd gotten tired of playing live just as Glenn Gould did it about
spk_0
the same period, fell in love with the studio.
spk_0
And from the time that I was in, you know, 12, 13, I loved that music, but also it felt
spk_0
like something was wrong.
spk_0
Something was missing from music that was only done for the studio could never ever be played live.
spk_0
So I've been obsessed ever since with how you have to get the best of both worlds,
spk_0
how you can have music which is spontaneous and expressive and in exchange between musicians
spk_0
or musicians and audience.
spk_0
Also has this richness and complexity that you can only do with the studio, with technology.
spk_0
And that led me to invent hyperinstruments which are basically instruments originally to be
spk_0
played by Virtua Ose, like Yo-Yo, Ma and Prince, as you mentioned.
spk_0
But where the instrument listens to how it's being played.
spk_0
And it not only knows what's being played, but it knows how it's being played and why.
spk_0
It knows what the expression is, what the feeling is.
spk_0
And it uses that information so that if a performer emphasizes certain things, pushes to a downbeat
spk_0
or plays an F sharp with a certain twinge, the instrument can morph and turn into something else.
spk_0
The cello can become a voice or an electronic sound or a single melodic line can become a whole orchestra.
spk_0
So I got very interested when I got to MIT in building this kind of instrument.
spk_0
And because of my background being interested both in classical music and rock, as we started
spk_0
developing this technology and people like Yo-Yo were doing amazing things with it, I also
spk_0
started thinking that the same technology could be used to measure what anybody does when they
spk_0
react to music.
spk_0
And I've got very interested in opening the doors of participating in music to anybody.
spk_0
And it turned out that the software we were writing that could listen to Yo-Yo's playing and expand it.
spk_0
And the way we were measuring his gestures, so he was making gestures with a bow.
spk_0
But it turns out the techniques we were using could measure anybody's gesture waving their hands.
spk_0
And so you could conduct music.
spk_0
And I made a project called the Brain Opera in the mid 90s for Lincoln Center in New York that was
spk_0
a kind of orchestra for anybody to play.
spk_0
And it turned out that some of my students who worked on that Brain Opera project really got hooked
spk_0
by this idea of making sophisticated, interesting musical experiences that could be touched by anybody.
spk_0
And they graduated, went down the block.
spk_0
There's still a few blocks away from the Media Lab in Cambridge.
spk_0
And they started this company for about 10 years.
spk_0
They made music games that were popular critically, but didn't sell very well.
spk_0
And as soon as they decided to make a plastic guitar, make this kind of guitar soloist model,
spk_0
and to make a game where you could very easily tell if you were progressing in the game or not,
spk_0
it was amazing because it was a skyrocket overnight success without any marketing.
spk_0
So to me that said, not only were they very clever, but the time had come where you really,
spk_0
first of all, people did want more from music than just listening to an iPod or
spk_0
listening to passively to a piece.
spk_0
People were willing to jump in and make music themselves.
spk_0
If you could make the right model, that was one model and it kind of opened a field.
spk_0
I want to find a way to move from there, which is the thing that people will have heard about you.
spk_0
I mean, I've known about you for a decade.
spk_0
I know how important you are to music, to the intersection of music and technology.
spk_0
I understand that.
spk_0
And yet I know that people who don't follow music may not have heard your name, may not know about you,
spk_0
except for this fabulously interesting, giant pop culture thing called Guitar Hero.
spk_0
But I want to find a way now to move from that to some of the other work you're doing,
spk_0
because that's my interest here, is the intersection of the music and the health.
spk_0
And so let me tell you about our last visitor here, who was just amazing, a woman named Alicia Claire.
spk_0
She's from Kansas and she's been working for several decades now in the intersection of how music is used for Alzheimer's.
spk_0
Mainly in late stage Alzheimer's patients.
spk_0
And she told us in great detail about the idea that sometimes when communication has stopped between people,
spk_0
as a person sort of disappears from their Alzheimer's disease,
spk_0
there's one thing that can bring them back.
spk_0
And that thing, I'm sure you know, is music.
spk_0
And so in reading that you are starting also to do some work with Alzheimer's and music and memory,
spk_0
I wanted to go directly to that and to find out exactly how your work with Alzheimer's is proceeding.
spk_0
In many ways, something like Guitar Hero is extremely promising,
spk_0
because it does allow everybody to participate.
spk_0
But at the same time, it's very limiting.
spk_0
And it's almost frustrating because of how much it does, but also how much it doesn't do.
spk_0
So we, over the last five or six years, got very interested in trying to do everything that Guitar Hero doesn't do,
spk_0
which is making interactive music environments that are much more personal, much more expressive,
spk_0
and also environments where you can make your own music.
spk_0
Because my own sense is that exploring creativity through music and simply having a powerful creative experience
spk_0
is central to motivation, well-being, and directly to health in many ways.
spk_0
So we started out by making a tool called HyperScore, which originally was for children.
spk_0
And it's a graphic software program that makes it possible for anybody to compose original music
spk_0
without knowing any of the rules of music or having any musical training.
spk_0
You make little motives and melodies with little symbols that look kind of like eggplants, you move them around.
spk_0
And each of those gets a color, and then you draw with those colors.
spk_0
And the shape of your drawing does a kind of variation on your melody, harmony, or rhythm.
spk_0
And then all the rules of music are built into this environment.
spk_0
So you can draw these things, you can leave them exactly where you place them,
spk_0
but you can also turn on harmony, for instance, and say,
spk_0
now I want to harmonize this, and I'm going to use another line to show how the harmony changes.
spk_0
Does it modulate? Does it move to different cores? Does it increase tension?
spk_0
So we had a lot of experience in 2004, 2005 with HyperScore and Children.
spk_0
We went around the world having children compose first pieces for symphony orchestras,
spk_0
like the Berlin Philharmonic, the Boston Symphony.
spk_0
And this was a computer software, but you could push a button.
spk_0
And this graphic notation turned into real notation for players.
spk_0
So it was real musicians can and still do play this.
spk_0
We had such amazing results with children that we just occurred to me that
spk_0
inviting anybody to create their own music and to have it played by professionals and have it
spk_0
mentored and share it with lots of people could open doors to many people who could benefit by
spk_0
making music. So I started thinking about people with various illnesses and people who
spk_0
maybe were in long-term care in hospitals who weren't really invited to do anything creatively.
spk_0
And this was five or six years ago. I'd also heard quite a bit about, especially about the
spk_0
relationship of memory and music and Alzheimer's. The same thing is true with Parkinson's disease.
spk_0
Many people who can't move because of Parkinson's. If you find the right kind of repetitive music,
spk_0
it's as if parts of the brain get synchronized that don't get synchronized any other way.
spk_0
It also seems to simplify circuits in the brain which get overloaded and all of a sudden,
spk_0
you can move over a threshold, a doorway for instance. Same thing with strokes.
spk_0
Many people who have strokes and lose the ability to speak.
spk_0
Most people still form words in their heads and any one of a number of pathways get destroyed
spk_0
or blocked. And you can imagine the words but you simply can't say them.
spk_0
Many of those same people can sing a song with the words perfectly but they can't translate that
spk_0
back to speaking. So actually one of my colleagues at Beth Israel Hospital Harvard Medical School
spk_0
in Boston named Gottfried Schlaug. He's at Harvard but his technique is called MIT, melodic
spk_0
intonation therapy. He found out that repeating words rhythmically and alternating each syllable
spk_0
with an interval and using a hand tap reinforces the ability of re-learn through sound.
spk_0
But we started out using hyperscore as a technique for people with many health problems to
spk_0
compose. Especially at a hospital called Tootsbury State Hospital outside of Boston.
spk_0
For five or six years now we've been working with groups in the mental health division and also
spk_0
the physical disability division. And I've had unbelievable results with people whose psychological
spk_0
conditions their symptoms are reduced enough that they set of staying for years and years in the
spk_0
hospital are able to go back to normal life. People whose physical conditions are at least
spk_0
brought under control when they're working with hyperscore. And so we have a long term kind of colleague
spk_0
named Dan Elsie who has cerebral palsy since birth. He's been a kind of collaborator partner to
spk_0
think about how both composing with hyperscore and then performing pieces that you've composed
spk_0
using special interfaces that we design can be really the only way for many people to show who they
spk_0
are and what they care about through music. Now in Alzheimer's we are in contact with and are
spk_0
very interested in therapies that use exactly the phenomenon that you mentioned the retention of
spk_0
connection to music into late stage Alzheimer's. But we've been interested in exactly the opposite
spk_0
because we're interested in music as an assessment tool especially through these composition activities.
spk_0
So one of the great things about music is that people simply love it. So people are willing to
spk_0
engage in a musical activity over and over and over again without being forced to. So if you can
spk_0
find the right kind of activity quite simply it's a very good way of getting data over over a
spk_0
long time with somebody. Music also uses so many different parts of the brain, coordinates
spk_0
different parts of the brain. You can isolate different parts of the brain depending on whether
spk_0
you're dealing with rhythm or melody or texture or form and so you can observe different mental
spk_0
capacities. What we've been concentrating on with Alzheimer's is whether music is a powerful
spk_0
early detection tool. It's exactly the opposite of late stage Alzheimer's because as we know with
spk_0
Alzheimer's by the time people show symptoms it's too late for any of the treatments that can slow
spk_0
down Alzheimer's. Of course there's no cure for Alzheimer's but we want to detect it as early as
spk_0
possible so we can slow it. So we developed a set of tools, there are two categories. One of them
spk_0
is like a music memory game and it's an application on an iPad or an iPhone and you see a bunch of
spk_0
circles on the device and you hear music in different styles and it's almost like what's that game
spk_0
with return over cards and concentration. It's almost like a musical version of concentration
spk_0
and we've done many tests about what kinds of music, what juxtapositions, how different the music
spk_0
should be. One of the interesting things we found out early on in our studies is that amazingly
spk_0
musicians and non-m musicians on this particular kind of test do rather similarly which is very
spk_0
useful but it turns out that what are called associative pairs for memory, memorizing not just a fact,
spk_0
not just a sound but a sound across media. So a sound and its position in space, doing that together
spk_0
access is a certain part of memory which is one of the very first parts of memory that deteriorates
spk_0
in Alzheimer's disease and so it turns out that this particular test, this test is in clinical
spk_0
trials now at McLean Hospital in Boston and that's been funded by the Alzheimer's Association
spk_0
and it turns out that it looks like it's more effective than just about anything else in finding out
spk_0
that this memory capacity is deteriorating. It doesn't even take all that long but over a
spk_0
course of a couple of weeks we find out that there really is a problem and then can probe more deeply.
spk_0
What we're doing right this second is taking this hyperscore composing language so it's nice to
spk_0
have this app on the iPod and your iPhone but it's really really nice to be composing your own
spk_0
music to use this hyperscore software so we're building right into hyperscore the ability to do these
spk_0
memories with the own your own music that you're making so you make music you'll make one of these
spk_0
melodies and without you asking all of a sudden a little test will pop up and say nice melody let's
spk_0
remember that so is this similar is that not similar and so your own music becomes the subject for
spk_0
this diagnostic tool so we believe very strongly that music as a diagnostic tool is very very powerful
spk_0
and we have this application in Alzheimer's and we're we're going to pursue it vigorously and it
spk_0
seems like it's it's it's already very successful and we're starting to publish I have a fantastic
spk_0
student one of my favorite people in in music in the brain named Adam Boulanger a name worth remembering
spk_0
he just just finished his PhD at the media lab and he's doing a postdoc this year what we're doing
spk_0
for the postdoc is doing all the tests to find out to accumulate the data to show how well this works
spk_0
how obviously how we can make it better but also we want to scale this we want to get these tests
spk_0
out to people and we want to have enough data to show that it's it's it's a test worth using
spk_0
as widely as possible because we think it can really help people now you've talked to us about
spk_0
something I guess that I would call for lack of better words looking at these issues from the outside
spk_0
in so how are you interfacing with brain scientists who are doing the FMRI work to who are looking at
spk_0
this from the inside out electrically and chemically and what's happening in the brain during these
spk_0
things or are you well we are luckily in Cambridge there's an unbelievable community of
spk_0
musicians and music therapists and brain scientists and brain imaging people just about every and
spk_0
and the community is getting closer and closer together so people approaching these ideas from
spk_0
every possible angle are are in close contact my own sense is that FMRI work is extremely useful for
spk_0
long-term research for understanding simply how the brain operates in various music
spk_0
situations so for instance my two daughters were part of Gottfried Schlags the first longitudinal
spk_0
study of how children's brains do or don't change if they study music instruments or not so he
spk_0
did a I'm sure you know he did a five-year study to see whether somebody's starting an instrument
spk_0
over five years had any significant change in brain development from a control group of people who
spk_0
had music once a week in school and then people who had no music this is very very interesting we
spk_0
think that maybe more real-time EEG measurement for instance we're doing a lot of work with somebody
spk_0
named Pavon Sinha at the Brain and Cognitive Science Department at MIT who's one of the great
spk_0
brain plasticity experts actually his field is brain adaptation after sight is restored
spk_0
but he's extremely interested in the visual arts and in music as well and he's doing experiments with
spk_0
simply how to pull better and better information out of EEGs EEGs are much faster they're much more
spk_0
immediate and if you want to look at how somebody's actually responding to a piece of music and
spk_0
what happens while they're performing or while they're composing my senses that will learn more
spk_0
faster with EEGs. Now a librarian here at the Library of Congress some years ago a fellow named
spk_0
Sam Burlovsky laid out the idea it was back in the 90s I think that we were headed toward a future
spk_0
where any music that existed in recorded form would be available to us in a sort of universal
spk_0
jukebox accessible to anybody at any time and I've been amazed at how the future seems to be
spk_0
catching up to his prediction ever since I wonder if this has any interest to you or any relevance to
spk_0
your work. Something that interests me more than having a billion pieces on your iPod is the idea
spk_0
of having a piece of music which which itself is maybe an unfinished form when you get it so a
spk_0
piece of music which might be partially created by whoever you know by yourself by by a famous artist
spk_0
whoever and and it's delivered to you in a form that is meant to be either constantly played out
spk_0
in different ways just on its own or perhaps constantly played out because it picks up other
spk_0
sounds from your environment or picks up something you're actually doing. I think that's one of the more
spk_0
interesting futures music which comes to you as as a kind of seed that is meant to be realized in
spk_0
different ways. It's like a recipe and you can mix different things into the recipe. There's a
spk_0
small company in England called RJDJ which you know it's kind of like a tar here. It's pretty cool.
spk_0
It's not all that great but it's going to change the world I think. They make a set of environments
spk_0
mostly for mobile phones and this is exactly what they do. They commission young artists to make
spk_0
pieces of music that are designed to play all day and constantly morph but they're also designed
spk_0
until two days ago what their stuff did is you downloaded to your iPhone and then it's very
spk_0
cool because instead of actually composing with it or playing with it it was designed just for you
spk_0
to walk around with your iPhone and so if you walked around and you were running it influenced the
spk_0
music in a different way if you turned it here and there and it was great because it was all designed
spk_0
so that it wasn't meant to be conscious. You weren't using an interface to change the music.
spk_0
What they released two days ago is something called Inception the app which is shot to like the number
spk_0
one app on the iPhone overnight and it's against not all that great but it's pretty interesting
spk_0
because what they call it is soundtrack to your everyday life and so what this does is it has it
spk_0
actually takes the soundtrack and bits of music by Hans Zimmer to Inception and then it measures
spk_0
how you're walking it measures the sounds in your room and it pulls all those in it makes them part
spk_0
of the music you can talk into it you can sing into it it takes any sound that you make and mixes
spk_0
it right into the music and makes it part of the experience and it does it in a very clever way.
spk_0
It's not Beethoven but it's you know people like it because it's done extremely well
spk_0
and so this is an idea of not only music that you shape but it's music that's
spk_0
also taking in your everyday life and reflects your everyday life and I think if we talk about
spk_0
brain science and music I think the the far extension of this idea is that 10 years down the road
spk_0
five 15 years down the road there's no reason why a musical environment like this can't measure
spk_0
not just how you're shaking your iPhone but can measure your facial expressions and
spk_0
biometric signals like your muscle tension and and the viscosity of your of your hands
spk_0
and measure your brain signals in all kinds of ways and because then there's really a kind of
spk_0
personal music where the music maybe music somebody else composes maybe it's something you made
spk_0
uh is truly adapting what what does it mean to adapt it can be mirroring the way you're feeling
spk_0
right now it can be compensating it can it can see the fact that you're dozing off or that you're
spk_0
depressed or that or that you're agitated and and either because of the particular
spk_0
piece or program you you choose or because of a button you push it's pulling you back to some
spk_0
different state it's it's a really I can imagine and not only imagine but one of my intentions is
spk_0
to help push forward a personal music that truly adapts to who you are at this very moment because
spk_0
of your musical experiences and personal experiences and your emotional and physiological state
spk_0
and and this kind of intimate relationship between a composition and you're being at a particular
spk_0
moment I think is one of the major places music is going before we leave I just want to ask you
spk_0
you know what you're working on right now that you're particularly excited about well I just
spk_0
finished you know one's always most excited about what they just finished this thing that I just
spk_0
finished is is a bohemith it's something that took me about 10 years so I'm really glad to be done
spk_0
but it's actually a piece which dramatizes and attempts to put audiences in a world similar to
spk_0
what we just discussed so it's a piece it's an opera called Death in the Powers and very briefly
spk_0
it's about a man who doesn't so much want to live forever he's 60 it late 60s rich powerful successful
spk_0
a little creepy a little bit like Bill Gates meets Howard Hughes meets Walt Disney and and he's
spk_0
actually tired of the world he wants to leave but he wants everything about himself to remain so
spk_0
he wants his memories and his and his and his ability to communicate with his loved ones and his
spk_0
ability to manipulate his businesses to stay so he figures out a way to download himself into his
spk_0
environment and he splits and the entire stage comes alive with him and so first of all the
spk_0
singers off stage so we have to measure exactly the kind of things I'm talking about we measure
spk_0
his singing and his and his muscle tension and his gestures and things he's conscious of and things
spk_0
he's not conscious of and these well up onto stage and the whole stage visually and musically
spk_0
becomes this person not in a literal sense but in a kind of emotional sense you know he's there
spk_0
and everybody who's left his wife and his daughter and his associates have to decide is this really
spk_0
him and they have to decide you know musically and visually do we want to relate to him like this
spk_0
and is something missing and actually is this a form this is kind of legacy that is made this
spk_0
thing that he's left of himself is this so powerful and beautiful is this a place that they want to
spk_0
go to do they want to leave the world and live in this system live in this other form and so it's
spk_0
really a question first of all about what it means to represent each of our lives what we can
spk_0
communicate what we can leave for other people in a living form is music a good way to do that
spk_0
what do other people think about this do they want our legacy left in this form each person in
spk_0
the opera reacts differently the daughter is the one who decides that or well I won't I won't tell
spk_0
you what she decides but she's the one who struggles with the idea of remaining human or going into
spk_0
this other form and the opera ends with a finale kind of tug of war between the two of them he
spk_0
trying to pull her into the system and she trying to to stay so we premiered this in Monaco in September
spk_0
and it comes to Boston in March Chicago in April and be great to have people come and say it it's got
spk_0
robots and moving walls and things you don't usually see in opera he's the recording we're listening
spk_0
to now from the Monaco performance yes it is all right let's listen a little bit
spk_0
my miss having a father
spk_0
I can hear the whistle I am someone's child I want at least something like a mother
spk_0
something a flesh and blood I miss having a father a flesh and blood I need to touch my mother
spk_0
I've been talking with Todd Macover the professor of music and media at the MIT media lab and
spk_0
director of the labs hyper instruments and opera of the future groups thanks a lot for joining us
spk_0
Todd thank you Steve this has been another one in our series of music and the brain podcasts from
spk_0
the Library of Congress I'm Steve Menture
spk_0
B.