Really interesting piece by Jesse Jarnow at Wired about "upmixing" - technicians who use AI to disassemble classic recordings (isolating the individual instruments that are smooshed together on a master tape) and reassemble them "in crisp, bold ways". It's something that I had long wondered about as the next frontier in sampling - could they find a way to pick out and separate an individual instrument - but at least to my layperson's mind, that seemed impossible. Apparently, I wasn't wrong - most engineers thought it would be impossible. But now some cunning people have found ways to do it. It also expanding the Frankensteinian scope of mash-ups and other forms of recreativity. But more than that it's likely to open up a whole new frontier of remarketing and reconsumption: people being persuaded to buy things they already own in a spiffed-up manner (bit like the radical remixing of Beatles songs done by George & Giles Martin on Love - one of the mid-to-late 2000s retro-events that sparked me to do Retromania, actually).
The term "upmixing" itself reminds me of "upcycling", although it's more of a rhyme than an actual parallel (upcycling is like a bourgeois domesticated version of "salvagepunk" maybe - salvage punk, depunked).
Choice bits from the Jarnow piece:
"Until the 1960s, almost all popular music was recorded and listened to monaurally—all the instrumental and vocal parts were recorded onto a single track of tape and played back through a single speaker. Once a song was on tape, it was basically finished....
".... software and audio engineers work together to transform old recordings in ways that were once unthinkable. Using machine learning, engineers have made inroads into “demixing” the voices and instruments on recordings into completely separate component tracks, often known as stems. Isolating the components of songs is a surprisingly hard problem—more like unswirling paint than using a pair of scissors. But once engineers have stems, they can take the isolated tracks and “upmix” them into something new and perhaps improved."
"the number of apps and tools for splitting songs has exploded, as has the community of academics and enthusiasts that surround the practice. For creators of sample-based music, demixing is conceivably the greatest sonic invention since the digital sampler that fueled the explosion of hip hop four decades ago. For karaoke fans, it’s a game changer. For the people who own the rights to classic but inferior recordings—or enthusiasts willing to wade into legal gray areas—upmixing presents a whole new way to hear the past."
"Go-to industry expert on upmixing" James Clarke worked on the "38-CD Grammy-nominated Woodstock–Back to the Garden: The Definitive 50th Anniversary Archive, which aimed to assemble every single performance from the 1969 mega-festival.... At one point during some of the festival’s heaviest rain, sitar virtuoso Ravi Shankar took to the stage.... The original festival multitracks disappeared long ago, leaving future reissue producers nothing but a damaged-sounding mono recording off the concert soundboard. Using only this monaural recording, Clarke was able to separate the sitar master’s instrument from the rain, the sonic crud, and the tabla player sitting a few feet away. The result was “both completely authentic and accurate,” with bits of ambiance still in the mix, says the box set’s coproducer, Andy Zax...."
"... if the human ear can naturally separate the sounds of instruments from one another, it should also be possible to model that same separation by machine. “I started researching deep learning to get more of a neural network approach to it,” Clarke says. He started experimenting with a specific goal in mind: pulling out George Harrison’s guitar from the early Beatles hit “She Loves You.” On the original recording, the instruments and vocals were all laid on a single track, which makes it nearly impossible to manipulate. Clarke started building an algorithm and trained it on every version of the song he could find—radio sessions, live versions, even renditions by tribute bands.... Over nine months, Clarke sifted through the guitar part a few seconds at a time, virtually hand-cleaning the track phrase by phrase. He listened for stray audio artifacts from other instruments and used spectral editing software to find and eliminate them. For the final step, he set out to recapture the track’s original ambience... As an Abbey Road employee, he could book time in the vaunted Studio Two, where “She Loves You” was originally recorded. He played his track into the room through the in-house speakers and recorded it anew, to capture some of the subtleties of the room’s well-preserved acoustics....
"The first generation of users are demixing and upmixing in creative ways. Some musicians are removing one instrument from a song to create tracks they can practice along to or to generate source material for new music....
" the newest company offering professional demixing services is the California-based startup Audioshake. The company will soon launch a service where music rights holders—both musicians and labels—can upload their tracks to the cloud and, within minutes, download high-quality stems ready for licensing in film, broadcasting, video games, and elsewhere.... Audioshake is also the first company to figure out how to automatically isolate guitars—or, more precisely, a single guitar.... “We refined the architecture of our deep-learning network to be specially tailored to the harmonics and timbre of the guitar....."
