The Ghost in the Machine

00:00:08: Welcome to Blaues Rauschen, the podcast where we explore the intersections between art, music, sound, analog and digital.

00:00:26: Following on from our previous episode, we want to shine the light on artificial intelligence more practically.

00:00:33: Therefore, Veronica Batstorfer and Tim Löhde met to discuss and primarily try out some music and sound generating models.

00:00:43: Veronika Batzdorfer is a postdoctoral researcher at KIT in Karlsruhe, whose work fuses computational social science with interactive and immersive media.

00:00:55: She explores how humans engage with digital technologies, building simulations and environments that probe cognitive biases, digital well-being and the societal impact of emerging tech.

00:01:07: She combines AI-driven behavioral modeling with experimental sound and motion systems to create responsive, multi-sensory performances and VR installations.

00:01:19: Tim Lörde is an artist and musician based in Düsseldorf and Vienna.

00:01:23: In his multi-sensory installations, he combines objects, synthesizers, piano and computers to create spatial sound networks that work with loss of control, materiality and algorithmic dynamics.

00:01:37: His practice moves between composition, performance and installation and explores how sound can be experienced as physical material.

00:01:46: He is interested in the interaction of space, perception and technology, as well as the question of how algorithmic systems can influence improvisation and intuition.

00:01:58: My name is Kira Preuss and I welcome everybody to this episode.

00:02:02: Now, let's listen to Veronica and Tim.

00:02:06: Hi, Veronica.

00:02:07: Hello Tim.

00:02:08: The next episode has the title Ghost in the Machine.

00:02:12: What was the idea for the title, Veronica?

00:02:15: The title actually came from two places.

00:02:18: One is a very specific audio project by Ryan McGuire, which I stumbled upon a long time ago.

00:02:24: And the other one is the question of what really happens in our brains when we listen to music made by AI.

00:02:29: And so for the first project... It's called The Ghost and the mp-free.

00:02:33: and what the artist really did was he took a song and he subtracted the mp-free version from the original uncompressed recording

00:02:42: and what

00:02:43: was left is everything the mp-free codec threw away.

00:02:47: So this is what he called this ghost track and it's quite haunting.

00:02:51: And so in this piece you get phantom harmonics, the underwater sort of, some sort of artifacts, and it's quite the sound of perceptual bias encoded in some form of standard.

00:03:04: And the Gaia basically excavated that deletion.

00:03:07: So it's the Piferic metaphor, a ghost, what is what is raised by a model of human perception.

00:03:14: Yeah, so how is that different when AI creates the music?

00:03:19: Are we dealing with a new kind of ghost in the machine?

00:03:22: Here's actually when it connects to AI.

00:03:24: Today we're just not only removing... information.

00:03:28: we are generating it from scratch and so just raises this question of a different kind of ghost in one way or another.

00:03:34: and when you look at newer science theories like this really robust finding that our brain just doesn't only process sound it simulates the gestures that made it so with our mirror neurons to fire as if our fingers are hitting those keys.

00:03:49: And so we don't really just hear music, we feel the body that produced it.

00:03:54: And this gives the music some sort of presence.

00:03:57: And so the question now with AI is, so what happens if there's nobody really involved when AI generated those trumpet lines?

00:04:05: Our mirror neurons still fire, but it's all the phantoms who become like disembodied and disconnect.

00:04:11: And the question that this podcast really asks is, framed by the ghost and the machine, can we still trust our ears?

00:04:18: And when we feel and be moved by music, are we connecting or is this just some statistical pattern matching?

00:04:27: A ghost is, of course, also referring to some kind of soul.

00:04:33: And our question today is also, can AI create authentic music?

00:04:40: But we don't want to answer it with words.

00:04:43: We met today because we want to try it out, to use some tools, some generators.

00:04:50: And we want to find out if it's even possible to make experimental music with it.

00:04:55: Therefore, both of us will record our own voice and create an improvised short sample that we will load into various tools and programs.

00:05:05: So, first a sample.

00:05:07: One, two, three.

00:05:16: For my side, I try out the free version of Zuno.

00:05:20: perhaps the most well-known music generating program right now.

00:05:24: It's very user-friendly and was actually recently sued by GEMA for using licensed music to train its models.

00:05:32: And also what we found today was Adobe Firefly that is usually used for image generation, but it can also generate sound.

00:05:43: I use sound synthesized by Reef which stands for real-time variational autoencoder and Reef in a nutshell is a trained neural network audio model.

00:05:54: that learns this very low dimensional representation of sound and decode samples from specifically that representation into continuous audio waveforms.

00:06:07: So in a nutshell, it does not really replay previously recorded material verbatim, but the output is synthesized from the model's learned statistics of

00:06:16: sound.

00:06:17: So, first it's Zunos turn.

00:06:20: I will upload our sample and use the following prompt.

00:06:24: Experimental track, long textures with supple synth and effects weaving in, slow builds featuring metallic percussive hits that punctuate long and ambient drones.

00:06:39: And then I hit the create button.

00:07:12: For the second experiment, we used the rave model in Python.

00:07:17: So we load first off some general purpose dependencies like the lab browser or torch libraries.

00:07:23: And then finally, we load in the parameters of these pre-tank model.

00:07:27: And specifically, the model is called vintage, which was trained on around eighty hours of vintage music.

00:07:35: And in the next step, more or less the algorithm encodes and decodes the audio signal.

00:07:50: Okay, then I will give Adobe Firefly a try.

00:07:54: So here we use our sample to control our timing and the modulation.

00:07:58: And we use a very short prompt, experimental, slow build and drones.

00:08:15: So these were our model outputs from Zuno, Rave and Adobe Firefly.

00:08:21: And Mironika, you and me, we have many thoughts about this.

00:08:25: With the results by Suno, it's interesting because the beginning of the song is... I would say it has some experimental vibe, but the longer you listen to it, it's getting more and more into some dubstep or pop or techno.

00:08:41: So you can really see that this model is trained by the mass media, the music that is from the mass media.

00:08:49: Yeah, I think that makes it very difficult to try some experimental music with it.

00:08:54: and Adobe Firefly, the other engine.

00:08:57: It has a different approach because I think it's made for creating sound effects for movies, I guess.

00:09:03: Therefore, I think the output is more interesting because you can create really new samples that you can actually use again to make music with it.

00:09:13: And now you will only go, what is about rave?

00:09:15: What do you think?

00:09:16: When we passed our sound through rave, the model didn't invent something from scratch.

00:09:21: It reshaped our sound using statistical patterns.

00:09:24: And I think you can see this in the sample for these are all things that are based on previous recordings.

00:09:31: So the structure still carries our human decisions, but the texture reflects patterns from the training data.

00:09:38: So we end up with this strange I think in between object where it's human in structure, but machine shaped in texture.

00:09:46: And these are interesting questions to ask because if the form is ours, but this surface is learned, who is the author really?

00:09:53: and can we say that these are our results?

00:09:56: Yes, and what does it mean for us as a musician, as an artist, as a scientist?

00:10:02: To connect to this?

00:10:03: at the end of the day, it still depends on human intention and as we've seen in our little experiment, models like RAVE are tools.

00:10:11: not replacements, ways to explore, siren to experiment with textures and style, but we still need to be aware of the agency aspect.

00:10:21: So when we engage with them, we embody the music, the gestures, our timing and our imagination.

00:10:28: And I think we are here in a discussion that we already had before when drum machines came to the market or when Sangly was possible to use samples or sampled instruments.

00:10:39: I think the authenticity is created by the musicians themselves because they will use new AI created samples or the music and they have to edit it, they have to work on it.

00:10:50: Once you mentioned the term black box models, so these models like Suno, where even the company doesn't know what exact material was used for training, maybe this is our conclusion that the authenticity lies in the future, where artists and musicians create their own models, or where they just use AI, create a material for their own projects.

00:11:16: Of course, anyone can generate basic music with AI by clicking a button, but creating music that challenges our perception requires deliberate artistic choices and deeper engagement.

00:11:28: And this is the big difference between content generation and artistic creation.

00:11:33: You've been listening to Blaues Rauschen, the podcast.

00:11:36: A big thanks to our guests, Veronica Batstorfer and Tim Löde for opening up the black box of AI-generating models and reminding us that the ghost in the machine still depends on human intention.

00:11:49: You can find more information and episodes on our website at blauesrauschen.de.

00:11:55: Thanks for listening and see you at the next festival.

00:11:57: Blue

00:11:57: is Rauch.

00:11:58: Rauch.

00:12:04: Rauch.