Shoreline. Photo by Alex Shapiro.


 

 

AI AND IMMERSIVE MEDIA
IN MUSIC PERFORMANCE AND EDUCATION

 

 




Quick links on this page

 

 
 
 
 
Composer Alex Shapiro in her studio on San Juan Island, WA, February 2024.

 

 

WHAT'S THIS PAGE ABOUT?

 

 

 

 

Greetings! This resource collection includes well over 100 of the gazillions of links that I read through while preparing my March 2024 presentation for the CBDNA Western/Northwestern Division Conference in Las Vegas, Nevada titled, REAL TALK ABOUT THE VIRTUAL WORLD: THE FUTURE OF MUSIC PERFORMANCE AND EDUCATION. Since then, I continue to update and add to these links.


I put this companion page together with the intent for it to be a helpful, if very random, starting point for anyone interested in learning more about the newer tools available to us as music-makers.


This is by no means intended to be an exhaustive list; an AI-driven search engine like Perplexity will go far deeper! But I hope that wandering through this page and clicking will further spark your curiosity and creativity as it did mine, as each link leads you down a fascinating rabbit hole of discovery!


The multimedia presentation is a dynamic one-hour overview of the uses of AI and immersive media in our field. It demystifies the concepts behind the many new terms we hear every day, and looks at the past 30 years of advances that led us to this point. I explore how the technology of the present and of the near future impact our creative lives as music-makers and educators, balancing excitement for the new opportunities with caution for the abuses that accompany them. If you're interested in seeing it, and/or having my lively, non-generated self offer it either in person or online, please drop me an email.

Have fun clicking around this page!

Alex



 


 
 

 


RESOURCE LINKS TO EXPLORE

Click any cheery yellow link to dive in!

 

 

Eagle. Photo by Alex Shapiro.
TERMINOLOGY
VIRTUAL REALITY

 

 

There are three main categories of virtual reality simulations:
non-immersive, semi-immersive, and fully immersive.

The umbrella term for all of these is XR: Extended Reality.
This encompasses:


AR: Augmented Reality.


MR: Mixed Reality.


VR: Virtual Reality.


 
ARTIFICIAL INTELLIGENCE

 

 

There are various types of Artificial Intelligence,
and the distinctions are important to understand:


Narrow AI, also known as Weak AI, or Artificial Narrow Intelligence (ANI), is what's used by apps like Siri or Alexa— they're not creating info, just serving it up to us.


Generative AI (AGI), on the other hand, can produce various types of content, including text, imagery, audio and synthetic data.

Agentic AI, focuses on autonomous systems able to make decisions and perform tasks with limited or no human intervention.


Artificial Superintelligence (ASI), is capable of surpassing human intelligence by developing thinking skills of its own. It's not in full swing at the moment, but as your imagination can conceive, this is the stuff of dreams and nightmares for the future. Remember "2001: A Space Odyssey"? "Open the pod bay doors, HAL." …. "I'm sorry Dave, I'm afraid I can't do that."


 
HOW BOTS LEARN

 

 

Machine Learning (ML): focuses on the use of data and algorithms to enable AI to imitate the way that humans learn, gradually improving its accuracy.


Natural Language Processing (NLP): enables computers to understand, interpret, and respond to human language naturally. Examples of this would be voice recognition and grammar-checking software.


Large Language Model (LLM): neural networks that learn from enormous amounts of digital text, including books, Wikipedia articles, chat logs and other information culled from the internet.


Neural Network: technology that digital assistants like Siri and Alexa and Chat GPT4 use to recognize spoken commands, and that self-driving cars use to identify pedestrians.




 
Two foxes. Photo by Alex Shapiro.

APPLICATIONS


This is only a random set of links to get you started!
Alex has NO affiliation with any of these companies,
and they are listed in no particular order.

This page is regularly updated and expanded!
Please email Alex to suggest additions or changes.

TEXT and INFORMATION

 

Search for, "Lyric generator"

Grammarly

OpenAI ChatGPT4

Perplexity

Google Gemini

Microsoft Copilot

 

IMAGES

 

Generative:


Open AI DALL-E (included with OpenAI subscription)

Davinci

Midjourney (via Discord)

MiraMuse AI

Adobe Firefly

Night Cafe Studio

OpenArt

Clipdrop

DreamStudio

Stability AI

Bing Image Generator

Gemini



Image and video and more:



Based Labs

Canva "Magic Media"

RunwayML



Image enhancing:


Remini

FaceTune

Pixlr Editor

 
VIDEO

 

PIKA

Invideo

Lumen 5

Crayo.ai

Clipdrop

OpenAI Sora

Deepmotion (text to animation)


9 Best AI Video Generators (April 2024)


For an interesting look under the OpenAI vid-gen hood, click HERE.


And if you're curious about how to record green screen video, click HERE.

 
TEXT TO SPEECH

 

ElevenLabs

IBM Watson, and a history of AI since the 1950s.

NaturalReader

 
MUSIC ED

 

MuseScore Studio

Moises

Splitter

Chordify

ForScore

Overture Games


Conductor Jonathan Girard used an app called Tilt Brush to teach conducting patterns.


Theresa Hoover has put together an excellent page of ideas in "An Overview of AI in Music Education".


Here's a VERY thorough and fascinating May 2023 66-page deep dive on AI and education,
with recommendations from the U.S. Department of Education.


And Midnight Music offers many resources, plus an optimistic outlook for music education.


 
MUSIC and AUDIO PRODUCTION

 

Generative:


OUTPUT Co-Producer
(samples from text)

Gennie (sample generator)

SynthGPT (synths from text)

Flow Machines

Magenta (part of Ableton Live)

Beathoven

Mubert

Soundful

Soundraw

AIVA

MuseNet

MusicLM

Musicflow

Kits.ai

GoTalk (AI voiceovers)

Suno



How to Make a Singing AI Voice


A Complete Guide to AI Music Generators



Mixing and mastering:


Neutron 4

DAACI

Smart EQ 4

Datamindaudio

LANDR

Dolby Atmos



Webcasting



Riverside

Podcastle

Adobe Podcast



Sheet music tablet apps


forScore

MusicNotes

 
MUSIC MATCHING

 

Shazam

ACRCloud

Musixmatch

 
FRAUD DETECTION and PREVENTION

 

AI Or Not

Nightshade

Glaze

DeleteMe

Scamalytics

Seon

SynthID

Snopes



Sheet music piracy tracking and prevention:

MUSO

 
Three Orcas. Photo by Alex Shapiro.
VARIED, INTERESTING INFO
OVERVIEWS



Rick Beato: The AI Effect: A New Era in Music
and Its Unintended Consequences.



Learn more about data poisoning, in articles from MIT and the New York Times.


The History of Video Conferencing from 1870 to Today.


Here's a run-down of music possibilities from 6Connex.


Consider the many contrasting facets of the AI diamond in this essay from Irish composer and performer Jennifer Walshe.


Pew research has published an excellent 230-page overview.


Two excellent newsletters for keeping up to date with the latest tech are
AI Secret and TLDR.

 
ASCAP and AI

 

ASCAP has taken the lead in protecting the rights of copyright holders.
Read about its initiatives HERE, and HERE!


Listen to a VERY clear and informative podcast in which ASCAP's Executive Vice President and Chief Strategy & Digital Officer Nicholas Lehman gives host Mindy Peterson a thorough overview of the positive opportunities and the troubling problems with AI, and what ASCAP is doing to protect the creative community.

Watch the 30-minute documentary, Prelude in AI Major: Crafting a Creator-First Future for Music & AI. Featuring five teams of composers and musicians developing innovative uses for AI, it presents a balanced view that pairs enthusiasm for technology with the need to protect copyright.

 
IMMERSIVE EXPERIENCES

 

Epson Teamlab: Borderless


ION Concert Media: Muséik


Read how Detroit Opera used green screen and gaming technology
to bring Wagner to audiences.

 
HAPTICS and ACCESSIBILITY



The Haptic Baton can level the playing field for disabled musicians.


Read about the variety of tools from Human Instruments.


Learn about Imogen Heap's Mi.Mu gloves


Lightbuzz (body tracking sensor)


 
HARDWARE

 

HoloLens 2 AR Headset: On Stage Live Demonstration


Vision Pro demonstration


Meta Quest 3


Google Glass


BMW Connected Ride Smartglasses


Rabbit R1

 
STARTUPS




Dotdot, which is developing apps allowing online event-goers to move around in interactive video bubbles during meetings and in live performance spaces.

ASCAP Lab

NYC Media Lab

 
AI MUSIC GENERATION



Technology and media expert Shelley Palmer has a terrific list of links worth exploring. And, he links to a great many other AI resources as well.


A fascinating tour of 12 songs written with AI


5 Examples of AI-Generated Music


 
HOLOGRAPHY

 

Want to see how pseudo-holography is done? Read about a stage-craft technique from the 1860s known as Pepper's Ghost, that brought Tupak Shakur back to "life" in 2012 at Coachella Music Festival.


Learn what went into producing ABBA's Voyage holographic show.


Tensor Holography



 
CONCERTS in the METAVERSE

 

Here's a piece about Virtual concerts in the pop and TikTok world,
with many artist links to explore.

 
HOW THE BOTS WORK

 

If you wanna geek out and learn more about how the bots learn, here are readable explanations of the differences between machine learning, deep learning, and natural language processing, HERE, and HERE.


With TensorFlow, you can create your own machine learning model.


 
SPAM and AI

 

Ever wonder about all the spam AI content you're bombarded with? Get into the weeds of the dark side, thanks to a very thorough piece from 404 Media.

 


 
PLAY AROUND!



Play Bach duets with a bot. Read more about it HERE.


Have some fun with creator David Li HERE, and HERE!


 
River otter. Photo by Alex Shapiro.

 

Get in touch!

 

To contact Alex Shapiro,
email is best.
Click here to send Alex a note email
email Alex!

 

You can also reach Alex on the socials:

 

 

           
           
           
Bluesky
LinkedIn
YouTube
Facebook
Instagram
Soundcloud
           
           

 

 

 



©2026 by Alex Shapiro
All photos taken, (not generated!) by Alex Shapiro.
All rights reserved to design.