Hi friends,

can you recommend some open-source (self-hosted) AI tool for generating sound effects?
it can be something in development/research state,

for reference, something like Meta's AudioBox text to audio: https://audiobox.metademolab.com/capabilities/text_to_audio

cheers

 

Not exactly what you're looking for, but for game type of FX, the various sfxr forks out there are pretty fun to play with.

https://raylibtech.itch.io/rfxgen < web and standalone
https://jfxr.frozenfractal.com/ < web

TextToSample needs to ping their server every startup, but once you've downloaded the model, nothing else needs to be downloaded.

 

sample diffusion is pretty cool, there are a bunch of models on harmonai's discord.
also pretty easy to train your own models with google colab

https://github.com/sudosilico/sample-diffusion

 

I keep getting all sorts of music as a result, is there anything that needs to be changed to get SFX, not music?

 

Hm, I just tried with 'wind chimes struck by mallets' and 'creepy drone' and it gave me those - but yeah, not a single 'transient' you can drop into your DAW.

 

last time I tried some "train going through railway station", I got some kind of rairoad tycoon country soundtrack xD
maybe it's really picky about proper prompts?

 

Haha, you're right. I just tried and got a similar result. Guess it wasn't trained on non-musical sounds.

 

I've collected some for the HyMPS project under AUDIO section \ AI-based page: enjoy.

 

There's some cool stuff there, but I think you should remove all the projects that have been dead for 4+ years, because of how quickly things are moving.

 

Well, but some ideas behind those could be still interesting/inspiring even if dated...

...anyway that's why I've putted (colored) "Last commit" badge.

 

Here's a strong contender text for your AI cover application


*spam* Creates stunning, personalized cover songs in minutes with no musical experience required.

You can find it at *spam*