Hi friends,

can you recommend some open-source (self-hosted) AI tool for generating sound effects?
it can be something in development/research state,

for reference, something like Meta's AudioBox text to audio: https://audiobox.metademolab.com/capabilities/text_to_audio

cheers

 

Not exactly what you're looking for, but for game type of FX, the various sfxr forks out there are pretty fun to play with.

https://raylibtech.itch.io/rfxgen < web and standalone
https://jfxr.frozenfractal.com/ < web

TextToSample needs to ping their server every startup, but once you've downloaded the model, nothing else needs to be downloaded.

 

sample diffusion is pretty cool, there are a bunch of models on harmonai's discord.
also pretty easy to train your own models with google colab

https://github.com/sudosilico/sample-diffusion

 

I keep getting all sorts of music as a result, is there anything that needs to be changed to get SFX, not music?

 

Hm, I just tried with 'wind chimes struck by mallets' and 'creepy drone' and it gave me those - but yeah, not a single 'transient' you can drop into your DAW.

 

last time I tried some "train going through railway station", I got some kind of rairoad tycoon country soundtrack xD
maybe it's really picky about proper prompts?

 

Haha, you're right. I just tried and got a similar result. Guess it wasn't trained on non-musical sounds.