Is there anything that will allow me to clone my voice, then produce a vocal singing track by inputting text (for the words) and midi (for the pitch, duration, etc). I've seen things where someone can clone their voice and then have the resulting clone speak. I've seen things where you can sing into a system and have it clone someone else's voice. I've seen things where you can give a text prompt such as "sing me a song about cowboy living" and have the AI generate words and music, but nothing that exactly meets what I'm looking for. I'd like to clone my voice, then type the text I want the clone to sing, and input MIDI notes to drive how the resulting voice sings. Does such a thing exist? Is there a workaround to achieve something similar?

 

Maybe you can:
1- Use Synthesizer V to input your text and melody using a voice that nearly match with your own voice. Export the voice track to audio file.
2- Create an AI model of your voice with RVC (google to find how to)
3 - Use Replay (https://www.tryreplay.io/) to replace Synthesizer V voice track with your voice AI model.

Warning : It's a lot of work

 

~

Very interesting, thank you.

 

yes I had a similar thought but wasn’t sure if there was a more streamlined workflow. Do you think I can do all this with the free version?

 

You can try with the free version of synthesizer V. Free voices won't be as good as paid ones but you can already test if the workaround suits you.

 

You can try a software called UTAU, it's similar to Vocaloid and it's possible to record your own voicebank, don't know which languages are supported

 

Would there be any reason to use RVC over Eleven Labs for voice cloning? Eleven Labs seems to be pretty popular.

 

Isn't Eleven Labs use for speech only. I don't know much about it?
RVC models are use by Replay. Maybe you should ask a member call cutified, he's the creator of Replay.
In one of his post he's talking about training RVC model on https://replicate.com/replicate/train-rvc-model

 

Here's an example.
Not my taste but it's possible.

 

I’m thinking this might be more effort than it’s worth for me right now