AI Cloning Voice and instruments

Discussion in 'Music' started by Ryck, Jun 3, 2023.

  1. Ryck

    Ryck Guest

    I tell you interesting things I found.
    Curiously when I started using Synthesizer V, the voice would fit the song, without using effects, without using compression, and I wondered how? despite sounding robotic it fit well. Then I realized that it has no background noise, it's totally clean. Yesterday I created a model of my voice with AI, and you get that, the voice as if you had recorded in a studio isolated from ambient sound, so you hear the voice right up front. This could be a tool for those who do not have the necessary resources to record their voice without ambient noise.


    Another use that I found, is that by using another voice, you have a different view of what you are doing. When you sing your song, you hear it in one way, but when another person sings it, or an AI, you have another panorama, which will help you make the adjustments you need.

    Another use that you can give it is that sometimes our voices are very tired, it usually happens to me because I sing a lot, I sing on the subway 3 to 4 hours a day, and when the weekend comes I can't sing in the tone I want, therefore, if I have an idea for a song I can't do it or I have to strain my voice, with the AI, this would be helpful, since you could make your song, and after you already have the idea, when you already Your voice is fine, you sing it.

    Like any tool, I believe that it can be used for productive things and things that are not so productive, although this is somewhat complex, since what may not be creative for you is creative for me, and vice versa, perhaps for you. That seems creative to you, not to me. A clear reference is the musical styles. For some styles of music they lack creativity, but for other people, they are totally creative.
     
    • Like Like x 1
    • Interesting Interesting x 1
    • List
  2. BEAT16

    BEAT16 Audiosexual

    Joined:
    May 24, 2012
    Messages:
    9,082
    Likes Received:
    6,997
    Surely you've heard that some AI can change or completely replace entire voices. You can theoretically already offer perfect fakes on the internet today. You take a video with a message and give it a different content without the viewer noticing.

    I would like to see something that recognizes bad voices that don't sound good and can set the voice to a higher level. So that my inharmonious voice sounds great.

    By the way the word "Intelligence" is a marketing invention to sell the AI products better, because AI is the new business.
     
    • Like Like x 1
    • Agree Agree x 1
    • List
  3. Ryck

    Ryck Guest

    Completely agree with you. It's nothing more than giving a new "face" to technology to make it sound like "oh god! artificial intelligence"
     
    • Like Like x 1
    • Agree Agree x 1
    • List
  4. Ryck

    Ryck Guest

    Use the two Colabs that I leave above, if you need Paul's voice, I have the Drive link. I could send it to you by PM, because I don't think if I can put that link here, I'm not sure
     
  5. Ryck

    Ryck Guest

    look, other things that I found super interesting, and that I think will change all this of the Kontakt sound libraries (I think). It is instrument cloning. So far, it seems, it's only possible with monophonic instruments. So I've cloned a bass, but so far I only have 50 Epoch, because I "feed" it with more than 1 hour of samples and it takes too long. That is to say, for it to sound "decent" it should be approximately 500 "Epoch" And looking at this AI as a tool, I think it would be very useful for those who don't have a bass and could record it with their guitar, to me, it sounds more realistic than a library, as far as articulation, we know midis capabilities don't they reach the realism of an instrument played by hand. But look at these examples that I upload here. The first audio is a bass line played on a guitar, and the second audio is the AI emulation of a bass. As I said before, it still lacked a process, in terms of quality. But look at how well it achieves nuances, and also, there's no line noise.


    So, I imagine how creative we can be, I have thought of mixing instruments in the AI feed, for example, how would it sound if I put a synth, a violin and a guitar? maybe we would have new sounds
    that we have never heard and with a flexibility of a guitar, a saxophone, etc.

    PD:
    I will gladly share (with Oly's permission) the links of the instrument AIs if someone needs it or send them by PM. Or any other instrument.
     
    • Like Like x 3
    • Creative Creative x 1
    • List

    Attached Files:

  6. AC2023

    AC2023 Newbie

    Joined:
    Jun 6, 2023
    Messages:
    3
    Likes Received:
    0
    This sounds amazing Ryck, how did you get to this "tone transfer"? RVC?
     
    Last edited: Jun 6, 2023
  7. Ryck

    Ryck Guest

    Yes, exactly. I used RVC (Retrieval-based Voice Conversion). The ones I left in this thread are in Spanish, but I'm editing one in English. I hope to have it ready in a few days to share with you all. Essentially, you first train the model with a variety of samples where you can see the instrument executing all the articulations. Then, the other Colab is for transforming the model into audio. I haven't finished training the model yet; I'm currently at 50 epochs because I'm encountering an error. However, I've tested it with a saxophone at 50 epochs, and it works very well. Then, with your mouth, you can produce a wind sound like "turururur," and you pass it through RVC with the loaded model, and the instrument comes out. So, yes, it is RVC.
     
  8. AC2023

    AC2023 Newbie

    Joined:
    Jun 6, 2023
    Messages:
    3
    Likes Received:
    0
    I can't wait to play with that! I've tried to join the colab in Spanish and it worked out then I've tried the first link too but this won't open, it says something like:
    Request had invalid authentication credentials. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.
    tb@https://ssl.gstatic.com/colaboratory-static/common/a4d1f45790f57b92a2977b5b52bd32cd/external_polymer_binary_l10n__it.js:90:517
    jC@https://ssl.gstatic.com/colaboratory-static/common/a4d1f45790f57b92a2977b5b52bd32cd/external_polymer_binary_l10n__it.js:1560:2482
    oG@https://ssl.gstatic.com/colaboratory-static/common/a4d1f45790f57b92a2977b5b52bd32cd/external_polymer_binary_l10n__it.js:2008:424
    LGa@https://ssl.gstatic.com/colaboratory-static/common/a4d1f45790f57b92a2977b5b52bd32cd/external_polymer_binary_l10n__it.js:2144:224
    QGa/<@https://ssl.gstatic.com/colaboratory-static/common/a4d1f45790f57b92a2977b5b52bd32cd/external_polymer_binary_l10n__it.js:2159:389
    xa@https://ssl.gstatic.com/colaboratory-static/common/a4d1f45790f57b92a2977b5b52bd32cd/external_polymer_binary_l10n__it.js:21:57
    ua.prototype.throw_@https://ssl.gstatic.com/colaboratory-static/common/a4d1f45790f57b92a2977b5b52bd32cd/external_polymer_binary_l10n__it.js:20:201
    laa/this.throw@https://ssl.gstatic.com/colaboratory-static/common/a4d1f45790f57b92a2977b5b52bd32cd/external_polymer_binary_l10n__it.js:22:90
    c@https://ssl.gstatic.com/colaboratory-static/common/a4d1f45790f57b92a2977b5b52bd32cd/external_polymer_binary_l10n__it.js:22:340
     
    Last edited: Jun 6, 2023
  9. Ryck

    Ryck Guest

    I don't know what it is, and I don't know why you're experiencing that. The link I shared is public, so you should be able to access it without any issues. I am currently editing one myself, but I'm still learning about Colab. This is new to me. Right now, I'm only editing the language, but the code should be the same, and the code works fine. I'm not sure if the creator of that Colab notebook has put any restrictions for other countries. I don't think so. I don't believe there are such restrictions in Colab. When you share something, you share it for everyone.
    What you can do, and it's what I do myself, is to rely on OpenAI. When I don't understand something or encounter an error, I ask OpenAI for help, and it tells me how to solve it. I recommend using OpenAI's ChatGPT; it's very useful
     
  10. AC2023

    AC2023 Newbie

    Joined:
    Jun 6, 2023
    Messages:
    3
    Likes Received:
    0
    Yes definitely quite weird. Just ask you on PM if you can reshare the link and I'll see how to deal with that someway.
     
  11. RachProko

    RachProko Producer

    Joined:
    Sep 25, 2022
    Messages:
    242
    Likes Received:
    121
    I always wonder why some people feel the need to compare software to human vs animal capabilities that are physically stronger and faster than us in order to make some sort of point? But it totally doesn't make any sense? It's like comparing the capacity of a hard disk to the size of an elephant?!

    It's most likely because they really don't have a clue on how software and in particular AI really works?
     
  12. RachProko

    RachProko Producer

    Joined:
    Sep 25, 2022
    Messages:
    242
    Likes Received:
    121
    Yes, humanity has always had to deal with innovations and has always overcome it somehow. But that has always been physical. Machines taking over manual labor. Computers taking over administrative tasks as opposed to pencil and paper. It was entirely different because back then very few people were dealing with technology. They just suffered the consequences of its progress and it was mainly work related. But today the technology is on our personal phones and computers.

    Today, we all carry a phone and have computers at home that determine our lives! We want to be able to control things via the “Internet of things”. We have become very dependent on it.

    Now we can draw paintings on our phone without having touched any brush in our lives. Pretty soon we can have an app create music for us by putting in a few keywords without knowing what the scale of C means.

    Sure, it's great that this is all possible today! But I wonder what it will do to the creative mind? What’s left for us humans to explore if everything you want to achieve is always done better by an AI app?

    I guess we’ll have to wait and see what the future brings?
     
  13. Kingvrage

    Kingvrage Kapellmeister

    Joined:
    Mar 1, 2016
    Messages:
    201
    Likes Received:
    72
    Location:
    nowhere
    I'm not worried about AI. It'll do what every trend does eventually. It'll become samey and fizzle out in some areas, be reigned in and used at a minimum in others. Besides, in the entertainment world there's really way more garbage than anything worth your attention. What's more on wood on the fire? Might find 1 cool stick shaped like a Lazer gun but that's about it.
     
  14. Xupito

    Xupito Audiosexual

    Joined:
    Jan 21, 2012
    Messages:
    6,986
    Likes Received:
    3,859
    Location:
    Europe
    In the short-term musically speaking... oh, I got it!:

    (RATM) Rage Against The Machine will come back refurbished as RATAI (Rage Against The AI).
    Are they still alive?
     
  15. Daisy69

    Daisy69 Platinum Record

    Joined:
    Oct 3, 2022
    Messages:
    543
    Likes Received:
    178
    You are telling this to me?

    This is to me or around me?
     
  16. Ryck

    Ryck Guest

    As BEAT16 said, and I agree with him, "AI" is more of a commercial name for what already exists, a kind of apocalyptic impact like "OHHH AI." That being said, humans have always sought to simplify their work, from the very beginning, using pulleys and wood to help move rocks and avoid effort. This is technology, but it's primitive technology. As humans have advanced and learned to develop new technologies, we have reached "AI," which is nothing more than algorithms created by humans with the ability to "learn from them." It is not yet a completely independent technology, as it still requires humans to correct the codes and other aspects. What you say about phones and taking notes on paper is true, and I agree with you. But we all do the same thing, "blaming an invisible hand," as if someone is imposing things on us. Nobody forces us to use a phone, nobody forces us to use AI. Some people don't realize that by not using ChatGPT, they are still contributing to and using AI, and then they go and use Facebook, which is a developer of AI and actually uses bots and more. It's like some vegans who don't eat meat because they feel sorry for animals but wear leather belts. The point is that if someone disagrees with something and wants to defend their ideals, they should stay away from anything that contains what they refuse to use. I don't know if I'm making myself clear. I see technology in general, not just "AI," as a tool. Here we have VST Auto Mastering, "Auto EQ," "auto, auto, auto." In my case, I prefer to be creative and use my own mastering. But wait! It doesn't mean that someone using Auto Mastering is not creative. As Oly once said, maybe the person plays the guitar and sings and wants to be creative in their music and not waste time learning how to master. The topic is complex, but well, I believe it's up to us to give things a creative use. If we make the best creative use, we will get better results. That's my humble opinion
     
  17. RachProko

    RachProko Producer

    Joined:
    Sep 25, 2022
    Messages:
    242
    Likes Received:
    121
    Today AI has gone beyond ‘just’ a marketing gimmick as BEAT16 implies. It’s not the stupid ‘question and answer’ game with an ignorant computer as it was a few years back. It can now exponentially learn fast from all the input humans provide.

    Yes, that’s where it started. But now those algorithms have evolved to a point where it doesn’t need any human input or intervention anymore. It now learns and collects data at an incredible rate every minute as we speak!

    Wrong! This is where a lot of people don’t realize the current state of this technology? They still think it needs our input to solve problems? That was still the case even a year ago.Today its learning capabilities operates completely independently.

    Agreed. No one is forcing you to use anything. But our daily lives have been so entangled with computers, phones and technology that it’s impossible not to have to deal with some sort of AI. There’s just no escaping AI today whatever you do!

    I understand what you’re saying and agree with you. And I can see massive advantages when it comes to mixing and mastering. But it won’t stop there. AI will also learn about music composition, harmonization, arranging and orchestration. It can already create exclusive images of people that do or don't exist and are almost indistinguishable from the real thing.

    How long before it can create hit songs or even compose symphonies like Beethoven or the 5th piano concerto based on Rachmaninoff's legacy?
     
  18. Ryck

    Ryck Guest

    Technology ceased to be a simple question-and-answer program a long time ago. When I was 17 years old, I started working at a glass factory, and once we received many orders for cutting glass. So, you have a whole sheet of glass, I don't remember the exact measurements, but the point is that you have to try to make the most of all the cuts. Then the supervisor said to me, 'Come, let's see on the computer how we can make the most of the sheet,' and the guy entered the measurements of the glass, and the computer arranged each glass in a way that they should be cut to make the most efficient use of the space. I'm talking about the year 1997. Doesn't it seem like a kind of AI?

    Look, I've been using OpenAI for a while now, and while it has its successes, it also has its flaws. Sometimes it makes mistakes in very simple codes. According to the chat, in addition to its existing information, it can rely on logic, but that's not entirely true. For example, yesterday I told it that I was working on Jupyter Cloud, which is an online platform that gives me a local URL. However, the OpenAI chat kept telling me to open my command prompt and execute certain commands. I kept telling it that I was using Jupyter Cloud, which is online and requires a public URL, but it kept insisting, and it frustrated me. This happens often, and it's because a "physical" person, a human being, should work on correcting the codes or the supposed logic. It's also not just a matter of learning from anyone. I mean, it doesn't learn from me or you, and I'm talking about what I know. I'm basing my statements on OpenAI; this chat relies on information that is considered reliable by the company. They have trained this chat to provide information, but it doesn't learn on its own because otherwise, it would be some kind of social network where the information wouldn't be accurate. Therefore, it will always need those people who work for OpenAI in this case to verify and update the information.


    Well, yes, but we didn't start using smartphones overnight. Most of the things we use are based on our own choices. Nobody forced us to create social media accounts, and nobody forced us to buy cell phones. And if over time a new system emerges that is intertwined with all of that, and it becomes difficult to opt out, it doesn't mean that "society as a whole" cannot opt out. Perhaps you and those who dislike the system may find it challenging, but it's a system that has been collectively chosen. We live in a society, we live within a social contract. It's similar to when we talk about the State; the State exists because there is a social contract for having representatives who govern us, even though there may be individuals who disagree. I don't want to digress, but the point is that we make choices. When I was a child, parents preferred to buy their children a Sega game console and have them stay indoors playing all day, rather than paying someone to look after them or out of fear of them going out on the streets. We gradually locked ourselves away more and more, to the point that today it is increasingly difficult for society to establish meaningful connections. But society chose this path; nobody imposed it upon us.
    But that has been happening for a while now. I remember a few years back, when graphic design started incorporating programs that automatically generated logos. It became easier to pay a company that used this software or purchase the software itself rather than hiring a designer. Now, can you really argue that a logo created by a machine is the same as one created by a person? I don't think so. I've seen many AI-generated images, and I've even tried them using Bing. Do you honestly believe they are better than an artist who knows how to draw? I don't think so. There are already programs that "compose" music, and in my opinion, they are garbage. Now you're saying that AI will soon be able to create successful songs, but I believe that's a taboo. There are no inherently successful songs; there are songs that are driven toward success. There are many successful artists out there, but their songs can be dreadful. I won't mention any names out of respect, but success is by no means associated with a song filled with "talent."

    A while ago, I touched upon these topics here and I asked myself, like you, if AI would be capable of replacing humans. I believe that we are also a kind of highly advanced AI, and everything we do is based on information, just like an AI. If someone says to you, "Hey, could you pass me that glass?", you would pick up a glass because your brain (your hard drive) has stored information about what a glass is. Similarly, if I were to ask the AI, "Hey, could you give me a code to compress a file?" it would provide me with the code if it has it in its database. However, if I were to ask you, "Can you hand me a tafaluzardurak?" you would say, "What? What is that?" because that information doesn't exist in your database. I'm not sure if the idea is clear, but AI is still very, very far from surpassing us.
     
  19. RachProko

    RachProko Producer

    Joined:
    Sep 25, 2022
    Messages:
    242
    Likes Received:
    121
    No, that has nothing to do with AI at all! This is the basic thing that computers have been invented for to begin with: Using ‘mathematics’ to quickly calculate things like what you described. Also doing financial tasks like bookkeeping, making technical drawings of buildings and calculating all sorts of values to make it possible to actually build what you’re drawing. Again these things are all based on mathematics that were programmed by people to achieve a certain goal.

    AI are computer algorithms that were designed by humans but with the goal to let computers make decisions on whole other things than just mathematical problems. They were designed to let computers learn from human interaction and learn from their input with the ultimate goal to let them make decisions on their own without human intervention.


    I could and would really like to reply to everything you said and you have a lot of valid points. But I think I can suffice with making my point why I think AI is a danger for humanity. And this danger of AI is not because of the fear of replacing us humans.

    The real danger lies in how AI is slowly but surely invading our lives and how we and instances will increasingly start to depend on it.

    Governments, banks, insurance companies, etc already use AI based algorithms for risk assessment and to predict fraudulent behavior. If you have the wrong name, the wrong cultural background and/or live in the wrong city or neighborhood you might well be listed as a person that will not be eligible for a loan, mortgage or even an insurance for a car. Not because you’ve done anything wrong? No, just because you fit the profile of this AI algorithm.

    This very horrible scenario has happened already in the country where I live a few years back. People that received child benefits were marked as frauds by the IRS without any form of hear back or process. From one day to the next they were marked as frauds and summoned to pay back any child support they've received and fined with the same amount. It ruined the lives of 1000’s of families to the point where even child care removed the children of many of those families because of financial problems! The government has admitted that it was wrong what the IRS did. But even today the IRS has not reinstated most of these people and persists on its algorithms to determine which people are really eligible for compensation. It’s really sick!

    This is just one example of how destructive AI algorithms can be just because the government relies on these. And this is just the beginning! What’s next? Hospitals relying on AI for your treatment? Your pharmacy relying on AI for your medication? Your garage relying on AI to service your car? Your investment broker making financial decisions based on AI?

    What if AI is used in labs where the most deadly viruses to human kind are managed and AI receives a wrong input and decides to create a deadly virus to humans that has no cure?

    What if the media let AI manage the news feeds on all global social networks and it starts to create fake news that will create a conflict between the USA, China and Russia that can potentially unleash a nuclear war?

    So the real danger is not AI itself but the people in charge and the instances that embrace it and decide to use it as a tool to let it make impacting decisions for them? Decisions that can potentially impact the whole of humanity!
     
    Last edited: Jun 24, 2023
  20. Ryck

    Ryck Guest

    Well, I liked your opinion, or I can say that I fully agree with your final point, which is that the real danger lies in whose hands this powerful tool, AI, is in. Sometimes, when I debate about AI, a very drastic example comes to mind. It's like a knife that can be used to cut food, carve wood, or harm another person. So, what I think is that it's not the tool itself, but how it is used. Regarding the injustice you mentioned, it has been happening for a long time. Bots and AI are just a way to evade human responsibility. We have all experienced being unfairly charged for something, such as an extra fee for a service, and when you call, no one answers except a bot, or they tell you that the system marks it that way and they can't do anything. It's no longer a person handling the problems; it's an automated system that bills you automatically. That's why I say that all these things have been happening for a long time, and AI is just the progress of technology.

    It's true what you mentioned, that AI has the ability to learn on its own. Well, I can't assure that, as I've only been using OpenAI's chat, which sometimes has many flaws, and the cloning of instruments and voices also has flaws. And it's not the AI that corrects those flaws; it's the people behind the project. For example, the other day they updated Gradio, I don't know if you understand it, and all the code to clone voices didn't work because there was an error in the source code. So why didn't the AI correct itself? I searched forums and physically asked people to find a solution, and it turned out that it was just changing one line of code. According to OpenAI, it claims to be capable of reasoning, but I strongly doubt that it can reason. Moreover, we would need to define what reasoning is, and that leads to philosophical thinking. From my point of view, reasoning is nothing more than the comparison of acquired information, like the examples I mentioned earlier. Is it possible to reason a problem without information? I don't think so. But well, we humans are too complex. And I understand your point about the danger that AI can represent.

    But if we go back in time to the year 2000, there was a large number of people who committed suicide because they believed that all the computers in the world were going to shut down and the satellites were going to fall. That's why I say that to me, these are the same scenarios, but more advanced. However, we always find ourselves facing something "more powerful," so to speak, and we always get alarmed. If you're alarmed by AI, we can also think about how long the atomic bomb has been in the hands of people who could potentially endanger all of existence, yet here we are. I think that, in the end, these tools or weapons, as they can be both, like a knife, are in the hands of people with great power. We will have to have faith that they will do something good with it. I suppose it will be the eternal struggle of powers, but this time with AI.

    Edit:
    I would also like to add something to what we have been discussing, and it's something I always think about. 'What have we done to ensure that something replaces us?' Let me give you an example. Some time ago, I decided to limit my meat consumption, which led to a deficiency of vitamin B12. So, I was told that I needed to buy B12 supplements. I went to the pharmacy and told the pharmacist, 'Look, I need a vitamin B12 supplement because I hardly consume meat.' Reluctantly, the pharmacist handed me a bottle of pills and wanted to charge me for it. However, I doubted their attitude, so I looked at the brand and said, 'I'll think about it.'

    Then I turned to an AI chat and it told me that I needed to take a type of B12 vitamin that is specifically for vegans since it provides double the required dosage. It even told me the name of the product. So, I went to another pharmacy and found it. Now, the point of this, and it's not the first time something like this has happened to me, and I think it happens to all of us, is that we also choose things ourselves, whether it's bots, programs, or AI, that make our lives easier. And sometimes, we don't do enough to protect our own work.

    You mentioned AI in hospitals. Do you know how many times I've waited up to 8 hours in hospitals while doctors were enjoying coffee and having a good time, while there was a large number of people waiting? The healthcare system is overwhelmed, and sometimes, some professionals don't even feel like working, and I'm not saying it's everyone. But I don't know if you understand my point. Sometimes, we self-destruct, and then something improved comes along and pushes us aside, and we cry and ask, 'Why?' But how many times do we ask ourselves, 'Did I do something to protect my job?'

    What I'm trying to say when I mention that AI is nothing more than technological progress is that it's not entirely new; it's the advancement of a technology. Because I believe that all inventions have a limit. A PC, a bot, and now the continuation of that limit is AI. For example: Vinyl, cassette, CD, flash drive. If you look at it, it's the same thing, but it has evolved, and only the name has changed. It's logical that we will always be alarmed by something new. Now we say that AI will make decisions for us and be present in every profession. But it will have a limit, and (perhaps) a new desire of mankind will emerge, like 'mind-reading,' something that AI won't be capable of. Just imagine the future generations if that were to happen. They would say, just like us, but in a different scenario, 'This mind-reading thing is macabre. At least with AI, it could "reason" and learn on its own, but mind-reading, what will happen to us?'

    sorry for rambling so much.:rofl:
     
    Last edited by a moderator: Jun 24, 2023
Loading...
Loading...