I made a free online tool to unpack EZKeys MIDI packs.

Discussion in 'Software' started by kabaragoya, Dec 31, 2023.

  1. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    It's possible, but not likely in the near-term because I don't know if Toontrack is going to, you know, do something about it.

    It's easier to turn off a website than it is to convince a judge that I've somehow tracked down every copy of the source code and deleted it.
     
  2. clone

    clone Audiosexual

    Joined:
    Feb 5, 2021
    Messages:
    7,417
    Likes Received:
    3,268
    If you were to make a tool like that commercially available for purchase; you can bet they would not be happy about it at all.
    But if it was something like KLU for Kontakt, just a free tool they do not appreciate the finer aspects of; you probably would have a much smaller problem. Even if you code a nice free app, there will always be someone to give you grief about it.

    That programmer Jeffrey ended up taking down his online tool called Jeffrey's Exif Viewer due to the bandwidth use, or the more shady purposes people had used it for. Obviously, he's not going to say that. Just an example.
     
  3. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    I'm not familiar with Jeffrey or his infamous tool :wink: but it's so weird that someone even had to make an online Exif viewer, if you think about it. After all, it's *really* easy to access Exif data for a file on every OS.

    I feel like charging for a tool like mine radically undermines the very legitimate outrage which I channeled to power through actually writing it. I'm so freaking tired of corporations treating customers like they can do anything they want, and then also decide whether to do anything about it. It's obviously not how things were supposed to work in society, but we've grown up in it so we just accept whatever we're told.

    Reminding people that they don't have to take the shit that comes down the trough is enough for me.
     
    • Like Like x 1
    • Love it! Love it! x 1
    • List
  4. clone

    clone Audiosexual

    Joined:
    Feb 5, 2021
    Messages:
    7,417
    Likes Received:
    3,268
    I'm not sure about out in the middle of nowhere how great it would work, like getting bars on a phone. It was very accurate, like maybe a few hundred feet or closer.
     
  5. tvandlover

    tvandlover Producer

    Joined:
    Aug 27, 2016
    Messages:
    283
    Likes Received:
    142
    I'm not sure why you need this. I just audition the midi in ez keys and then drag into the DAW.
     
  6. El Cycer

    El Cycer Producer

    Joined:
    Sep 9, 2023
    Messages:
    303
    Likes Received:
    135
    Bro, don't do that
     
  7. DonCaballero

    DonCaballero Producer

    Joined:
    Feb 6, 2016
    Messages:
    166
    Likes Received:
    92
    Looks useful, however the website says only EZKeys 3 is supported in multiple places, but AFIAK that doesn't exist, and EZKeys 2 is the most recent version.
    :dunno:

    Also someone else mentioned 000600@EZKEYS_2 giving a GTFO error likely due to the larger file size. It would be great if this could be fixed.

    My guess is the MIDI conversions are cached and the website gives you the correct .zip based on the upload filename, but if not, that could be an idea for EZKEYS_2 so it doesn't need to be uploaded/processed each time.
     
    Last edited: Jan 1, 2024
  8. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    I think that I do not understand what Jeffrey's Exif viewer did. Are you [not?] talking about the extended properties available on many image formats?
     
  9. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    That works great, unless you don't have (or want) EZKeys, a $179 product.

    Right now, you're like a parkour enthusiast telling people to just run up the side of a building.
     
  10. clone

    clone Audiosexual

    Joined:
    Feb 5, 2021
    Messages:
    7,417
    Likes Received:
    3,268
    If the image was taken with a phone camera with Location Services enabled; it would analyze the Exif Data, and geolocate it. Another click or two and it would pull up Google Street View.
     
  11. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    Huh! You're right. I got confused because I'm primarily an EZDrummer 3 user, and I've had several backchannel conversations with folks about the core library. I just added the text yesterday to avoid confusion, but it's clear that the text is the confusion. I'll kill it right now.

    As it currently stands, I don't own EZKeys 2 and so I don't have what is apparently the core library to test it. I have the converter set to reject anything over 1MB, because the packs all seem to be 200-400kb. How big is the pack?

    It's far more likely that there's simply something different about the pack format for that file.

    Here's the thing... if you own EZKeys 2, then you don't need my tool to access the content that ships with EZKeys 2.

    That's a good guess, but it is completely wrong. :)

    As I explain earlier in this thread, I spent several days reverse engineering their proprietary pack file format. The entire process is performed in-memory and no files are ever written to the filesystem; not even temp files.

    Most significantly, I don't have the packs to perform this sleight of hand. If I did, my hourly rate is far higher than the amount I would save if I attempted to manually export 10-15k individual MIDI files. Also, that would be the most insanely boring thing I can possibly imagine doing - and then giving that labour away for free... it just doesn't make sense.

    So, to be very explicit: there's no caching, the upload filenames are not relevant, and unfortunately the core library is not likely to be supported because this is a tool for people to access content that they've paid for.

    I don't judge people who get assets via other channels, but that's just not what this is.
     
    Last edited: Jan 1, 2024
  12. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    Oh, wow! That's freaking cool. Very "Enemy of the State".

    I don't particularly need this in my life, but for what it's worth... this would be exceptionally easy to recreate. Hell, I bet GPT-4 could just do it:

    https://chat.openai.com/share/cab5cc53-eb58-4535-a6d6-0179193da544

    ^ yes, it could.
     
  13. korniceman3000

    korniceman3000 Ultrasonic

    Joined:
    Jun 13, 2011
    Messages:
    94
    Likes Received:
    34
    WOW!! This is such an awesome tool!! Just wanted to show my gratitude and say thank you for your hard work!! Makes it so much easier for me to access the midi without having to open EZKeys every time just to extract so I can use the midi on a different vst. This is greatly appreciated!!
     
    • Like Like x 1
    • Love it! Love it! x 1
    • List
  14. DonCaballero

    DonCaballero Producer

    Joined:
    Feb 6, 2016
    Messages:
    166
    Likes Received:
    92
    Thanks for the clarification. I haven't fully checked the results, but the tool seems to have worked for all EZKeys files I have with that naming format (the earliest dated 2016 well before EZKeys 2.)

    The factory library file is 17.8mb btw, so well above the 1mb limit. I was really impressed with the speed of the results (nearly instant), and as there is a very finite number of add-on packs I just assumed it might make sense to store the cache the conversions rather than convert the same synthwave pack 500 times, but it's likely not resource/bandwidth intensive so caching isn't necessary. I wasn't trying to imply you were a Mechanical Turk manually exporting the MIDI. ;)

    I understand your frustration of buying a pack and not being able to use it at all, but it's also annoying for those who own EZKeys not to be able to use the factory MIDI library easily outside the plugin the same way as their other products (I bought Superior Drummer 1 when it came out, but the sample library was overkill at the time so used EZDrummer instead.)

    Anyway, I really appreciate you making this.:) I find the EZKeys MIDI is high quality, but can't stand having to use the plugin to access it.

    The only suggestion for improvement would be to append the tempo and key from the sub-folders to the file names to differentiate them as currently many files in the same pack will have identical names. This would help a lot when filtering too since we could search by key and get results regardless of the pack.

    Also, it's nice the MIDI files include tempo and key metadata which most commercial packs lack, but I noticed the tempo metadata often differs by 1 bpm, and the key is different than the folder name. The tempo doesn't really matter to me as Reaper automatically matches it to the project, and the metadata key is sometimes a better fit than the folder name (Am vs. C etc.), just thought I'd point it out in case there was something unexpected happening.
     
    Last edited: Jan 2, 2024
  15. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    I just removed the file size restriction so that you could try uploading the base library, because I certainly am curious. Please let me know when you've done so and if it worked for you. I don't think I can leave it this way, because it's actually a significant DoS attack vector. However, I do love a good experiment.

    Thanks for the suggestion. I just implemented it as a change; give it a shot?

    See kids, ask nicely and we can have nice things!

    Now that is interesting! Let's tackle these one at a time:

    Code:
      def calculate_bpm_from_tempo_data(midi_file)
        tempo_event_identifier = "\xFF\x51\x03".force_encoding('ASCII-8BIT')
        pos = midi_file.index(tempo_event_identifier)
     
        if pos
          # Extract the tempo data (3 bytes following the identifier)
          tempo_data = midi_file[pos + 3, 3].unpack('C*').inject(0) { |sum, byte| (sum << 8) + byte }
          bpm = 60000000 / tempo_data
        else
          # Default BPM if no tempo event is found
          bpm = 120
        end
     
        bpm
      end
    
    Can you give me a specific example of a file that has a BPM that is off-by-one? And can you generally say whether the off-by-one is one below or one above?

    My current hunch is that different tools convert floating point numbers differently, specifically when it comes to rounding up or down. If there's a distinct pattern, I could attempt to coax things in the right direction.

    Code:
      def calculate_key_signature(midi_file)
        key_sig_event_identifier = "\xFF\x59\x02".force_encoding('ASCII-8BIT')
        pos = midi_file.index(key_sig_event_identifier)
     
        if pos
          key_sig_data = midi_file[pos + 3, 2].unpack('cC')
          key = case key_sig_data[0]
                when -7..-1 then ["F", "Bb", "Eb", "Ab", "Db", "Gb", "Cb"][key_sig_data[0] + 7]
                when 0 then "C"
                when 1..7 then ["G", "D", "A", "E", "B", "F#", "C#"][key_sig_data[0] - 1]
                end
          key += "m" if key_sig_data[1] == 1 # Minor key
          key
        else
          "C" # Default key signature (C major)
        end
      end
    
    Here's an admission: I'm a drummer, and I barely know/understand Western musical theory and notation. I had to ask GPT why there's no Fb and why F# and C# have hash characters and the others don't.

    I had to work a lot of this out from staring at the MIDI spec, which is not a recommended way to learn music theory. If you or anyone looking at this spots an obvious logical error, please jump in.

    In particular, I'd really appreciate it if you could give me a few examples of specific files that have the problem, and what key you believe it should be instead of whatever it says. I'd love to get this right.
     
  16. DonCaballero

    DonCaballero Producer

    Joined:
    Feb 6, 2016
    Messages:
    166
    Likes Received:
    92
    Sweet!
    I was able to upload both the EZKeys 2 library and the legacy library which is also included (likely in v2 format already.)

    At first I didn't think it worked as the 000600@EZKEYS_2 file is 18.3mb and the zip was 1.2mb. Strangely, 000500@EZKEYS_MIDI is 1.9mb, but the zip was actually larger at 1.3mb.

    The website says EZKeys 2 comes with "More than 1,350 individually played MIDI files", and the zip had 1395 files so it seems like everything is there. The base library file includes different styles so unlike the add-ons it's hard to know what's what, but it's still great to have them.

    It works! Thanks a lot, this will be much better. :)
    I'll likely hold off reconverting all the packs for now while the tempo/key thing is being investigated.

    There seems to be quite a few, but not all of them. I'd have to check more files, but so far it seems the metadata is right and the folder name is wrong. With the files below when I select all the notes and use Reaper's Scale Finder it returns F Major and D Minor (which has the same notes). Cb doesn't match even if I just select the root notes. :dunno:
    [​IMG]
    I also did a BPM to time conversion and 8 bars at 68BPM = 28.235s which is the exact length in Reaper. 67BPM would be 28.657s so this confirms it.

    My math/coding skills are pretty lacking, and if I knew much about music theory I wouldn't be using EZKeys so unfortunately I can't help you much there.:rofl:

    If you're using the functions above to calculate the tempo/key for the folder names, you might be able to get the correct info from the MIDI metadata instead. As previously mentioned very few commercial MIDI packs include metadata at all, but the plugin likely requires it to function properly which is why these have it.

    Anyway, I hope that helps get to the bottom of things. I'll keep an eye on the thread to test any tweaks. :wink:
     
    Last edited: Jan 2, 2024
  17. tvandlover

    tvandlover Producer

    Joined:
    Aug 27, 2016
    Messages:
    283
    Likes Received:
    142
    1212
     
    Last edited: Jan 2, 2024
  18. tvandlover

    tvandlover Producer

    Joined:
    Aug 27, 2016
    Messages:
    283
    Likes Received:
    142
    I had no idea you had to buy EZKeys. I can see now why you would want to pirate the midi files
     
  19. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    I'm glad that you were able to get both files converted successfully. I've reenabled the 1MB upload limit, but added exceptions for the two core library files. I'm going to leave the message suggesting that core library files are not supported, because I don't really want to engage with folks who want a long list of exceptions that I have to manually add.

    Thanks for your help with the BPM. I just pushed a fix.

    As for the key stuff... like I said, I'd really appreciate some specific examples. It helps me narrow things down quickly.
     
  20. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    You don't need to pirate anything.

    This tool accepts the midi files you purchased in the locked format they ship them in, and gives it right back to you as the MIDI files that you rightfully assumed that you were purchasing.

    No shade but it says a lot about how messed up our society is that taking matters into your own hands to obtain what you were marketed and sold could ever be perceived as piracy.
     
Loading...
Loading...