I made a free online tool to unpack EZKeys MIDI packs.

Discussion in 'Software' started by kabaragoya, Dec 31, 2023.

  1. bagpipe

    bagpipe Noisemaker

    Joined:
    Jan 3, 2024
    Messages:
    9
    Likes Received:
    5
    The format of the pack files is proprietary archive file like a zip with no compression. In synthpop it looks they accidentally put a copy of the archive inside the archive (name inception archive is 000851@SYNTH_POP.dlx). The contents were identical to the outer part and I never encountered a similar example.

    As for the header files those are empty place holder files so the associated empty song title name directory exists. I left in for consistency and not lose metadata, and while real zip files technically support empty folders most tools don't preserve them.

    Code:
    126-S090@_SWEET_AS_SUGAR_126_BPM
    126-S091@Intro
    126-S092@Verse
    126-S093@Pre_Chorus
    126-S094@Chorus
    126-S095@Bridge
    
    These directory names are as they appear in the pack file. The song directory names are clearly tempo, and then what I believe would be message catalog number with song parts grouped in a block of 10s. The part after the @ is not meant for display, the Snnn I guess is looked up in a translation (l12n) table. But that table is not in the pack file. If one knows where they are or what they look they can be applied for renaming. Names left as they are to leave allow for custom transformation.

    I uploaded sometime since you tested but there is usually Aversion.txt files in the root. It is just a text file with the pack version. Why Aversion and not just version? No idea, but decided to not change things other than add .txt extension.

    Every midi has a related midinfo file with what appears to be simple instructions in a custom language to declare the chord progression in the MIDI file. There probably should be an option to include those.
     
    Last edited: Jan 3, 2024
  2. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    I'm genuinely excited that this project has led to you learning to code! That's freaking awesome.

    I'll take a look-see at my looping approach to try and figure out why some files are being dropped.

    I don't have the VST so it's not possible for me to visually compare; hence, I had no idea there was an issue until now. If you can give me a specific example of a file that appears to be missing from Synth Pop, Indie Rock or Dream Pop (the three packs I have) that would be helpful.
     
  3. AgafonovTimur

    AgafonovTimur Noisemaker

    Joined:
    Aug 7, 2022
    Messages:
    26
    Likes Received:
    6
    maybe its just better to unzip all midi in diffrent folders and post it on on sister site ?
     
  4. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    You do you, but this conversation has nothing to do with piracy.
     
  5. bagpipe

    bagpipe Noisemaker

    Joined:
    Jan 3, 2024
    Messages:
    9
    Likes Received:
    5
    Is always the first one physically occurring in the pack file. For dreampop is 4/4 D Major 61BPM (full path 105@Straight_4#4/061-S051@Intro), SHA1 starting 98738ca3.
     
  6. bagpipe

    bagpipe Noisemaker

    Joined:
    Jan 3, 2024
    Messages:
    9
    Likes Received:
    5
    Here's a python script to extract the bundle files to either the filesystem or copy them to a zip file. It also describes the format.
    https://gist.github.com/bagopipes/1f76d4fc7bd2fe56769445e0dff2b4c4

    Output not the same as the web app, it is more interesting. It doesn't do any renaming by default, it just extracts all the files with the original names, bu it doesn't filter anything.

    This is interesting because at least some packs in addition to the Aversion file have a midiDB file which is an sqlite 3.x database with a ton of a metadata including all the display names mapping to the directory names. This also unpacks the midinfo files with the chord progressions.

    EDIT: The script has been updated with a rename option that will append key, time sig and tempo to midi files and associated metadata files, and option to skip sidecar and empty files.

    It has been brought to my attention that the name mapping databases (midiDB files) for the expansion packs are shipped with EZKeys. Furthermore the ttfs pack format is used for other VST resources including graphics assets and configuration. The linked tool will unpack these as well.

    https://ttfstool.web.app This version has the above mentioned features.
     
    Last edited: Jan 9, 2024
  7. Jay F.

    Jay F. Newbie

    Joined:
    Jan 9, 2024
    Messages:
    1
    Likes Received:
    0
    I would like to side with Toontrack a bit...
    1. If TT sells the MIDI packages the way you want them to, who else should buy their main product?
    2. As much as it's annoying when you buy something "incorrectly", it's only your fault if you don't read the description correctly. Each MIDI pack has the following written in bold letters on a red background:
    SYSTEM REQUIREMENTS
    This product requires a working installation of EZkeys1 or EZkeys 2 to operate

    Could the manufacturers make it easier for us? Most certainly!
    But not everything a manufacturer does NOT, has to do with ignorance and poor customer service.
    I have been working with Toontrack products for a long time and during this time many improvements have been incorporated into updates based on customer requests.

    Apart from that, you created a very cool converter, thank you for your effort!
    All the best!
     
  8. bagpipe

    bagpipe Noisemaker

    Joined:
    Jan 3, 2024
    Messages:
    9
    Likes Received:
    5
    I don't disagree with any of this. Just to clarify there were two distinct conversion efforts here. The initial one which is more of a dump whatever MIDI looking data out of a binary file and the one I released which specifically parses and optionally converts to ZIP the toontrack archive format, not even just MIDI files.
    After finding that the midiDB metadata for all the packs is distributed not in the pack but in the VST itself its clear that the intent of the packs is more of add-on DLC for the VST and not a standalone product.
    The motivation for ttfstool was not primarily about getting MIDI data. On that note a generator/mod tool is the next useful step.
     
  9. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    When you buy MIDI packs on the web, you can reasonably assume that you are buying a zip file full of .mid files. EZDrummer and EZBass MIDI pack products are, in fact, packs of MIDI files. The literal meaning of words matter, here. The EZKeys MIDI packs are not MIDI files.

    ToonTrack refuses to refund money to people who mistakenly bought content that they otherwise cannot use. Yes, there is a warning; EZDrummer and EZBass MIDI packs give the exact same bold letter warnings, and yet you don't need to own expensive VSTs to access the MIDI data.

    They used to provide a trial version which could have - in a pinch - been used to extract the MIDI data you purchased, but that has ended. You say that they take customer feedback but this has been a problem people have complained about on their forum for over half a decade.

    For us, buying these packs is the literal equivalent of throwing your money into a hole. We don't need people standing near the hole telling us that we wouldn't have this problem if it wasn't for all of our personal failings. Nobody likes those people.

    TL;DR: If you don't need it, feel free not to use it.
     
  10. DonCaballero

    DonCaballero Producer

    Joined:
    Feb 6, 2016
    Messages:
    166
    Likes Received:
    92
    I really appreciate sharing the code. :)
    I prefer the naming of the original tool as the file names will be unique within/between packs, and when I search by key and import files into my DAW it's easy to tell the original "song"

    Unfortunately I did all my packs individually before the naming/metadata was fixed, so I think it would be faster to modify your code with Grimoire and batch convert than re-convert one by one.

    *EDIT*
    Turns out it would be more work than I thought as the code would need refactoring to add a GUI with tkinter.

    Also FYI, the file for the newest expansion 000751@EKX_SOUL_ROADS gives a "too big" error. It's just over 1.5mb (much smaller zipped, but zips aren't supported.)
     
    Last edited: Feb 4, 2024
  11. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    Ask and ye shall receive... could you try soul roads again?
     
  12. canbi

    canbi Kapellmeister

    Joined:
    Jun 12, 2023
    Messages:
    148
    Likes Received:
    50
    drag-drop standalone local script could be more versatile but at least there is something :):):):)
     
    Last edited: Feb 11, 2024
  13. bagpipe

    bagpipe Noisemaker

    Joined:
    Jan 3, 2024
    Messages:
    9
    Likes Received:
    5
    The script has a --rename option, does that do suitable naming?

    EDIT: Ok, I think I understand what you're saying, with the numbering. Though doesn't it make more sense to have Key signature first?
     
    Last edited: Feb 7, 2024
  14. bagpipe

    bagpipe Noisemaker

    Joined:
    Jan 3, 2024
    Messages:
    9
    Likes Received:
    5
    https://ttfstool.web.app/ is pretty close though as it will accept zip files. So you can zip the directory up and drag it in one go.
     
  15. DonCaballero

    DonCaballero Producer

    Joined:
    Feb 6, 2016
    Messages:
    166
    Likes Received:
    92
    Just a friendly heads up that the 5 newest packs are between 2 to 5.5mb, which is far larger than previous ones (too big).
    14.2mb in total, but zip to 2.1mb so there must be a lot of redundancy.

    My guess is other future packs will be similarly bloated so to maintain compatibility while keeping bandwidth low, you might be able to support zip uploads with the same function library handling the zip output.
     
  16. kabaragoya

    kabaragoya Ultrasonic

    Joined:
    Dec 31, 2023
    Messages:
    31
    Likes Received:
    25
    Thank you so much for letting me know. I just bumped the upload size limit to 10M. YOLO or something, right?

    Please give them a test and let me know if anything goes horribly wrong.
     
  17. tornado

    tornado Newbie

    Joined:
    Sep 1, 2023
    Messages:
    15
    Likes Received:
    1
    friends, I know it's not the right place, but there are some nice ones here, for example, when we direct the user midis to the chords in ezkeys, sometimes it doesn't translate properly, sometimes it adds notes to you, for example, melodic or bass midis add notes other than chords to them, how can we fix them, ezkeysin does not have this problem in its own midis, it only happens in user midis.
     
Loading...
Loading...