How to achieve the BEST file/archive compression?

Discussion in 'Software' started by ToadMonster, Jan 17, 2024.

  1. ToadMonster

    ToadMonster Ultrasonic

    Joined:
    Apr 28, 2019
    Messages:
    43
    Likes Received:
    27
    Over the years I've been using WinZip, WinACE, WinRAR and 7-Zip, but I've always wondered how to achieve the BEST compression. (In my experience WinRAR was the best for me.) - My question to you guys are.. - How do I achieve the best and safest compression of soundlibraries, (that also includes the recovery record?) - Is WinRAR the best choice, or is it 7-Zip? - Also should it be "solid archives" or not?

    Thanks!

     
  2.  
  3. DoubleTake

    DoubleTake Audiosexual

    Joined:
    Jul 16, 2017
    Messages:
    2,235
    Likes Received:
    1,203
    Good question.
    I was unable to match R2R's compression for a few things in the past but it was no big difference when i recompressed for my own archiving.
    But NOW, with R2R using 7-zip, the difference is huge so I always keep their 7-zip within my archive.
     
    • Like Like x 4
    • Agree Agree x 2
    • List
  4. ELJUNTADERO2022

    ELJUNTADERO2022 Producer

    Joined:
    Jun 10, 2022
    Messages:
    389
    Likes Received:
    111
    same here! as far i know, the CPU takes an important role , am i right?
    eg: if i have a 100mb file, the far i can compressed with winrar in w11 64x with a ryzen 5 3600, its almost of 20mb... i dont know if this is the ratio to calculate or its more random at every file.
     
  5. Will Kweks

    Will Kweks Platinum Record

    Joined:
    Oct 31, 2023
    Messages:
    335
    Likes Received:
    196
  6. xorome

    xorome Audiosexual

    Joined:
    Sep 28, 2021
    Messages:
    889
    Likes Received:
    680
    One approach is to use one or several filters that pre-condition the data to be more compressible. Most 'super tiny' game releases use a de-duplication filter before the archiver.

    Using a single block / fully solid / single thread for your archive often increases compression ratio, but makes decompression take more time and RAM.

    You could use a re-compressor that changes parameters like word size, dictionary size etc automatically until it finds the smallest archive size possible. Google LZMA recompressor for a start.

    There are also a number of LZMA (used by 7-zip) implementations that swap out parts of LZMA out for more efficient implementations.

    https://fileforums.com/forumdisplay.php?f=55 for non-technical discussion and hands-on experience with 'extreme' compression.

    https://encode.su/forums/2-Data-Compression for technical discussion.
     
    • Like Like x 2
    • Winner Winner x 1
    • Interesting Interesting x 1
    • Love it! Love it! x 1
    • List
  7. tzzsmk

    tzzsmk Audiosexual

    Joined:
    Sep 13, 2016
    Messages:
    3,408
    Likes Received:
    2,084
    Location:
    Heart of Europe
    another vote for encode.su forum above - IF you are ready to learn the answer :hillbilly:
     
  8. BlackHawk

    BlackHawk Producer

    Joined:
    Nov 28, 2021
    Messages:
    266
    Likes Received:
    130
    You really think this discussion is worth it? Confused ...
     
  9. ELJUNTADERO2022

    ELJUNTADERO2022 Producer

    Joined:
    Jun 10, 2022
    Messages:
    389
    Likes Received:
    111
    hey for me its has a value, im not a serial compresser guy but sometimes i need to do this the best i can, and never can get a good compression :/ lets say u have a project to share or to save into any backup, which the file its 3GB , if u know how to compress well and have the hardware ( i mean good processor if im not wrong ) to do it, and can achieve a final 1gb large file, that would be very niiiiiiiiiiceeeeeee :D
    like i use time ago winamp to convert wav to FLAC/mp4 file, which was excelent. songs from 50mb converted to 10 or 5 mb x_x insane compression without damage the file.
     
    • Like Like x 2
    • Agree Agree x 1
    • Interesting Interesting x 1
    • List
  10. lbnv

    lbnv Platinum Record

    Joined:
    Nov 19, 2017
    Messages:
    350
    Likes Received:
    191
    CPU doesn't take important role in compression excluding speed. The more powerful the processor, the faster the compression.
     
    • Like Like x 2
    • Agree Agree x 2
    • List
  11. ToadMonster

    ToadMonster Ultrasonic

    Joined:
    Apr 28, 2019
    Messages:
    43
    Likes Received:
    27
  12. ToadMonster

    ToadMonster Ultrasonic

    Joined:
    Apr 28, 2019
    Messages:
    43
    Likes Received:
    27
    I didn't know about the encode.su forum, it's been bookmarked.. Thanks a lot!

    By the way, WinRAR 7 is probably released pretty soon - I'm curious to know what you think about the new WinRAR features?
     
  13. ArticStorm

    ArticStorm Moderator Staff Member

    Joined:
    Jun 7, 2011
    Messages:
    7,504
    Likes Received:
    3,777
    Location:
    AudioSexPro
    depends on the nature of the file. it is not depending on the CPU. The performance is only depending on CPU not the quality.

    I suggest reading into some compression technics and the original LZMA paper from the 70s, its amazing, how the dictionary is build and how to use other technics to improve compressions.

    also other exotic variants of zip are amazing to learn and try to understand from wikipedia.
     
  14. Dr. Black

    Dr. Black Producer

    Joined:
    Jun 20, 2023
    Messages:
    274
    Likes Received:
    102
    Location:
    @
    No. Compression works like this...
    Bible = so - 01 = total.

    In other words...
    Compression = 01 = 0 = book.

    The trick is... What Book you use.

    A book with 101 will be 010.
    Reversed.
    BUT... now here comes the trick...
    101010 = ?
    swich that to 01010
    And the 1 exists.

    in other words...
    Thesis = Re-Enthesis in 1 or 0.

    A computer program is static.
    Life is not.

    https://en.wikipedia.org/wiki/Quantum_computing

    It comes down to...
    101.
     
    Last edited by a moderator: Jan 17, 2024
  15. Pim Sukkel

    Pim Sukkel Member

    Joined:
    Nov 18, 2022
    Messages:
    9
    Likes Received:
    7
    Best Answer
    hi,
    after intensive testing found out that 7-zip works best for me, giving R2R-like compression, using following settings (see picture):

    7zip best settings.jpg

    amount of compression is heavily source-dependent, but in general those settings give very good results. compressing whole folders to a single file is not a problem. don't know how safe it is, but I never encountered any problem (at least not yet ).
    hope this helps
     
    • Like x 2
    • Interesting x 2
    • Useful x 2
    • Agree x 1
    • Winner x 1
    • List
  16. Will Kweks

    Will Kweks Platinum Record

    Joined:
    Oct 31, 2023
    Messages:
    335
    Likes Received:
    196
    Quick! Somebody introduce this guy to dynamic linking and loading!
     
    • Funny Funny x 2
    • Like Like x 1
    • List
  17. ToadMonster

    ToadMonster Ultrasonic

    Joined:
    Apr 28, 2019
    Messages:
    43
    Likes Received:
    27
    Thanks Pim Sukkel, the settings you use in 7-zip, does it include a recovery record of sorts, or don't you ever get CRC errors?
     
  18. Will Kweks

    Will Kweks Platinum Record

    Joined:
    Oct 31, 2023
    Messages:
    335
    Likes Received:
    196
    Not @Pim Sukkel but 7-zip doesn't have a recovery feature as such, if you need that, then you can add a PAR2 record (a separate tool), or use RAR which does have it built in.
     
  19. Pim Sukkel

    Pim Sukkel Member

    Joined:
    Nov 18, 2022
    Messages:
    9
    Likes Received:
    7
    Hi,
    until now I have not encoutered any error in 7-zip, but I really can't tell you how save it is. important files I always keep in various locations (discs can get corrupted or lost...) and in at least one location uncompressed.
     
  20. SineWave

    SineWave Audiosexual

    Joined:
    Sep 4, 2011
    Messages:
    4,376
    Likes Received:
    3,482
    Location:
    Where the sun doesn't shine.
    7-zip with -mx9 (maximum compression) switch will give you the best compression with some caveats. Other defaults are set for excellent compression already. There's no use to fiddle with huge dictionary sizes as huge sizes don't help much, but make compression times even longer. General consensus for 7zip is up to 32-64mb dictionary size, but you can make it bigger if you're compressing huge files.

    - making such 7z archive is much, much slower than RAR.
    - RAR works better for WAV and audio files in general.
    - there's no redundancy (data recovery record), only hash number, so if the archive becomes corrupted... it's 100% gone.

    So for example, if you're compressing a Kontakt WAV sample bank (not compressed, for compressed sample banks either 7zip or RAR will do), you would compress everything but the samples with 7zip and then compress everything with RAR and set data recovery record to 5% if you like to be on the safe side. By default it is set to -rr 3% which is rarely enough if the archive becomes corrupted, but even 3% is better than nothing. :wink:

    So generally, I still use RAR a lot because it's much faster and safer, with these switches: -m5 -md64m -rr
    or -m5 -md64m -rr5 if I want to store 5% data recovery record for safety. -md64m is for 64mb dictionary size. -m5 is for maximum compression. If you want RAR to make solid archives, add -s switch to it.

    Cheers! :headbang:

    I just remembered one thing: making solid archives will greatly influence the compression. It's the most important compression parameter aside from compression algorithm. But if you tend to update your archives with additional files, the whole archive has to be repacked when it's solid, and if you like to view or play files from the archive, the whole archive has to be unpacked. Non solid archives are also easier to recover, if that's important to you. :wink:
     
    Last edited: Jan 22, 2024
  21. Will Kweks

    Will Kweks Platinum Record

    Joined:
    Oct 31, 2023
    Messages:
    335
    Likes Received:
    196
    This is very true! The old *nix method of TAR+compress does this, which is combining files into one, and then compressing it in one go. Still, it can come in handy sometimes (a lot of small files of similar kind, like code or text). tar+7z is good.
     
Loading...
Similar Threads - achieve BEST file Forum Date
How do I achieve a Bass Stereo Image like this Mixing and Mastering Mar 6, 2024
How is this achieved? [solved] Mixing and Mastering Feb 24, 2024
How can i achieve this synth stab in "Something About You" - Rudimental & Elderbrook? how to make "that" sound Mar 11, 2023
default sound from canopener? any way to achieve that how to make "that" sound Sep 30, 2022
How on Gods Green Earth, do I achieve a Mix like this? Mixing and Mastering Nov 19, 2021
Loading...