Best Compression?

[wheelman]

I use .rar with best compression method and check the boxes for 'solid archive' and 'put recovery record'.

This can only be done by selecting the 'add files to archive' menu option when right clicking on the file to zip.

For large files I split them into a reasonable size.

[orinmx]

rar man ;D

[Xenocrates]

ACE - far superior compression. It's a pity it's not as popular. An ACE archive has all of the features of the ZIP and RAR standards plus the ability to rebuild itself in the event of corruption from data transfer. All that RAR does is provide a Simple File Verification (SFV) text file with the hexadecimal hash-list for each file.

ACE archives take that a step further by allowing you to rebuild a damaged archive with a bad checksum. It does this through using the archive bits of other files in the archive set that are ok, and recalculating it's own hex hash from the hex hash of the good files. This is similar to how a RAID 5 disk set would rebuild data for a replaced drive. The Entire archive can then be rebuilt. With a RAR archive set, you need to have the SFV file to rebuild the set. ACE files
have this data in the file itself.

Furthermore, unlike the ZIP and the RAR standards, the ACE algorithm uses multiple levels of compression for each operation using a highly sophisticated algorithm that is actually built on the LHA and ARJ standards. This is to provide unprecedented levels of compression especially for binary and multimedia files. That means, each file is compressed several times, without adding any bytes to the physical size of the file as would the ZIP and the RAR standards.

Try compressing an MP3 file with WinZip and WinRAR. Because MP3's are already compressed, those standards will not reduce the size of the file significantly, or at all. In the case of WinZip, the file size will actually increase in some cases. Only the ACE standard can physically reduce the size of the already compressed multimedia standards developed by the Motion Pictures Executive Group (MPEG), the Joint Pictures Executive Group (JPEG), Compuserve's Graphical Interleave Format (GIF), Apple's Quicktime Multimedia Formats or the newer 32-bit Windows Bitmap formats.

Winzip is only more popular because the ZIP standard came first with the pkunzip/pkzip line of utilities that was later bought by NikoMac Computing. It was also easy to use and the first to get a GUI update. This skyrocketed its internet usage exponentially over the years.

When the RAR standard came out, it quickly dwarfed the ZIP standard's capabilities. But at the time it became popular, only pirates used it to distribute warez. This was because the RAR standard was the first to create the use of an archive set, that didn't require the use of diskettes (as does the Zip standard). This became very convenient for distributing games and software via ftp sites since end users don't have to download one MASSIVE ZIP file.

Furthermore, if the Zip file became corrupt midway through the download, you'd have to start all over again. RAR offered users a choice of downloading individual chunks of consistent size, thus significantly reducing data corruption and the necessity for a total redownload.

ACE came pretty late along the lines, incorporating all of the previous compression technologies to provide the most superior performance. However, the ACE standard is maintained by a German Company and not much marketing has been done for it. In fact, it was only recently that the ACE standard got an ENGLISH language update. Therefore it will never become as popular as the ZIP standard - which is now a built in feature of Windows XP.

[wheelman]

So what about WinACE the program does it use the same methods?

[Xenocrates]

I deliberately steered clear of program names because the Standard is what is established by the International Standards Organisation. But the best compression software out there is definitely WinACE, not only because it uses the ACE standard, but it also has the best user interface (when I last checked) and comes with some VERY powerful tools.

[wheelman]

But I rate WInrar still mainly becuase it under a meg and its compression rate is very good. But I knew about WinAce long time still.

[deakie]

well, you convinced me xeno, now i just got to convince everybody i know who always using rar... :-\

[Xenocrates]

Good luck. RAR has been in the spotlight for a very loong time. It's just like WinZip. Do you know how many non-tech savvy people out there who use the internet prefer WinZip to WinRAR? Try indoctrinating them for a change. That's a lot harder than convincing a techie that WinACE is better than WinRAR hands down. At least a techie will notice the diff in filesizes and be convinced. An ordinary end user will however think: "but what's so wrong with WinZip? It works doesn't it?"

[matronyx]

All those you listed are inferior when it comes to tar | gzip ;D

[wheelman]

I'll be doing a test shortly so be on the lookout

[Xenocrates]

It would seem as though I'm rubbing off on someone....

Pretty stubborn aren't ya Matro? ;D

I've used GZIP and TAR before (were't those originally UNIX based compressors?) and although they DO pack quite a punch (pun intended) I have yet to see physical evidence of a compressor that can pack tighter than ACE. ACE layers the file's data on top of each other in addition to compressing them (hence the smaller footprint). That's why it can compress even MPEG files!

ACE rulez man.

[wheelman]

I say RAR rulez

[wheelman]

It really depends on the type of file you compressing

[matronyx]

It really depends on the type of file you compressing


Let's all download the tlogo.gif file and compress and see.....

http://www.techjamaica.com/images/tjlogo.gif

[matronyx]

Ok,....my bad ;D

That wouldn't work. Anybody have a text file or supn we could use?

[wheelman]

Results

[wheelman]

and with a mpeg movie

[wheelman]

come'on ppl post your own compression findings.

[matronyx]

Ok...since you asked so nicely....

[matronyx]

where would i get the movie file??

[deakie]

just use your own. :P

[wheelman]

http://www.veriotrainingfmyers.testgroup24.com/stuffy/Evo_VIII_Ad.mpeg

[matronyx]

Ok... I'm gonna use a movie you all should be able to access....
http://progressive1.stream.aol.com/wb/gl/wbonline/progressive/thematrix/us/med/trailer_final_320_dl.zip

[deakie]

thats like 17mb and 32mb files. think of our guys on dial up man. :-\
53.5kb/sec offa yur space matron.... 8)
and me 'hall n pull up' devs at 59.9kb/sec.... :P

[wheelman]

My file is 5.xxx megs ;D

[deakie]

true...a mi a drive da red car deh yu kno... ;D a dev in a di silva one... :P

[wheelman]

jus cool nuh my yout ;D

[Xenocrates]

I'm sure you have at least 1 MP3 on your hard drive somewhere. That's the best type of file to test it with. MP3 files have 10 layers of compression tightly tucked in already. Try compressing an MP3 file with maximum compression using all three and see the results. Compressing such a tiny GIF as the techj logo will not give you a big enough archive to look at in Windows. The file will always be 1k.

[Arch_Angel]

Well, what can I say? Xeno has touched on all the good points about each compression utility. And I'm afraid I fall into one of those categories that mainly used .zip because it was popular.

I download a lot of programs and they are in a .zip format, which required me to have a utility that opened it. I don't do much archiving/compressing of files. Maybe when emailing a friend a file or sending a large file via IRC/IM etc. And when I do I just use what I have, which is winzip.

I also have WinRAR and have used it a few times, since a few warez sites use this compression. But I never really got a hand of it. Most times I got some error when opening these .rar files with WinRAR.

I have heard of ace but never used it or ran into a .ace file.

[wheelman]

ok gonna post the pics of the compression tests soon

[wheelman]

You wouldn't believe who won...I tell you seh you learn something new everyday....Well take a look...

[wheelman]

and here are the specs on the fiel for all you detailed techies...

[wheelman]

I gotta check if .cab compression is the best thing way to go.
Gonna do some test with a really large movie file...brb

[wheelman]

Winace time for compression the movie as a cab file.

i was doig some other stuff on the pc at the time.

[wheelman]

results

[matronyx]

It looks like cab leads all the way... :o

Notice that there are two .zip archives for the movie downloaded from www.thematrix.com
The smaller of the two was the original zip file from the site. It seems those guys do not use winzip to manage their archives.

Programs used in the test:

bzip2 1.0.2
gzip 1.3.3
winace 2.2
winrar 3.11
winzip 8.1

All programs had maximum compression turned on.

[orinmx]

Who said Microsoft was good for nothing ;D

[wheelman]

Fi real Matronyx...cab a gwaan well, but I think it depends on the file type really.

[matronyx]

Who said Microsoft was good for nothing ;D


uhhh.... i didn't say that.
I said they are not good for ANYTHING... lol.

just kidding...they have some good in their name, if it werent for them, how would we know what was better? Who would linux,bsd,unix systems compete with?

[Xenocrates]

Indeed. The compression tests on the MP3 file devnull posted proved my point about ACE versus ZIP and RAR compressions. But it would appear as though the the nature of the file's inner compression affects the results dramatically with the other formats and it would also appear as though you guys are using different versions of the same software. Each subsequent version has a more powerful algorithm. Please bear that in mind.

If you're going to compare the ACE, ZIP and RAR compression with CAB, then that would be a bit unfair don't you think? Do you remember installing Windows 95 from a set of 20+ diskettes? Don't you remember the file format of the archive on each disk? It was a CAB file! That CAB file contained about 3-4 mb of already compressed data for each disk, which was only 1.6mb in size!

The Cabinet standard is an industrial strength compression format that was developed for backing up the entire contents of a hard drive (which at the time, the biggest ones were 60 mb) onto 10 mb tapes.

Microsoft pioneered the format for their use when distributing software. But if you look on your game CD, you will often notice a very BIG cabinet file usually with a name like data1.cab. Have you ever installed a game from 1 CD that occupied like 1.xx Gigabytes on your hard drive? Ok then...

While we're on industrial strength compressors, I'm sure you've heard of TREE and PAK files. The Wing Commander games used TREE files heavily. They were able to compress gigabytes of full motion video data so that they could fit on a single 650mb CD. Arch_Angel should be familiar with what I'm talking about. Usually Tree files have the extension *.tre and can allow for the streaming of data from the file directly into main memory without decompressing it's contents to the hard drive.

A similar compression format are PAK files. All you people who've played any ID game like DOOM, Wolfenstein etc. or even Duke Nukem 3D from 3D Realms will know PAK files. They're the same as TREE files and were in fact the precursors to TREE files. It provides the same kind of functionality as the TREE file, albeit, with not as powerful a compression.

What we should do though devnull, is post the version of the compressor you are using. I noticed that you and Matronyx are using two different qualities for the same song. The higher the quality, the lesser the effect of compression. Try to keep external variables like version number, song quality etc. consistent for consistent results. The version is also important because recent developments into RAR technology actually allows it to outdo the ACE standard.

Try compressing a DLL file next. I've done my own test here and I will post my results shortly.

[yogi_hm]

Cab has really shined. I agree with devnull, it seems to depend on the file it self. For a simple explanation of how files are comprressed visit the link below.

http://www.howstuffworks.com/file-compression.htm

[Xenocrates]

This is a test with a DLL file. The versions used are:

WinZip 8.0
WinRAR 3.20
WinACE 2.2

[Xenocrates]

This is a test with an EXE file.

[Xenocrates]

But this test is with a BINARY file dumped from a ROM Chip. Notice the difference in this test.

[Xenocrates]

But look at this MP3 test and compare it with DevNull's... The RAR compression actually outdoes the ACE compression. Remember, this is the very latest version of WinRAR, which was released TODAY!

RARLabs has actually improved their algorithm, making it that much stronger than even the latest version of WinACE, the last update for which was in July of last year. Currently, the RAR standard is the king of commercial compression.....well at least until a new version of WinACE comes out.

[wheelman]

Try doing a compression test with the cab extension on those files.

[Xenocrates]

We already know what the results are for CAB files. No need to test those. ;) The original poll was concerned with ZIP, RAR and ACE. That's what I posted in reference to.

[matronyx]

We already know what the results are for CAB files. No need to test those. ;) The original poll was concerned with ZIP, RAR and ACE. That's what I posted in reference to.


Ok guys....I didn't vote for any of those methods.
I said gzip was better. I turns out that I was wrong. bzip is better ;D

As far as using the latest winace goes Xeno, keep in mind that we used winace to create the .jar, .ace, and .cab files.

I will keep sticking with my good ole linux compressors though....
I am gonna look for the industrial ones and post em soon.

[Ropy]

rar works fine for me. How good are those linux compression???

[Dea_mos]

Still on the topic of compression..has any one seen the massive...devaluation ;D of the wave and mp3 file system...by the oog converter.. much smaller than WMa

man this makes mp3 look like godzilla...

its called ogg yup you heard it OGG...there is even an OGG player..

whats up with these new ogg PLAYERS...anyone know of a burning software that can burn ogg format...

MP3 CD RIPPER converts to ogg..incase you'll wannah know..

ohh i support ZIP...(i might be old fashion though) when it comes to certain things

[Ropy]

OGG is pretty good I hear, I think most burning softwares support it. It's like Napster and Kazaa, both were out around the same time. But people just seemed to prefer the MP3 compression over OGG. Not that it's dead...

[Dea_mos]

i still beleive that the OGG format could beat the mp3 and the ZIP..MATRO

well i haven't even seen another ogg player than the one that is bundled with mp3 cd ripper

[Dea_mos]

any techie like my new dance moves... ;D ;D ..i am teaching dance now

[wheelman]

have you compared the bitrate - size of OGG and MP3?