# Compression test


- [Compression test](#compression-test)
  - [Test setup](#test-setup)
  - [Results](#results)
    - [Non compressible data](#non-compressible-data)
    - [Lots of data with mixed types and sizes](#lots-of-data-with-mixed-types-and-sizes)
    - [A lot of many small text files](#a-lot-of-many-small-text-files)
    - [A few text files](#a-few-text-files)
    - [A few mixed files](#a-few-mixed-files)

I made some test on multiple of the compression algorithms and tools found on linux.

## Test setup
The algorithms were tested on the following situations:
 - **Lots of data with mixed types and sizes:** A lot of various types of files of varying sizes. The total size of all the files is 4.36 GiB
 - **Non-compressible data:** A lot of barely compressible files, in thes case jpeg images. The total size of all the files is 0.62 GiB
 - **A lot of many small text files:** A lot of small text files, all the files are under 1MB in size. The total size of all the files is 0.96 GiB
 - **A few text files:** A few text files. The total size of all the files is 9.54 MiB
 - **A few mixed files:** A few files varying in size and type. The total size of all the files is 6.66 MiB

The algorithms tested were the following:
 - XZ
 - LZMA
 - PGZIP (parallel gzip)
 - PBZIP2 (parallel bzip2)
 - LZ4
 - PLZIP (parallel lzip)
 - ZSTD
 - RAR
 - 7ZIP
 - ZIP

The tests were run on a system with the following specs
 - Ryzen 9 3900X CPU
 - 32 GB DDR4 2133 MT/s memory
 - nvme gen4 ssd
 - Arch linux kernel version 6.12.8
 - xfs file system

Notes: All the algorithms were run with the maximum compression mode.

## Results
### Non compressible data

Test with "Non compressible data" sorted by "Time"

| Compression Algorithm |    Time | Compression | Time normalized compression |
| :-------------------- | ------: | ----------: | --------------------------: |
| PGZIP                 | 1.26777 |     1.00505 |                    0.784826 |
| LZ4                   |  1.2858 |    0.999934 |                    0.777779 |
| PBZIP2                |  3.8217 |     1.02804 |                    0.254526 |
| RAR                   | 13.2534 |     1.00414 |                   0.0751413 |
| ZIP                   | 16.3882 |     1.00645 |                   0.0606285 |
| 7ZIP                  | 17.2523 |     1.01995 |                   0.0568293 |
| ZSTD                  | 26.1111 |     1.01446 |                   0.0377519 |
| PLZIP                 | 37.1494 |     1.01846 |                   0.0264305 |
| XZ                    | 104.294 |     1.02264 |                    0.009376 |
| LZMA                  | 351.912 |     1.02573 |                  0.00277034 |


Test with "Non compressible data" sorted by "Compression"

| Compression Algorithm |    Time | Compression | Time normalized compression |
| :-------------------- | ------: | ----------: | --------------------------: |
| LZ4                   |  1.2858 |    0.999934 |                    0.777779 |
| RAR                   | 13.2534 |     1.00414 |                   0.0751413 |
| PGZIP                 | 1.26777 |     1.00505 |                    0.784826 |
| ZIP                   | 16.3882 |     1.00645 |                   0.0606285 |
| ZSTD                  | 26.1111 |     1.01446 |                   0.0377519 |
| PLZIP                 | 37.1494 |     1.01846 |                   0.0264305 |
| 7ZIP                  | 17.2523 |     1.01995 |                   0.0568293 |
| XZ                    | 104.294 |     1.02264 |                    0.009376 |
| LZMA                  | 351.912 |     1.02573 |                  0.00277034 |
| PBZIP2                |  3.8217 |     1.02804 |                    0.254526 |


Test with "Non compressible data" sorted by "Time normalized compression"

| Compression Algorithm |    Time | Compression | Time normalized compression |
| :-------------------- | ------: | ----------: | --------------------------: |
| PGZIP                 | 1.26777 |     1.00505 |                    0.784826 |
| LZ4                   |  1.2858 |    0.999934 |                    0.777779 |
| PBZIP2                |  3.8217 |     1.02804 |                    0.254526 |
| RAR                   | 13.2534 |     1.00414 |                   0.0751413 |
| ZIP                   | 16.3882 |     1.00645 |                   0.0606285 |
| 7ZIP                  | 17.2523 |     1.01995 |                   0.0568293 |
| ZSTD                  | 26.1111 |     1.01446 |                   0.0377519 |
| PLZIP                 | 37.1494 |     1.01846 |                   0.0264305 |
| XZ                    | 104.294 |     1.02264 |                    0.009376 |
| LZMA                  | 351.912 |     1.02573 |                  0.00277034 |

### Lots of data with mixed types and sizes

Test with "Lots of data with mixed types and sizes" sorted by "Time"

| Compression Algorithm |    Time | Compression | Time normalized compression |
| :-------------------- | ------: | ----------: | --------------------------: |
| PGZIP                 | 21.8042 |     1.70312 |                   0.0269288 |
| PBZIP2                | 23.2807 |     2.08055 |                   0.0206456 |
| LZ4                   | 38.7711 |     1.61392 |                   0.0159812 |
| RAR                   | 70.6871 |     2.76305 |                     0.00512 |
| 7ZIP                  |   84.22 |     3.56446 |                  0.00333112 |
| ZIP                   | 116.646 |     1.69067 |                  0.00507074 |
| PLZIP                 | 135.211 |       3.343 |                  0.00221234 |
| ZSTD                  | 138.242 |     3.09079 |                   0.0023404 |
| XZ                    | 236.184 |     3.46817 |                  0.00122081 |
| LZMA                  | 1208.73 |     3.51061 |                 0.000235662 |


Test with "Lots of data with mixed types and sizes" sorted by "Compression"

| Compression Algorithm |    Time | Compression | Time normalized compression |
| :-------------------- | ------: | ----------: | --------------------------: |
| LZ4                   | 38.7711 |     1.61392 |                   0.0159812 |
| ZIP                   | 116.646 |     1.69067 |                  0.00507074 |
| PGZIP                 | 21.8042 |     1.70312 |                   0.0269288 |
| PBZIP2                | 23.2807 |     2.08055 |                   0.0206456 |
| RAR                   | 70.6871 |     2.76305 |                     0.00512 |
| ZSTD                  | 138.242 |     3.09079 |                   0.0023404 |
| PLZIP                 | 135.211 |       3.343 |                  0.00221234 |
| XZ                    | 236.184 |     3.46817 |                  0.00122081 |
| LZMA                  | 1208.73 |     3.51061 |                 0.000235662 |
| 7ZIP                  |   84.22 |     3.56446 |                  0.00333112 |


Test with "Lots of data with mixed types and sizes" sorted by "Time normalized compression"

| Compression Algorithm |    Time | Compression | Time normalized compression |
| :-------------------- | ------: | ----------: | --------------------------: |
| PGZIP                 | 21.8042 |     1.70312 |                   0.0269288 |
| PBZIP2                | 23.2807 |     2.08055 |                   0.0206456 |
| LZ4                   | 38.7711 |     1.61392 |                   0.0159812 |
| RAR                   | 70.6871 |     2.76305 |                     0.00512 |
| ZIP                   | 116.646 |     1.69067 |                  0.00507074 |
| 7ZIP                  |   84.22 |     3.56446 |                  0.00333112 |
| ZSTD                  | 138.242 |     3.09079 |                   0.0023404 |
| PLZIP                 | 135.211 |       3.343 |                  0.00221234 |
| XZ                    | 236.184 |     3.46817 |                  0.00122081 |
| LZMA                  | 1208.73 |     3.51061 |                 0.000235662 |

### A lot of many small text files

Test with "A lot of many small text files" sorted by "Time"

| Compression Algorithm |    Time | Compression | Time normalized compression |
| :-------------------- | ------: | ----------: | --------------------------: |
| PGZIP                 | 3.20727 |     4.95067 |                   0.0629797 |
| LZ4                   | 4.07333 |     4.21619 |                   0.0582278 |
| PBZIP2                |  4.4668 |     6.37939 |                   0.0350933 |
| ZIP                   | 23.2824 |     3.73557 |                   0.0114978 |
| RAR                   | 34.9045 |     4.04528 |                  0.00708224 |
| 7ZIP                  | 46.3945 |     7.21744 |                  0.00298642 |
| ZSTD                  | 47.4384 |     7.19689 |                  0.00292904 |
| PLZIP                 | 49.5483 |     7.60392 |                   0.0026542 |
| XZ                    | 95.9468 |     7.61049 |                  0.00136948 |
| LZMA                  | 380.253 |     7.65804 |                 0.000343407 |


Test with "A lot of many small text files" sorted by "Compression"

| Compression Algorithm |    Time | Compression | Time normalized compression |
| :-------------------- | ------: | ----------: | --------------------------: |
| ZIP                   | 23.2824 |     3.73557 |                   0.0114978 |
| RAR                   | 34.9045 |     4.04528 |                  0.00708224 |
| LZ4                   | 4.07333 |     4.21619 |                   0.0582278 |
| PGZIP                 | 3.20727 |     4.95067 |                   0.0629797 |
| PBZIP2                |  4.4668 |     6.37939 |                   0.0350933 |
| ZSTD                  | 47.4384 |     7.19689 |                  0.00292904 |
| 7ZIP                  | 46.3945 |     7.21744 |                  0.00298642 |
| PLZIP                 | 49.5483 |     7.60392 |                   0.0026542 |
| XZ                    | 95.9468 |     7.61049 |                  0.00136948 |
| LZMA                  | 380.253 |     7.65804 |                 0.000343407 |


Test with "A lot of many small text files" sorted by "Time normalized compression"

| Compression Algorithm |    Time | Compression | Time normalized compression |
| :-------------------- | ------: | ----------: | --------------------------: |
| PGZIP                 | 3.20727 |     4.95067 |                   0.0629797 |
| LZ4                   | 4.07333 |     4.21619 |                   0.0582278 |
| PBZIP2                |  4.4668 |     6.37939 |                   0.0350933 |
| ZIP                   | 23.2824 |     3.73557 |                   0.0114978 |
| RAR                   | 34.9045 |     4.04528 |                  0.00708224 |
| 7ZIP                  | 46.3945 |     7.21744 |                  0.00298642 |
| ZSTD                  | 47.4384 |     7.19689 |                  0.00292904 |
| PLZIP                 | 49.5483 |     7.60392 |                   0.0026542 |
| XZ                    | 95.9468 |     7.61049 |                  0.00136948 |
| LZMA                  | 380.253 |     7.65804 |                 0.000343407 |

### A few text files

Test with "A few text files" sorted by "Time"

| Compression Algorithm |      Time | Compression | Time normalized compression |
| :-------------------- | --------: | ----------: | --------------------------: |
| PGZIP                 | 0.0423446 |     1.13541 |                     20.7993 |
| LZ4                   |  0.137808 |     1.13707 |                     6.38176 |
| ZIP                   |  0.241944 |     1.08497 |                     3.80951 |
| PBZIP2                |  0.258476 |     1.37485 |                       2.814 |
| RAR                   |  0.322799 |     1.10265 |                     2.80952 |
| 7ZIP                  |  0.433445 |     2.15245 |                     1.07185 |
| ZSTD                  |  0.993853 |     2.15193 |                    0.467573 |
| XZ                    |   1.64646 |     2.15047 |                    0.282433 |
| LZMA                  |    1.6571 |     2.14211 |                    0.281714 |
| PLZIP                 |   2.65034 |     2.14365 |                    0.176013 |


Test with "A few text files" sorted by "Compression"

| Compression Algorithm |      Time | Compression | Time normalized compression |
| :-------------------- | --------: | ----------: | --------------------------: |
| ZIP                   |  0.241944 |     1.08497 |                     3.80951 |
| RAR                   |  0.322799 |     1.10265 |                     2.80952 |
| PGZIP                 | 0.0423446 |     1.13541 |                     20.7993 |
| LZ4                   |  0.137808 |     1.13707 |                     6.38176 |
| PBZIP2                |  0.258476 |     1.37485 |                       2.814 |
| LZMA                  |    1.6571 |     2.14211 |                    0.281714 |
| PLZIP                 |   2.65034 |     2.14365 |                    0.176013 |
| XZ                    |   1.64646 |     2.15047 |                    0.282433 |
| ZSTD                  |  0.993853 |     2.15193 |                    0.467573 |
| 7ZIP                  |  0.433445 |     2.15245 |                     1.07185 |


Test with "A few text files" sorted by "Time normalized compression"

| Compression Algorithm |      Time | Compression | Time normalized compression |
| :-------------------- | --------: | ----------: | --------------------------: |
| PGZIP                 | 0.0423446 |     1.13541 |                     20.7993 |
| LZ4                   |  0.137808 |     1.13707 |                     6.38176 |
| ZIP                   |  0.241944 |     1.08497 |                     3.80951 |
| PBZIP2                |  0.258476 |     1.37485 |                       2.814 |
| RAR                   |  0.322799 |     1.10265 |                     2.80952 |
| 7ZIP                  |  0.433445 |     2.15245 |                     1.07185 |
| ZSTD                  |  0.993853 |     2.15193 |                    0.467573 |
| XZ                    |   1.64646 |     2.15047 |                    0.282433 |
| LZMA                  |    1.6571 |     2.14211 |                    0.281714 |
| PLZIP                 |   2.65034 |     2.14365 |                    0.176013 |

### A few mixed files

Test with "A few mixed files" sorted by "Time"

| Compression Algorithm |      Time | Compression | Time normalized compression |
| :-------------------- | --------: | ----------: | --------------------------: |
| PGZIP                 | 0.0472157 |      2.8845 |                     7.34247 |
| PBZIP2                | 0.0996904 |      3.3937 |                     2.95579 |
| LZ4                   |  0.178676 |     2.56754 |                     2.17979 |
| RAR                   |  0.187353 |     3.48817 |                     1.53018 |
| ZIP                   |  0.240878 |     2.77821 |                      1.4943 |
| 7ZIP                  |   0.53761 |     4.15235 |                    0.447959 |
| LZMA                  |   1.36919 |      4.1687 |                      0.1752 |
| XZ                    |   1.56128 |     4.16868 |                    0.153646 |
| ZSTD                  |   1.64639 |     3.75015 |                    0.161965 |
| PLZIP                 |   2.20772 |     4.18295 |                    0.108287 |


Test with "A few mixed files" sorted by "Compression"

| Compression Algorithm |      Time | Compression | Time normalized compression |
| :-------------------- | --------: | ----------: | --------------------------: |
| LZ4                   |  0.178676 |     2.56754 |                     2.17979 |
| ZIP                   |  0.240878 |     2.77821 |                      1.4943 |
| PGZIP                 | 0.0472157 |      2.8845 |                     7.34247 |
| PBZIP2                | 0.0996904 |      3.3937 |                     2.95579 |
| RAR                   |  0.187353 |     3.48817 |                     1.53018 |
| ZSTD                  |   1.64639 |     3.75015 |                    0.161965 |
| 7ZIP                  |   0.53761 |     4.15235 |                    0.447959 |
| XZ                    |   1.56128 |     4.16868 |                    0.153646 |
| LZMA                  |   1.36919 |      4.1687 |                      0.1752 |
| PLZIP                 |   2.20772 |     4.18295 |                    0.108287 |


Test with "A few mixed files" sorted by "Time normalized compression"

| Compression Algorithm |      Time | Compression | Time normalized compression |
| :-------------------- | --------: | ----------: | --------------------------: |
| PGZIP                 | 0.0472157 |      2.8845 |                     7.34247 |
| PBZIP2                | 0.0996904 |      3.3937 |                     2.95579 |
| LZ4                   |  0.178676 |     2.56754 |                     2.17979 |
| RAR                   |  0.187353 |     3.48817 |                     1.53018 |
| ZIP                   |  0.240878 |     2.77821 |                      1.4943 |
| 7ZIP                  |   0.53761 |     4.15235 |                    0.447959 |
| LZMA                  |   1.36919 |      4.1687 |                      0.1752 |
| ZSTD                  |   1.64639 |     3.75015 |                    0.161965 |
| XZ                    |   1.56128 |     4.16868 |                    0.153646 |
| PLZIP                 |   2.20772 |     4.18295 |                    0.108287 |