1## zlib-ng 2*zlib data compression library for the next generation systems* 3 4Maintained by Hans Kristian Rosbach 5 aka Dead2 (zlib-ng àt circlestorm dót org) 6 7|CI|Status| 8|:-|-| 9|GitHub Actions|[](https://github.com/zlib-ng/zlib-ng/actions) [](https://github.com/zlib-ng/zlib-ng/actions) [](https://github.com/zlib-ng/zlib-ng/actions)| 10|Buildkite|[](https://buildkite.com/circlestorm-productions/zlib-ng)| 11|CodeFactor|[](https://www.codefactor.io/repository/github/zlib-ng/zlib-ng)| 12|OSS-Fuzz|[](https://bugs.chromium.org/p/oss-fuzz/issues/list?sort=-opened&can=1&q=proj:zlib-ng) 13|Codecov|[](https://codecov.io/github/zlib-ng/zlib-ng/)| 14 15 16Features 17-------- 18 19* Zlib compatible API with support for dual-linking 20* Modernized native API based on zlib API for ease of porting 21* Modern C11 syntax and a clean code layout 22* Deflate medium and quick algorithms based on Intel’s zlib fork 23* Support for CPU intrinsics when available 24 * Adler32 implementation using SSSE3, AVX2, AVX512, AVX512-VNNI, Neon, VMX & VSX 25 * CRC32-B implementation using PCLMULQDQ, VPCLMULQDQ, ACLE, & IBM Z 26 * Hash table implementation using CRC32-C intrinsics on x86 and ARM 27 * Slide hash implementations using SSE2, AVX2, Neon, VMX & VSX 28 * Compare256 implementations using SSE2, AVX2, Neon, & POWER9 29 * Inflate chunk copying using SSE2, AVX, Neon & VSX 30 * Support for hardware-accelerated deflate using IBM Z DFLTCC 31* Unaligned memory read/writes and large bit buffer improvements 32* Includes improvements from Cloudflare and Intel forks 33* Configure, CMake, and NMake build system support 34* Comprehensive set of CMake unit tests 35* Code sanitizers, fuzzing, and coverage 36* GitHub Actions continuous integration on Windows, macOS, and Linux 37 * Emulated CI for ARM, AARCH64, PPC, PPC64, SPARC64, S390x using qemu 38 39 40History 41------- 42 43The motivation for this fork came after seeing several 3rd party 44contributions containing new optimizations not getting implemented 45into the official zlib repository. 46 47Mark Adler has been maintaining zlib for a very long time, and he has 48done a great job and hopefully he will continue for a long time yet. 49The idea of zlib-ng is not to replace zlib, but to co-exist as a 50drop-in replacement with a lower threshold for code change. 51 52zlib has a long history and is incredibly portable, even supporting 53lots of systems that predate the Internet. This is great, but it does 54complicate further development and maintainability. 55The zlib code has numerous workarounds for old compilers that do not 56understand ANSI-C or to accommodate systems with limitations such as 57operating in a 16-bit environment. 58 59Many of these workarounds are only maintenance burdens, some of them 60are pretty huge code-wise. For example, the [v]s[n]printf workaround 61code has a whopping 8 different implementations just to cater to 62various old compilers. With this many workarounds cluttered throughout 63the code, new programmers with an idea/interest for zlib will need 64to take some time to figure out why all of these seemingly strange 65things are used, and how to work within those confines. 66 67So I decided to make a fork, merge all the Intel optimizations, merge 68the Cloudflare optimizations that did not conflict, plus a couple 69of other smaller patches. Then I started cleaning out workarounds, 70various dead code, all contrib and example code as there is little 71point in having those in this fork for various reasons. 72 73A lot of improvements have gone into zlib-ng since its start, and 74numerous people and companies have contributed both small and big 75improvements, or valuable testing. 76 77Please read LICENSE.md, it is very simple and very liberal. 78 79 80Build 81----- 82 83There are two ways to build zlib-ng: 84 85### Cmake 86 87To build zlib-ng using the cross-platform makefile generator cmake. 88 89``` 90cmake . 91cmake --build . --config Release 92ctest --verbose -C Release 93``` 94 95Alternatively, you can use the cmake configuration GUI tool ccmake: 96 97``` 98ccmake . 99``` 100 101### Configure 102 103To build zlib-ng using the bash configure script: 104 105``` 106./configure 107make 108make test 109``` 110 111Build Options 112------------- 113 114| CMake | configure | Description | Default | 115|:-------------------------|:-------------------------|:--------------------------------------------------------------------------------------|---------| 116| ZLIB_COMPAT | --zlib-compat | Compile with zlib compatible API | OFF | 117| ZLIB_ENABLE_TESTS | | Build test binaries | ON | 118| WITH_GZFILEOP | --without-gzfileops | Compile with support for gzFile related functions | ON | 119| WITH_OPTIM | --without-optimizations | Build with optimisations | ON | 120| WITH_NEW_STRATEGIES | --without-new-strategies | Use new strategies | ON | 121| WITH_NATIVE_INSTRUCTIONS | --native | Compiles with full instruction set supported on this host (gcc/clang -march=native) | OFF | 122| WITH_SANITIZER | | Build with sanitizer (memory, address, undefined) | OFF | 123| WITH_FUZZERS | | Build test/fuzz | OFF | 124| WITH_BENCHMARKS | | Build test/benchmarks | OFF | 125| WITH_MAINTAINER_WARNINGS | | Build with project maintainer warnings | OFF | 126| WITH_CODE_COVERAGE | | Enable code coverage reporting | OFF | 127 128 129Install 130------- 131 132WARNING: We do not recommend manually installing unless you really 133know what you are doing, because this can potentially override the system 134default zlib library, and any incompatibility or wrong configuration of 135zlib-ng can make the whole system unusable, requiring recovery or reinstall. 136If you still want a manual install, we recommend using the /opt/ path prefix. 137 138For Linux distros, an alternative way to use zlib-ng (if compiled in 139zlib-compat mode) instead of zlib, is through the use of the 140_LD_PRELOAD_ environment variable. If the program is dynamically linked 141with zlib, then zlib-ng will temporarily be used instead by the program, 142without risking system-wide instability. 143 144``` 145LD_PRELOAD=/opt/zlib-ng/libz.so.1.2.11.zlib-ng /usr/bin/program 146``` 147 148### Cmake 149 150To install zlib-ng system-wide using cmake: 151 152``` 153cmake --build . --target install 154``` 155 156### Configure 157 158To install zlib-ng system-wide using the configure script: 159 160``` 161make install 162``` 163 164Contributing 165------------ 166 167Zlib-ng is a aiming to be open to contributions, and we would be delighted to 168receive pull requests on github. 169Just remember that any code you submit must be your own and it must be zlib licensed. 170Help with testing and reviewing of pull requests etc is also very much appreciated. 171 172If you are interested in contributing, please consider joining our 173IRC channel #zlib-ng on the Freenode IRC network. 174 175 176Acknowledgments 177---------------- 178 179Thanks to Servebolt.com for sponsoring my maintainership of zlib-ng. 180 181Thanks go out to all the people and companies who have taken the time to contribute 182code reviews, testing and/or patches. Zlib-ng would not have been nearly as good without you. 183 184The deflate format used by zlib was defined by Phil Katz. 185The deflate and zlib specifications were written by L. Peter Deutsch. 186 187zlib was originally created by Jean-loup Gailly (compression) 188and Mark Adler (decompression). 189 190 191Advanced Build Options 192---------------------- 193 194| CMake | configure | Description | Default | 195|:--------------------------------|:----------------------|:--------------------------------------------------------------------|------------------------| 196| ZLIB_DUAL_LINK | | Dual link tests with system zlib | OFF | 197| FORCE_SSE2 | --force-sse2 | Skip runtime check for SSE2 instructions (Always on for x86_64) | OFF (x86) | 198| FORCE_TZCNT | --force-tzcnt | Skip runtime check for TZCNT instructions | OFF | 199| WITH_AVX2 | | Build with AVX2 intrinsics | ON | 200| WITH_AVX512 | | Build with AVX512 intrinsics | ON | 201| WITH_AVX512VNNI | | Build with AVX512VNNI intrinsics | ON | 202| WITH_SSE2 | | Build with SSE2 intrinsics | ON | 203| WITH_SSE41 | | Build with SSE41 intrinsics | ON | 204| WITH_SSE42 | | Build with SSE42 intrinsics | ON | 205| WITH_PCLMULQDQ | | Build with PCLMULQDQ intrinsics | ON | 206| WITH_VPCLMULQDQ | --without-vpclmulqdq | Build with VPCLMULQDQ intrinsics | ON | 207| WITH_ACLE | --without-acle | Build with ACLE intrinsics | ON | 208| WITH_NEON | --without-neon | Build with NEON intrinsics | ON | 209| WITH_ALTIVEC | --without-altivec | Build with AltiVec (VMX) intrinsics | ON | 210| WITH_POWER8 | --without-power8 | Build with POWER8 optimisations | ON | 211| WITH_CRC32_VX | --without-crc32-vx | Build with vectorized CRC32 on IBM Z | ON | 212| WITH_DFLTCC_DEFLATE | --with-dfltcc-deflate | Build with DFLTCC intrinsics for compression on IBM Z | OFF | 213| WITH_DFLTCC_INFLATE | --with-dfltcc-inflate | Build with DFLTCC intrinsics for decompression on IBM Z | OFF | 214| WITH_UNALIGNED | --without-unaligned | Allow optimizations that use unaligned reads if safe on current arch| ON | 215| WITH_INFLATE_STRICT | | Build with strict inflate distance checking | OFF | 216| WITH_INFLATE_ALLOW_INVALID_DIST | | Build with zero fill for inflate invalid distances | OFF | 217| INSTALL_UTILS | | Copy minigzip and minideflate during install | OFF | 218 219 220Related Projects 221---------------- 222 223* Fork of the popular minigzip https://github.com/zlib-ng/minizip-ng 224* Python tool to benchmark minigzip/minideflate https://github.com/zlib-ng/deflatebench 225* Python tool to benchmark pigz https://github.com/zlib-ng/pigzbench 226* 3rd party patches for zlib-ng compatibility https://github.com/zlib-ng/patches 227
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}
){kind=link}