stefan pushed a commit to branch master.
http://git.enlightenment.org/core/efl.git/commit/?id=4314257d8cf9c367cab2333cfdc7b10b0109de90
commit 4314257d8cf9c367cab2333cfdc7b10b0109de90
Author: Stefan Schmidt <s.schm...@samsung.com>
Date: Thu May 7 10:52:54 2015 +0200
lz4: Update our internal copy to release r128
Looking through the git log it is unclear which release we used before as
nobody
stated it there. :/ We updated after the security issues last year so my
best
guess is that we have something like r119.
To see what changed I now included the NEWS file and also the LICENSE file
from
upstream. Upstream in now hosted here: https://github.com/Cyan4973/lz4 and
http://www.lz4.info
I recommend STRONGLY that you check if your distro ships liblz4 as an up to
date library package and use the --enable-liblz4 configure option to use the
system version. I consider making the system version default for upcoming
releases and only carry the internal one as fallback for systems that do not
provide it.
Fix T2374
---
src/Makefile_Emile.am | 4 +-
src/static_libs/lz4/LICENSE | 24 ++
src/static_libs/lz4/NEWS | 153 +++++++
src/static_libs/lz4/README | 7 -
src/static_libs/lz4/README.md | 19 +
src/static_libs/lz4/lz4.c | 918 ++++++++++++++++++++++++------------------
src/static_libs/lz4/lz4.h | 208 +++++-----
src/static_libs/lz4/lz4hc.c | 886 +++++++++++++++++-----------------------
src/static_libs/lz4/lz4hc.h | 125 +++---
9 files changed, 1268 insertions(+), 1076 deletions(-)
diff --git a/src/Makefile_Emile.am b/src/Makefile_Emile.am
index 4d6e43f..fe2515a 100644
--- a/src/Makefile_Emile.am
+++ b/src/Makefile_Emile.am
@@ -58,7 +58,9 @@ if ! ENABLE_LIBLZ4
lib_emile_libemile_la_LIBADD += @LIBLZ4_LIBS@
endif
-EXTRA_DIST += static_libs/lz4/README \
+EXTRA_DIST += static_libs/lz4/README.md \
+static_libs/lz4/LICENSE \
+static_libs/lz4/NEWS \
static_libs/lz4/lz4.c \
static_libs/lz4/lz4.h \
static_libs/lz4/lz4hc.c \
diff --git a/src/static_libs/lz4/LICENSE b/src/static_libs/lz4/LICENSE
new file mode 100644
index 0000000..b566df3
--- /dev/null
+++ b/src/static_libs/lz4/LICENSE
@@ -0,0 +1,24 @@
+LZ4 Library
+Copyright (c) 2011-2014, Yann Collet
+All rights reserved.
+
+Redistribution and use in source and binary forms, with or without
modification,
+are permitted provided that the following conditions are met:
+
+* Redistributions of source code must retain the above copyright notice, this
+ list of conditions and the following disclaimer.
+
+* Redistributions in binary form must reproduce the above copyright notice,
this
+ list of conditions and the following disclaimer in the documentation and/or
+ other materials provided with the distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
+ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
+WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
+DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR
+ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
+(INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
+LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON
+ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
+SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
\ No newline at end of file
diff --git a/src/static_libs/lz4/NEWS b/src/static_libs/lz4/NEWS
new file mode 100644
index 0000000..2eeb948
--- /dev/null
+++ b/src/static_libs/lz4/NEWS
@@ -0,0 +1,153 @@
+r128:
+New : lz4cli sparse file support
+New : command -m, to compress multiple files in a single command
+Fixed : Restored lz4hc compression ratio (was slightly lower since r124)
+New : lz4 cli supports long commands
+New : lz4frame & lz4cli frame content size support
+New : lz4frame supports skippable frames
+Changed:Default "make install" directory is /usr/local
+New : lz4 cli supports "pass-through" mode
+New : datagen can generate sparse files
+New : scan-build tests
+New : g++ compatibility tests
+New : arm cross-compilation test
+Fixed : Fuzzer + frametest compatibility with NetBSD (issue #48)
+Added : Visual project directory
+Updated:Man page & Specification
+
+r127:
+N/A : added a file on SVN
+
+r126:
+New : lz4frame API is now integrated into liblz4
+Fixed : GCC 4.9 bug on highest performance settings, reported by Greg Slazinski
+Fixed : bug within LZ4 HC streaming mode, reported by James Boyle
+Fixed : older compiler don't like nameless unions, reported by Cheyi Lin
+Changed : lz4 is C90 compatible
+Changed : added -pedantic option, fixed a few mminor warnings
+
+r125:
+Changed : endian and alignment code
+Changed : directory structure : new "lib" directory
+Updated : lz4io, now uses lz4frame
+Improved: slightly improved decoding speed
+Fixed : LZ4_compress_limitedOutput(); Special thanks to Christopher Speller !
+Fixed : some alignment warnings under clang
+Fixed : deprecated function LZ4_slideInputBufferHC()
+
+r124:
+New : LZ4 HC streaming mode
+Fixed : LZ4F_compressBound() using null preferencesPtr
+Updated : xxHash to r38
+Updated library number, to 1.4.0
+
+r123:
+Added : experimental lz4frame API, thanks to Takayuki Matsuoka and Christopher
Jackson for testings
+Fix : s390x support, thanks to Nobuhiro Iwamatsu
+Fix : test mode (-t) no longer requires confirmation, thanks to Thary Nguyen
+
+r122:
+Fix : AIX & AIX64 support (SamG)
+Fix : mips 64-bits support (lew van)
+Added : Examples directory, using code examples from Takayuki Matsuoka
+Updated : Framing specification, to v1.4.1
+Updated : xxHash, to r36
+
+r121:
+Added : Makefile : install for kFreeBSD and Hurd (Nobuhiro Iwamatsu)
+Fix : Makefile : install for OS-X and BSD, thanks to Takayuki Matsuoka
+
+r120:
+Modified : Streaming API, using strong types
+Added : LZ4_versionNumber(), thanks to Takayuki Matsuoka
+Fix : OS-X : library install name, thanks to Clemens Lang
+Updated : Makefile : synchronize library version number with lz4.h, thanks to
Takayuki Matsuoka
+Updated : Makefile : stricter compilation flags
+Added : pkg-config, thanks to Zbigniew Jędrzejewski-Szmek (issue 135)
+Makefile : lz4-test only test native binaries, as suggested by Michał Górny
(issue 136)
+Updated : xxHash to r35
+
+r119:
+Fix : Issue 134 : extended malicious address space overflow in 32-bits mode
for some specific configurations
+
+r118:
+New : LZ4 Streaming API (Fast version), special thanks to Takayuki Matsuoka
+New : datagen : parametrable synthetic data generator for tests
+Improved : fuzzer, support more test cases, more parameters, ability to jump
to specific test
+fix : support ppc64le platform (issue 131)
+fix : Issue 52 (malicious address space overflow in 32-bits mode when using
large custom format)
+fix : Makefile : minor issue 130 : header files permissions
+
+r117:
+Added : man pages for lz4c and lz4cat
+Added : automated tests on Travis, thanks to Takayuki Matsuoka !
+fix : block-dependency command line (issue 127)
+fix : lz4fullbench (issue 128)
+
+r116:
+hotfix (issue 124 & 125)
+
+r115:
+Added : lz4cat utility, installed on POSX systems (issue 118)
+OS-X compatible compilation of dynamic library (issue 115)
+
+r114:
+Makefile : library correctly compiled with -O3 switch (issue 114)
+Makefile : library compilation compatible with clang
+Makefile : library is versioned and linked (issue 119)
+lz4.h : no more static inline prototypes (issue 116)
+man : improved header/footer (issue 111)
+Makefile : Use system default $(CC) & $(MAKE) variables (issue 112)
+xxhash : updated to r34
+
+r113:
+Large decompression speed improvement for GCC 32-bits. Thanks to Valery
Croizier !
+LZ4HC : Compression Level is now a programmable parameter (CLI from 4 to 9)
+Separated IO routines from command line (lz4io.c)
+Version number into lz4.h (suggested by Francesc Alted)
+
+r112:
+quickfix
+
+r111 :
+Makefile : added capability to install libraries
+Modified Directory tree, to better separate libraries from programs.
+
+r110 :
+lz4 & lz4hc : added capability to allocate state & stream state with custom
allocator (issue 99)
+fuzzer & fullbench : updated to test new functions
+man : documented -l command (Legacy format, for Linux kernel compression)
(issue 102)
+cmake : improved version by Mika Attila, building programs and libraries
(issue 100)
+xxHash : updated to r33
+Makefile : clean also delete local package .tar.gz
+
+r109 :
+lz4.c : corrected issue 98 (LZ4_compress_limitedOutput())
+Makefile : can specify version number from makefile
+
+r108 :
+lz4.c : corrected compression efficiency issue 97 in 64-bits chained mode
(-BD) for streams > 4 GB (thanks Roman Strashkin for reporting)
+
+r107 :
+Makefile : support DESTDIR for staged installs. Thanks Jorge Aparicio.
+Makefile : make install installs both lz4 and lz4c (Jorge Aparicio)
+Makefile : removed -Wno-implicit-declaration compilation switch
+lz4cli.c : include <stduni.h> for isatty() (Luca Barbato)
+lz4.h : introduced LZ4_MAX_INPUT_SIZE constant (Shay Green)
+lz4.h : LZ4_compressBound() : unified macro and inline definitions (Shay Green)
+lz4.h : LZ4_decompressSafe_partial() : clarify comments (Shay Green)
+lz4.c : LZ4_compress() verify input size condition (Shay Green)
+bench.c : corrected a bug in free memory size evaluation
+cmake : install into bin/ directory (Richard Yao)
+cmake : check for just C compiler (Elan Ruusamae)
+
+r106 :
+Makefile : make dist modify text files in the package to respect Unix EoL
convention
+lz4cli.c : corrected small display bug in HC mode
+
+r105 :
+Makefile : New install script and man page, contributed by Prasad Pandit
+lz4cli.c : Minor modifications, for easier extensibility
+COPYING : added license file
+LZ4_Streaming_Format.odt : modified file name to remove white space characters
+Makefile : .exe suffix now properly added only for Windows target
diff --git a/src/static_libs/lz4/README b/src/static_libs/lz4/README
deleted file mode 100644
index 718f773..0000000
--- a/src/static_libs/lz4/README
+++ /dev/null
@@ -1,7 +0,0 @@
-This iz the lz4 tree copied in:
- http://lz4.googlecode.com/svn/trunk
-by:
- yann.collet...@gmail.com
-Copyright/licensing info in source files here.
-
-this was from revision 84.
diff --git a/src/static_libs/lz4/README.md b/src/static_libs/lz4/README.md
new file mode 100644
index 0000000..636c8be
--- /dev/null
+++ b/src/static_libs/lz4/README.md
@@ -0,0 +1,19 @@
+LZ4 - Library Files
+================================
+
+This directory contains many files, but you don't necessarily need them all.
+
+If you want to integrate LZ4 compression/decompression into your program, you
basically need to include "**lz4.c**" and "**lz4.h**" only.
+
+If you want more compression, at the cost of compression speed (but preserving
decompression speed), you will also have to include "**lz4hc.c**" and
"**lz4hc.h**". Note that lz4hc needs lz4 to work properly.
+
+Next level, if you want to produce files or data streams compatible with lz4
utility, you will have to use and include "**lz4frame.c**" and **lz4frame.h**".
This library encapsulate lz4-compressed blocks into the official interoperable
frame format. In order to work properly, lz4frame needs lz4 and lz4hc, and also
"**xxhash.c**" and "**xxhash.h**", which provide the error detection algorithm.
+
+A more complex "lz4frame_static.h" is also provided, although its usage is not
recommended. It contains definitions which are not guaranteed to remain stable
within future versions. Use only if you don't plan to update your lz4 version.
+
+The other files are not source code. There are :
+
+ - LICENSE : contains the BSD license text
+ - Makefile : script to compile or install lz4 library (static or dynamic)
+ - liblz4.pc.in : for pkg-config (make install)
+
diff --git a/src/static_libs/lz4/lz4.c b/src/static_libs/lz4/lz4.c
index 482a8ed..881d1af 100644
--- a/src/static_libs/lz4/lz4.c
+++ b/src/static_libs/lz4/lz4.c
@@ -1,6 +1,7 @@
/*
LZ4 - Fast LZ compression algorithm
- Copyright (C) 2011-2014, Yann Collet.
+ Copyright (C) 2011-2015, Yann Collet.
+
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
@@ -27,82 +28,77 @@
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- - LZ4 source repository : http://code.google.com/p/lz4/
+ - LZ4 source repository : https://github.com/Cyan4973/lz4
- LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
*/
+
/**************************************
Tuning parameters
**************************************/
/*
* HEAPMODE :
* Select how default compression functions will allocate memory for their
hash table,
- * in memory stack (0:default, fastest), or in memory heap (1:requires memory
allocation (malloc)).
+ * in memory stack (0:default, fastest), or in memory heap (1:requires
malloc()).
*/
#define HEAPMODE 0
+/*
+ * CPU_HAS_EFFICIENT_UNALIGNED_MEMORY_ACCESS :
+ * By default, the source code expects the compiler to correctly optimize
+ * 4-bytes and 8-bytes read on architectures able to handle it efficiently.
+ * This is not always the case. In some circumstances (ARM notably),
+ * the compiler will issue cautious code even when target is able to correctly
handle unaligned memory accesses.
+ *
+ * You can force the compiler to use unaligned memory access by uncommenting
the line below.
+ * One of the below scenarios will happen :
+ * 1 - Your target CPU correctly handle unaligned access, and was not well
optimized by compiler (good case).
+ * You will witness large performance improvements (+50% and up).
+ * Keep the line uncommented and send a word to upstream
(https://groups.google.com/forum/#!forum/lz4c)
+ * The goal is to automatically detect such situations by adding your
target CPU within an exception list.
+ * 2 - Your target CPU correctly handle unaligned access, and was already
already optimized by compiler
+ * No change will be experienced.
+ * 3 - Your target CPU inefficiently handle unaligned access.
+ * You will experience a performance loss. Comment back the line.
+ * 4 - Your target CPU does not handle unaligned access.
+ * Program will crash.
+ * If uncommenting results in better performance (case 1)
+ * please report your configuration to upstream
(https://groups.google.com/forum/#!forum/lz4c)
+ * This way, an automatic detection macro can be added to match your case
within later versions of the library.
+ */
+/* #define CPU_HAS_EFFICIENT_UNALIGNED_MEMORY_ACCESS 1 */
+
/**************************************
CPU Feature Detection
**************************************/
-/* 32 or 64 bits ? */
-#if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \
- || defined(__powerpc64__) || defined(__powerpc64le__) \
- || defined(__ppc64__) || defined(__ppc64le__) \
- || defined(__PPC64__) || defined(__PPC64LE__) \
- || defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) /*
Detects 64 bits mode */
-# define LZ4_ARCH64 1
-#else
-# define LZ4_ARCH64 0
-#endif
-
/*
- * Little Endian or Big Endian ?
- * Overwrite the #define below if you know your architecture endianess
+ * Automated efficient unaligned memory access detection
+ * Based on known hardware architectures
+ * This list will be updated thanks to feedbacks
*/
-#include <stdlib.h> /* Apparently required to detect endianess */
-#if defined (__GLIBC__)
-# include <endian.h>
-# if (__BYTE_ORDER == __BIG_ENDIAN)
-# define LZ4_BIG_ENDIAN 1
-# endif
-#elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) ||
defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) ||
defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN))
-# define LZ4_BIG_ENDIAN 1
-#elif defined(__sparc) || defined(__sparc__) \
- || defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \
- || defined(__hpux) || defined(__hppa) \
- || defined(_MIPSEB) || defined(__s390__)
-# define LZ4_BIG_ENDIAN 1
+#if defined(CPU_HAS_EFFICIENT_UNALIGNED_MEMORY_ACCESS) \
+ || defined(__ARM_FEATURE_UNALIGNED) \
+ || defined(__i386__) || defined(__x86_64__) \
+ || defined(_M_IX86) || defined(_M_X64) \
+ || defined(__ARM_ARCH_7__) || defined(__ARM_ARCH_8__) \
+ || (defined(_M_ARM) && (_M_ARM >= 7))
+# define LZ4_UNALIGNED_ACCESS 1
#else
-/* Little Endian assumed. PDP Endian and other very rare endian format are
unsupported. */
+# define LZ4_UNALIGNED_ACCESS 0
#endif
/*
- * Unaligned memory access is automatically enabled for "common" CPU, such as
x86.
- * For others CPU, such as ARM, the compiler may be more cautious, inserting
unnecessary extra code to ensure aligned access property
- * If you know your target CPU supports unaligned memory access, you want to
force this option manually to improve performance
+ * LZ4_FORCE_SW_BITCOUNT
+ * Define this parameter if your target system or compiler does not support
hardware bit count
*/
-#if defined(__ARM_FEATURE_UNALIGNED)
-# define LZ4_FORCE_UNALIGNED_ACCESS 1
-#endif
-
-/* Define this parameter if your target system or compiler does not support
hardware bit count */
#if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for Windows CE
does not support Hardware bit count */
# define LZ4_FORCE_SW_BITCOUNT
#endif
-/*
- * BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE :
- * This option may provide a small boost to performance for some big endian
cpu, although probably modest.
- * You may set this option to 1 if data will remain within closed environment.
- * This option is useless on Little_Endian CPU (such as x86)
- */
-
-/* #define BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE 1 */
-
/**************************************
- Compiler Options
+* Compiler Options
**************************************/
#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
/* "restrict" is a known keyword */
@@ -112,28 +108,20 @@
#ifdef _MSC_VER /* Visual Studio */
# define FORCE_INLINE static __forceinline
-# include <intrin.h> /* For Visual 2005 */
-# if LZ4_ARCH64 /* 64-bits */
-# pragma intrinsic(_BitScanForward64) /* For Visual 2005 */
-# pragma intrinsic(_BitScanReverse64) /* For Visual 2005 */
-# else /* 32-bits */
-# pragma intrinsic(_BitScanForward) /* For Visual 2005 */
-# pragma intrinsic(_BitScanReverse) /* For Visual 2005 */
-# endif
+# include <intrin.h>
# pragma warning(disable : 4127) /* disable: C4127: conditional
expression is constant */
+# pragma warning(disable : 4293) /* disable: C4293: too large shift
(32-bits) */
#else
-# ifdef __GNUC__
-# define FORCE_INLINE static inline __attribute__((always_inline))
+# if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
+# ifdef __GNUC__
+# define FORCE_INLINE static inline __attribute__((always_inline))
+# else
+# define FORCE_INLINE static inline
+# endif
# else
-# define FORCE_INLINE static inline
-# endif
-#endif
-
-#ifdef _MSC_VER /* Visual Studio */
-# define lz4_bswap16(x) _byteswap_ushort(x)
-#else
-# define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x) &
0xffu) << 8)))
-#endif
+# define FORCE_INLINE static
+# endif /* __STDC_VERSION__ */
+#endif /* _MSC_VER */
#define GCC_VERSION (__GNUC__ * 100 + __GNUC_MINOR__)
@@ -181,46 +169,132 @@
typedef unsigned long long U64;
#endif
-#if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS)
-# define _PACKED __attribute__ ((packed))
-#else
-# define _PACKED
-#endif
-#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
-# if defined(__IBMC__) || defined(__SUNPRO_C) || defined(__SUNPRO_CC)
-# pragma pack(1)
-# else
-# pragma pack(push, 1)
-# endif
-#endif
+/**************************************
+ Reading and writing into memory
+**************************************/
+#define STEPSIZE sizeof(size_t)
-typedef struct { U16 v; } _PACKED U16_S;
-typedef struct { U32 v; } _PACKED U32_S;
-typedef struct { U64 v; } _PACKED U64_S;
-typedef struct {size_t v;} _PACKED size_t_S;
+static unsigned LZ4_64bits(void) { return sizeof(void*)==8; }
-#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
-# if defined(__SUNPRO_C) || defined(__SUNPRO_CC)
-# pragma pack(0)
-# else
-# pragma pack(pop)
-# endif
+static unsigned LZ4_isLittleEndian(void)
+{
+ const union { U32 i; BYTE c[4]; } one = { 1 }; /* don't use static :
performance detrimental */
+ return one.c[0];
+}
+
+
+static U16 LZ4_readLE16(const void* memPtr)
+{
+ if ((LZ4_UNALIGNED_ACCESS) && (LZ4_isLittleEndian()))
+ return *(U16*)memPtr;
+ else
+ {
+ const BYTE* p = (const BYTE*)memPtr;
+ return (U16)((U16)p[0] + (p[1]<<8));
+ }
+}
+
+static void LZ4_writeLE16(void* memPtr, U16 value)
+{
+ if ((LZ4_UNALIGNED_ACCESS) && (LZ4_isLittleEndian()))
+ {
+ *(U16*)memPtr = value;
+ return;
+ }
+ else
+ {
+ BYTE* p = (BYTE*)memPtr;
+ p[0] = (BYTE) value;
+ p[1] = (BYTE)(value>>8);
+ }
+}
+
+
+static U16 LZ4_read16(const void* memPtr)
+{
+ if (LZ4_UNALIGNED_ACCESS)
+ return *(U16*)memPtr;
+ else
+ {
+ U16 val16;
+ memcpy(&val16, memPtr, 2);
+ return val16;
+ }
+}
+
+static U32 LZ4_read32(const void* memPtr)
+{
+ if (LZ4_UNALIGNED_ACCESS)
+ return *(U32*)memPtr;
+ else
+ {
+ U32 val32;
+ memcpy(&val32, memPtr, 4);
+ return val32;
+ }
+}
+
+static U64 LZ4_read64(const void* memPtr)
+{
+ if (LZ4_UNALIGNED_ACCESS)
+ return *(U64*)memPtr;
+ else
+ {
+ U64 val64;
+ memcpy(&val64, memPtr, 8);
+ return val64;
+ }
+}
+
+static size_t LZ4_read_ARCH(const void* p)
+{
+ if (LZ4_64bits())
+ return (size_t)LZ4_read64(p);
+ else
+ return (size_t)LZ4_read32(p);
+}
+
+
+static void LZ4_copy4(void* dstPtr, const void* srcPtr)
+{
+ if (LZ4_UNALIGNED_ACCESS)
+ {
+ *(U32*)dstPtr = *(U32*)srcPtr;
+ return;
+ }
+ memcpy(dstPtr, srcPtr, 4);
+}
+
+static void LZ4_copy8(void* dstPtr, const void* srcPtr)
+{
+#if GCC_VERSION!=409 /* disabled on GCC 4.9, as it generates invalid opcode
(crash) */
+ if (LZ4_UNALIGNED_ACCESS)
+ {
+ if (LZ4_64bits())
+ *(U64*)dstPtr = *(U64*)srcPtr;
+ else
+ ((U32*)dstPtr)[0] = ((U32*)srcPtr)[0],
+ ((U32*)dstPtr)[1] = ((U32*)srcPtr)[1];
+ return;
+ }
#endif
+ memcpy(dstPtr, srcPtr, 8);
+}
-#define A16(x) (((U16_S *)(x))->v)
-#define A32(x) (((U32_S *)(x))->v)
-#define A64(x) (((U64_S *)(x))->v)
-#define AARCH(x) (((size_t_S *)(x))->v)
+/* customized version of memcpy, which may overwrite up to 7 bytes beyond
dstEnd */
+static void LZ4_wildCopy(void* dstPtr, const void* srcPtr, void* dstEnd)
+{
+ BYTE* d = (BYTE*)dstPtr;
+ const BYTE* s = (const BYTE*)srcPtr;
+ BYTE* e = (BYTE*)dstEnd;
+ do { LZ4_copy8(d,s); d+=8; s+=8; } while (d<e);
+}
/**************************************
- Constants
+ Common Constants
**************************************/
-#define LZ4_HASHLOG (LZ4_MEMORY_USAGE-2)
-#define HASHTABLESIZE (1 << LZ4_MEMORY_USAGE)
-#define HASH_SIZE_U32 (1 << LZ4_HASHLOG)
-
#define MINMATCH 4
#define COPYLENGTH 8
@@ -228,13 +302,10 @@ typedef struct {size_t v;} _PACKED size_t_S;
#define MFLIMIT (COPYLENGTH+MINMATCH)
static const int LZ4_minLength = (MFLIMIT+1);
-#define KB *(1U<<10)
-#define MB *(1U<<20)
+#define KB *(1 <<10)
+#define MB *(1 <<20)
#define GB *(1U<<30)
-#define LZ4_64KLIMIT ((64 KB) + (MFLIMIT-1))
-#define SKIPSTRENGTH 6 /* Increasing this value will make the compression
run slower on incompressible data */
-
#define MAXD_LOG 16
#define MAX_DISTANCE ((1 << MAXD_LOG) - 1)
@@ -245,129 +316,147 @@ static const int LZ4_minLength = (MFLIMIT+1);
/**************************************
- Structures and local types
+* Common Utils
**************************************/
-typedef struct {
- U32 hashTable[HASH_SIZE_U32];
- U32 currentOffset;
- U32 initCheck;
- const BYTE* dictionary;
- const BYTE* bufferStart;
- U32 dictSize;
-} LZ4_stream_t_internal;
+#define LZ4_STATIC_ASSERT(c) { enum { LZ4_static_assert = 1/(int)(!!(c)) };
} /* use only *after* variable declarations */
-typedef enum { notLimited = 0, limitedOutput = 1 } limitedOutput_directive;
-typedef enum { byPtr, byU32, byU16 } tableType_t;
-typedef enum { noDict = 0, withPrefix64k, usingExtDict } dict_directive;
-typedef enum { noDictIssue = 0, dictSmall } dictIssue_directive;
+/**************************************
+* Common functions
+**************************************/
+static unsigned LZ4_NbCommonBytes (register size_t val)
+{
+ if (LZ4_isLittleEndian())
+ {
+ if (LZ4_64bits())
+ {
+# if defined(_MSC_VER) && defined(_WIN64) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
+ unsigned long r = 0;
+ _BitScanForward64( &r, (U64)val );
+ return (int)(r>>3);
+# elif defined(__GNUC__) && (GCC_VERSION >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
+ return (__builtin_ctzll((U64)val) >> 3);
+# else
+ static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2,
0, 3, 1, 3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0,
1, 2, 3, 3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5,
7, 6, 7, 7 };
+ return DeBruijnBytePos[((U64)((val & -(long long)val) *
0x0218A392CDABBD3FULL)) >> 58];
+# endif
+ }
+ else /* 32 bits */
+ {
+# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
+ unsigned long r;
+ _BitScanForward( &r, (U32)val );
+ return (int)(r>>3);
+# elif defined(__GNUC__) && (GCC_VERSION >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
+ return (__builtin_ctz((U32)val) >> 3);
+# else
+ static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0,
3, 2, 2, 1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 };
+ return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >>
27];
+# endif
+ }
+ }
+ else /* Big Endian CPU */
+ {
+ if (LZ4_64bits())
+ {
+# if defined(_MSC_VER) && defined(_WIN64) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
+ unsigned long r = 0;
+ _BitScanReverse64( &r, val );
+ return (unsigned)(r>>3);
+# elif defined(__GNUC__) && (GCC_VERSION >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
+ return (__builtin_clzll(val) >> 3);
+# else
+ unsigned r;
+ if (!(val>>32)) { r=4; } else { r=0; val>>=32; }
+ if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; }
+ r += (!val);
+ return r;
+# endif
+ }
+ else /* 32 bits */
+ {
+# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
+ unsigned long r = 0;
+ _BitScanReverse( &r, (unsigned long)val );
+ return (unsigned)(r>>3);
+# elif defined(__GNUC__) && (GCC_VERSION >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
+ return (__builtin_clz(val) >> 3);
+# else
+ unsigned r;
+ if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; }
+ r += (!val);
+ return r;
+# endif
+ }
+ }
+}
-typedef enum { endOnOutputSize = 0, endOnInputSize = 1 }
endCondition_directive;
-typedef enum { full = 0, partial = 1 } earlyEnd_directive;
+static unsigned LZ4_count(const BYTE* pIn, const BYTE* pMatch, const BYTE*
pInLimit)
+{
+ const BYTE* const pStart = pIn;
+
+ while (likely(pIn<pInLimit-(STEPSIZE-1)))
+ {
+ size_t diff = LZ4_read_ARCH(pMatch) ^ LZ4_read_ARCH(pIn);
+ if (!diff) { pIn+=STEPSIZE; pMatch+=STEPSIZE; continue; }
+ pIn += LZ4_NbCommonBytes(diff);
+ return (unsigned)(pIn - pStart);
+ }
+
+ if (LZ4_64bits()) if ((pIn<(pInLimit-3)) && (LZ4_read32(pMatch) ==
LZ4_read32(pIn))) { pIn+=4; pMatch+=4; }
+ if ((pIn<(pInLimit-1)) && (LZ4_read16(pMatch) == LZ4_read16(pIn))) {
pIn+=2; pMatch+=2; }
+ if ((pIn<pInLimit) && (*pMatch == *pIn)) pIn++;
+ return (unsigned)(pIn - pStart);
+}
+#ifndef LZ4_COMMONDEFS_ONLY
/**************************************
- Architecture-specific macros
+* Local Constants
**************************************/
-#define STEPSIZE sizeof(size_t)
-#define LZ4_COPYSTEP(d,s) { AARCH(d) = AARCH(s); d+=STEPSIZE;
s+=STEPSIZE; }
-#define LZ4_COPY8(d,s) { LZ4_COPYSTEP(d,s); if (STEPSIZE<8)
LZ4_COPYSTEP(d,s); }
-
-#if (defined(LZ4_BIG_ENDIAN) && !defined(BIG_ENDIAN_NATIVE_BUT_INCOMPATIBLE))
-# define LZ4_READ_LITTLEENDIAN_16(d,s,p) { U16 v = A16(p); v =
lz4_bswap16(v); d = (s) - v; }
-# define LZ4_WRITE_LITTLEENDIAN_16(p,i) { U16 v = (U16)(i); v =
lz4_bswap16(v); A16(p) = v; p+=2; }
-#else /* Little Endian */
-# define LZ4_READ_LITTLEENDIAN_16(d,s,p) { d = (s) - A16(p); }
-# define LZ4_WRITE_LITTLEENDIAN_16(p,v) { A16(p) = v; p+=2; }
-#endif
+#define LZ4_HASHLOG (LZ4_MEMORY_USAGE-2)
+#define HASHTABLESIZE (1 << LZ4_MEMORY_USAGE)
+#define HASH_SIZE_U32 (1 << LZ4_HASHLOG) /* required as macro for static
allocation */
+
+static const int LZ4_64Klimit = ((64 KB) + (MFLIMIT-1));
+static const U32 LZ4_skipTrigger = 6; /* Increase this value ==> compression
run slower on incompressible data */
/**************************************
- Macros
+* Local Utils
**************************************/
-#define LZ4_STATIC_ASSERT(c) { enum { LZ4_static_assert = 1/(!!(c)) }; }
/* use only *after* variable declarations */
-#if LZ4_ARCH64 || !defined(__GNUC__)
-# define LZ4_WILDCOPY(d,s,e) { do { LZ4_COPY8(d,s) } while (d<e); }
/* at the end, d>=e; */
-#else
-# define LZ4_WILDCOPY(d,s,e) { if (likely(e-d <= 8)) LZ4_COPY8(d,s) else do
{ LZ4_COPY8(d,s) } while (d<e); }
-#endif
+int LZ4_versionNumber (void) { return LZ4_VERSION_NUMBER; }
+int LZ4_compressBound(int isize) { return LZ4_COMPRESSBOUND(isize); }
-/****************************
- Private local functions
-****************************/
-#if LZ4_ARCH64
+/**************************************
+* Local Structures and types
+**************************************/
+typedef struct {
+ U32 hashTable[HASH_SIZE_U32];
+ U32 currentOffset;
+ U32 initCheck;
+ const BYTE* dictionary;
+ const BYTE* bufferStart;
+ U32 dictSize;
+} LZ4_stream_t_internal;
-int LZ4_NbCommonBytes (register U64 val)
-{
-# if defined(LZ4_BIG_ENDIAN)
-# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
- unsigned long r = 0;
- _BitScanReverse64( &r, val );
- return (int)(r>>3);
-# elif defined(__GNUC__) && (GCC_VERSION >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
- return (__builtin_clzll(val) >> 3);
-# else
- int r;
- if (!(val>>32)) { r=4; } else { r=0; val>>=32; }
- if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; }
- r += (!val);
- return r;
-# endif
-# else
-# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
- unsigned long r = 0;
- _BitScanForward64( &r, val );
- return (int)(r>>3);
-# elif defined(__GNUC__) && (GCC_VERSION >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
- return (__builtin_ctzll(val) >> 3);
-# else
- static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1,
3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3,
3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7
};
- return DeBruijnBytePos[((U64)((val & -(long long)val) *
0x0218A392CDABBD3FULL)) >> 58];
-# endif
-# endif
-}
+typedef enum { notLimited = 0, limitedOutput = 1 } limitedOutput_directive;
+typedef enum { byPtr, byU32, byU16 } tableType_t;
-#else
+typedef enum { noDict = 0, withPrefix64k, usingExtDict } dict_directive;
+typedef enum { noDictIssue = 0, dictSmall } dictIssue_directive;
-int LZ4_NbCommonBytes (register U32 val)
-{
-# if defined(LZ4_BIG_ENDIAN)
-# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
- unsigned long r = 0;
- _BitScanReverse( &r, val );
- return (int)(r>>3);
-# elif defined(__GNUC__) && (GCC_VERSION >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
- return (__builtin_clz(val) >> 3);
-# else
- int r;
- if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; }
- r += (!val);
- return r;
-# endif
-# else
-# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
- unsigned long r;
- _BitScanForward( &r, val );
- return (int)(r>>3);
-# elif defined(__GNUC__) && (GCC_VERSION >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
- return (__builtin_ctz(val) >> 3);
-# else
- static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2,
1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 };
- return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27];
-# endif
-# endif
-}
+typedef enum { endOnOutputSize = 0, endOnInputSize = 1 }
endCondition_directive;
+typedef enum { full = 0, partial = 1 } earlyEnd_directive;
-#endif
/********************************
- Compression functions
+* Compression functions
********************************/
-int LZ4_compressBound(int isize) { return LZ4_COMPRESSBOUND(isize); }
-static int LZ4_hashSequence(U32 sequence, tableType_t tableType)
+static U32 LZ4_hashSequence(U32 sequence, tableType_t const tableType)
{
if (tableType == byU16)
return (((sequence) * 2654435761U) >> ((MINMATCH*8)-(LZ4_HASHLOG+1)));
@@ -375,15 +464,15 @@ static int LZ4_hashSequence(U32 sequence, tableType_t
tableType)
return (((sequence) * 2654435761U) >> ((MINMATCH*8)-LZ4_HASHLOG));
}
-static int LZ4_hashPosition(const BYTE* p, tableType_t tableType) { return
LZ4_hashSequence(A32(p), tableType); }
+static U32 LZ4_hashPosition(const BYTE* p, tableType_t tableType) { return
LZ4_hashSequence(LZ4_read32(p), tableType); }
-static void LZ4_putPositionOnHash(const BYTE* p, U32 h, void* tableBase,
tableType_t tableType, const BYTE* srcBase)
+static void LZ4_putPositionOnHash(const BYTE* p, U32 h, void* tableBase,
tableType_t const tableType, const BYTE* srcBase)
{
switch (tableType)
{
- case byPtr: { const BYTE** hashTable = (const BYTE**) tableBase;
hashTable[h] = p; break; }
- case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] =
(U32)(p-srcBase); break; }
- case byU16: { U16* hashTable = (U16*) tableBase; hashTable[h] =
(U16)(p-srcBase); break; }
+ case byPtr: { const BYTE** hashTable = (const BYTE**)tableBase;
hashTable[h] = p; return; }
+ case byU32: { U32* hashTable = (U32*) tableBase; hashTable[h] =
(U32)(p-srcBase); return; }
+ case byU16: { U16* hashTable = (U16*) tableBase; hashTable[h] =
(U16)(p-srcBase); return; }
}
}
@@ -406,34 +495,14 @@ static const BYTE* LZ4_getPosition(const BYTE* p, void*
tableBase, tableType_t t
return LZ4_getPositionOnHash(h, tableBase, tableType, srcBase);
}
-static unsigned LZ4_count(const BYTE* pIn, const BYTE* pRef, const BYTE*
pInLimit)
-{
- const BYTE* const pStart = pIn;
-
- while (likely(pIn<pInLimit-(STEPSIZE-1)))
- {
- size_t diff = AARCH(pRef) ^ AARCH(pIn);
- if (!diff) { pIn+=STEPSIZE; pRef+=STEPSIZE; continue; }
- pIn += LZ4_NbCommonBytes(diff);
- return (unsigned)(pIn - pStart);
- }
- if (sizeof(void*)==8) if ((pIn<(pInLimit-3)) && (A32(pRef) == A32(pIn))) {
pIn+=4; pRef+=4; }
- if ((pIn<(pInLimit-1)) && (A16(pRef) == A16(pIn))) { pIn+=2; pRef+=2; }
- if ((pIn<pInLimit) && (*pRef == *pIn)) pIn++;
-
- return (unsigned)(pIn - pStart);
-}
-
-
static int LZ4_compress_generic(
void* ctx,
const char* source,
char* dest,
int inputSize,
int maxOutputSize,
-
limitedOutput_directive outputLimited,
- tableType_t tableType,
+ tableType_t const tableType,
dict_directive dict,
dictIssue_directive dictIssue)
{
@@ -454,7 +523,6 @@ static int LZ4_compress_generic(
BYTE* op = (BYTE*) dest;
BYTE* const olimit = op + maxOutputSize;
- const int skipStrength = SKIPSTRENGTH;
U32 forwardH;
size_t refDelta=0;
@@ -476,8 +544,8 @@ static int LZ4_compress_generic(
lowLimit = (const BYTE*)source;
break;
}
- if ((tableType == byU16) && (inputSize>=(int)LZ4_64KLIMIT)) return 0; /*
Size too large (not within 64K limit) */
- if (inputSize<LZ4_minLength) goto _last_literals; /*
Input too small, no compression (all literals) */
+ if ((tableType == byU16) && (inputSize>=LZ4_64Klimit)) return 0; /* Size
too large (not within 64K limit) */
+ if (inputSize<LZ4_minLength) goto _last_literals; /*
Input too small, no compression (all literals) */
/* First Byte */
LZ4_putPosition(ip, ctx, tableType, base);
@@ -486,27 +554,26 @@ static int LZ4_compress_generic(
/* Main Loop */
for ( ; ; )
{
- const BYTE* ref;
+ const BYTE* match;
BYTE* token;
{
const BYTE* forwardIp = ip;
unsigned step=1;
- unsigned searchMatchNb = (1U << skipStrength);
+ unsigned searchMatchNb = (1U << LZ4_skipTrigger);
/* Find a match */
do {
U32 h = forwardH;
ip = forwardIp;
forwardIp += step;
- step = searchMatchNb++ >> skipStrength;
- //if (step>8) step=8; // required for valid forwardIp ;
slows down uncompressible data a bit
+ step = searchMatchNb++ >> LZ4_skipTrigger;
if (unlikely(forwardIp > mflimit)) goto _last_literals;
- ref = LZ4_getPositionOnHash(h, ctx, tableType, base);
+ match = LZ4_getPositionOnHash(h, ctx, tableType, base);
if (dict==usingExtDict)
{
- if (ref<(const BYTE*)source)
+ if (match<(const BYTE*)source)
{
refDelta = dictDelta;
lowLimit = dictionary;
@@ -520,13 +587,13 @@ static int LZ4_compress_generic(
forwardH = LZ4_hashPosition(forwardIp, tableType);
LZ4_putPositionOnHash(ip, h, ctx, tableType, base);
- } while ( ((dictIssue==dictSmall) ? (ref < lowRefLimit) : 0)
- || ((tableType==byU16) ? 0 : (ref + MAX_DISTANCE < ip))
- || (A32(ref+refDelta) != A32(ip)) );
+ } while ( ((dictIssue==dictSmall) ? (match < lowRefLimit) : 0)
+ || ((tableType==byU16) ? 0 : (match + MAX_DISTANCE < ip))
+ || (LZ4_read32(match+refDelta) != LZ4_read32(ip)) );
}
/* Catch up */
- while ((ip>anchor) && (ref+refDelta > lowLimit) &&
(unlikely(ip[-1]==ref[refDelta-1]))) { ip--; ref--; }
+ while ((ip>anchor) && (match+refDelta > lowLimit) &&
(unlikely(ip[-1]==match[refDelta-1]))) { ip--; match--; }
{
/* Encode Literal length */
@@ -544,12 +611,13 @@ static int LZ4_compress_generic(
else *token = (BYTE)(litLength<<ML_BITS);
/* Copy Literals */
- { BYTE* end = op+litLength; LZ4_WILDCOPY(op,anchor,end); op=end; }
+ LZ4_wildCopy(op, anchor, op+litLength);
+ op+=litLength;
}
_next_match:
/* Encode Offset */
- LZ4_WRITE_LITTLEENDIAN_16(op, (U16)(ip-ref));
+ LZ4_writeLE16(op, (U16)(ip-match)); op+=2;
/* Encode MatchLength */
{
@@ -558,10 +626,10 @@ _next_match:
if ((dict==usingExtDict) && (lowLimit==dictionary))
{
const BYTE* limit;
- ref += refDelta;
- limit = ip + (dictEnd-ref);
+ match += refDelta;
+ limit = ip + (dictEnd-match);
if (limit > matchlimit) limit = matchlimit;
- matchLength = LZ4_count(ip+MINMATCH, ref+MINMATCH, limit);
+ matchLength = LZ4_count(ip+MINMATCH, match+MINMATCH, limit);
ip += MINMATCH + matchLength;
if (ip==limit)
{
@@ -572,14 +640,14 @@ _next_match:
}
else
{
- matchLength = LZ4_count(ip+MINMATCH, ref+MINMATCH, matchlimit);
+ matchLength = LZ4_count(ip+MINMATCH, match+MINMATCH,
matchlimit);
ip += MINMATCH + matchLength;
}
+ if ((outputLimited) && (unlikely(op + (1 + LASTLITERALS) +
(matchLength>>8) > olimit)))
+ return 0; /* Check output limit */
if (matchLength>=ML_MASK)
{
- if ((outputLimited) && (unlikely(op + (1 + LASTLITERALS) +
(matchLength>>8) > olimit)))
- return 0; /* Check output limit */
*token += ML_MASK;
matchLength -= ML_MASK;
for (; matchLength >= 510 ; matchLength-=510) { *op++ = 255;
*op++ = 255; }
@@ -598,10 +666,10 @@ _next_match:
LZ4_putPosition(ip-2, ctx, tableType, base);
/* Test next position */
- ref = LZ4_getPosition(ip, ctx, tableType, base);
+ match = LZ4_getPosition(ip, ctx, tableType, base);
if (dict==usingExtDict)
{
- if (ref<(const BYTE*)source)
+ if (match<(const BYTE*)source)
{
refDelta = dictDelta;
lowLimit = dictionary;
@@ -613,9 +681,9 @@ _next_match:
}
}
LZ4_putPosition(ip, ctx, tableType, base);
- if ( ((dictIssue==dictSmall) ? (ref>=lowRefLimit) : 1)
- && (ref+MAX_DISTANCE>=ip)
- && (A32(ref+refDelta)==A32(ip)) )
+ if ( ((dictIssue==dictSmall) ? (match>=lowRefLimit) : 1)
+ && (match+MAX_DISTANCE>=ip)
+ && (LZ4_read32(match+refDelta)==LZ4_read32(ip)) )
{ token=op++; *token=0; goto _next_match; }
/* Prepare next loop */
@@ -642,16 +710,16 @@ _last_literals:
int LZ4_compress(const char* source, char* dest, int inputSize)
{
#if (HEAPMODE)
- void* ctx = ALLOCATOR(LZ4_STREAMSIZE_U32, 4); /* Aligned on 4-bytes
boundaries */
+ void* ctx = ALLOCATOR(LZ4_STREAMSIZE_U64, 8); /* Aligned on 8-bytes
boundaries */
#else
- U32 ctx[LZ4_STREAMSIZE_U32] = {0}; /* Ensure data is aligned on
4-bytes boundaries */
+ U64 ctx[LZ4_STREAMSIZE_U64] = {0}; /* Ensure data is aligned on
8-bytes boundaries */
#endif
int result;
- if (inputSize < (int)LZ4_64KLIMIT)
+ if (inputSize < LZ4_64Klimit)
result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, 0,
notLimited, byU16, noDict, noDictIssue);
else
- result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, 0,
notLimited, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue);
+ result = LZ4_compress_generic((void*)ctx, source, dest, inputSize, 0,
notLimited, LZ4_64bits() ? byU32 : byPtr, noDict, noDictIssue);
#if (HEAPMODE)
FREEMEM(ctx);
@@ -662,16 +730,16 @@ int LZ4_compress(const char* source, char* dest, int
inputSize)
int LZ4_compress_limitedOutput(const char* source, char* dest, int inputSize,
int maxOutputSize)
{
#if (HEAPMODE)
- void* ctx = ALLOCATOR(LZ4_STREAMSIZE_U32, 4); /* Aligned on 4-bytes
boundaries */
+ void* ctx = ALLOCATOR(LZ4_STREAMSIZE_U64, 8); /* Aligned on 8-bytes
boundaries */
#else
- U32 ctx[LZ4_STREAMSIZE_U32] = {0}; /* Ensure data is aligned on
4-bytes boundaries */
+ U64 ctx[LZ4_STREAMSIZE_U64] = {0}; /* Ensure data is aligned on
8-bytes boundaries */
#endif
int result;
- if (inputSize < (int)LZ4_64KLIMIT)
+ if (inputSize < LZ4_64Klimit)
result = LZ4_compress_generic((void*)ctx, source, dest, inputSize,
maxOutputSize, limitedOutput, byU16, noDict, noDictIssue);
else
- result = LZ4_compress_generic((void*)ctx, source, dest, inputSize,
maxOutputSize, limitedOutput, (sizeof(void*)==8) ? byU32 : byPtr, noDict,
noDictIssue);
+ result = LZ4_compress_generic((void*)ctx, source, dest, inputSize,
maxOutputSize, limitedOutput, LZ4_64bits() ? byU32 : byPtr, noDict,
noDictIssue);
#if (HEAPMODE)
FREEMEM(ctx);
@@ -681,38 +749,48 @@ int LZ4_compress_limitedOutput(const char* source, char*
dest, int inputSize, in
/*****************************************
- Experimental : Streaming functions
+* Experimental : Streaming functions
*****************************************/
-void* LZ4_createStream()
+/*
+ * LZ4_initStream
+ * Use this function once, to init a newly allocated LZ4_stream_t structure
+ * Return : 1 if OK, 0 if error
+ */
+void LZ4_resetStream (LZ4_stream_t* LZ4_stream)
+{
+ MEM_INIT(LZ4_stream, 0, sizeof(LZ4_stream_t));
+}
+
+LZ4_stream_t* LZ4_createStream(void)
{
- void* lz4s = ALLOCATOR(4, LZ4_STREAMSIZE_U32);
- MEM_INIT(lz4s, 0, LZ4_STREAMSIZE);
+ LZ4_stream_t* lz4s = (LZ4_stream_t*)ALLOCATOR(8, LZ4_STREAMSIZE_U64);
+ LZ4_STATIC_ASSERT(LZ4_STREAMSIZE >= sizeof(LZ4_stream_t_internal)); /*
A compilation error here means LZ4_STREAMSIZE is not large enough */
+ LZ4_resetStream(lz4s);
return lz4s;
}
-int LZ4_free (void* LZ4_stream)
+int LZ4_freeStream (LZ4_stream_t* LZ4_stream)
{
FREEMEM(LZ4_stream);
return (0);
}
-int LZ4_loadDict (void* LZ4_dict, const char* dictionary, int dictSize)
+int LZ4_loadDict (LZ4_stream_t* LZ4_dict, const char* dictionary, int dictSize)
{
LZ4_stream_t_internal* dict = (LZ4_stream_t_internal*) LZ4_dict;
const BYTE* p = (const BYTE*)dictionary;
const BYTE* const dictEnd = p + dictSize;
const BYTE* base;
- LZ4_STATIC_ASSERT(LZ4_STREAMSIZE >= sizeof(LZ4_stream_t_internal));
/* A compilation error here means LZ4_STREAMSIZE is not large enough */
- if (dict->initCheck) MEM_INIT(dict, 0, sizeof(LZ4_stream_t_internal));
/* Uninitialized structure detected */
+ if (dict->initCheck) LZ4_resetStream(LZ4_dict); /*
Uninitialized structure detected */
if (dictSize < MINMATCH)
{
dict->dictionary = NULL;
dict->dictSize = 0;
- return 1;
+ return 0;
}
if (p <= dictEnd - 64 KB) p = dictEnd - 64 KB;
@@ -727,11 +805,11 @@ int LZ4_loadDict (void* LZ4_dict, const char* dictionary,
int dictSize)
p+=3;
}
- return 1;
+ return dict->dictSize;
}
-void LZ4_renormDictT(LZ4_stream_t_internal* LZ4_dict, const BYTE* src)
+static void LZ4_renormDictT(LZ4_stream_t_internal* LZ4_dict, const BYTE* src)
{
if ((LZ4_dict->currentOffset > 0x80000000) ||
((size_t)LZ4_dict->currentOffset > (size_t)src)) /* address space
overflow */
@@ -802,19 +880,18 @@ FORCE_INLINE int LZ4_compress_continue_generic (void*
LZ4_stream, const char* so
}
}
-
-int LZ4_compress_continue (void* LZ4_stream, const char* source, char* dest,
int inputSize)
+int LZ4_compress_continue (LZ4_stream_t* LZ4_stream, const char* source, char*
dest, int inputSize)
{
return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize,
0, notLimited);
}
-int LZ4_compress_limitedOutput_continue (void* LZ4_stream, const char* source,
char* dest, int inputSize, int maxOutputSize)
+int LZ4_compress_limitedOutput_continue (LZ4_stream_t* LZ4_stream, const char*
source, char* dest, int inputSize, int maxOutputSize)
{
return LZ4_compress_continue_generic(LZ4_stream, source, dest, inputSize,
maxOutputSize, limitedOutput);
}
-// Hidden debug function, to force separate dictionary mode
+/* Hidden debug function, to force separate dictionary mode */
int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict, const char* source,
char* dest, int inputSize)
{
LZ4_stream_t_internal* streamPtr = (LZ4_stream_t_internal*)LZ4_dict;
@@ -835,7 +912,7 @@ int LZ4_compress_forceExtDict (LZ4_stream_t* LZ4_dict,
const char* source, char*
}
-int LZ4_saveDict (void* LZ4_dict, char* safeBuffer, int dictSize)
+int LZ4_saveDict (LZ4_stream_t* LZ4_dict, char* safeBuffer, int dictSize)
{
LZ4_stream_t_internal* dict = (LZ4_stream_t_internal*) LZ4_dict;
const BYTE* previousDictEnd = dict->dictionary + dict->dictSize;
@@ -843,28 +920,28 @@ int LZ4_saveDict (void* LZ4_dict, char* safeBuffer, int
dictSize)
if ((U32)dictSize > 64 KB) dictSize = 64 KB; /* useless to define a
dictionary > 64 KB */
if ((U32)dictSize > dict->dictSize) dictSize = dict->dictSize;
- memcpy(safeBuffer, previousDictEnd - dictSize, dictSize);
+ memmove(safeBuffer, previousDictEnd - dictSize, dictSize);
dict->dictionary = (const BYTE*)safeBuffer;
dict->dictSize = (U32)dictSize;
- return 1;
+ return dictSize;
}
-/****************************
- Decompression functions
-****************************/
+/*******************************
+* Decompression functions
+*******************************/
/*
* This generic decompression function cover all use cases.
- * It shall be instanciated several times, using different sets of directives
+ * It shall be instantiated several times, using different sets of directives
* Note that it is essential this generic function is really inlined,
- * in order to remove useless branches during compilation optimisation.
+ * in order to remove useless branches during compilation optimization.
*/
FORCE_INLINE int LZ4_decompress_generic(
- const char* source,
- char* dest,
+ const char* const source,
+ char* const dest,
int inputSize,
int outputSize, /* If endOnInput==endOnInputSize,
this value is the max size of Output Buffer. */
@@ -872,36 +949,32 @@ FORCE_INLINE int LZ4_decompress_generic(
int partialDecoding, /* full, partial */
int targetOutputSize, /* only used if
partialDecoding==partial */
int dict, /* noDict, withPrefix64k,
usingExtDict */
- const char* dictStart, /* only if dict==usingExtDict */
- int dictSize /* note : = 0 if noDict */
+ const BYTE* const lowPrefix, /* == dest if dict == noDict */
+ const BYTE* const dictStart, /* only if dict==usingExtDict */
+ const size_t dictSize /* note : = 0 if noDict */
)
{
/* Local Variables */
const BYTE* restrict ip = (const BYTE*) source;
- const BYTE* ref;
const BYTE* const iend = ip + inputSize;
BYTE* op = (BYTE*) dest;
BYTE* const oend = op + outputSize;
BYTE* cpy;
BYTE* oexit = op + targetOutputSize;
- const BYTE* const lowLimit = (const BYTE*)dest - dictSize;
+ const BYTE* const lowLimit = lowPrefix - dictSize;
const BYTE* const dictEnd = (const BYTE*)dictStart + dictSize;
-//#define OLD
-#ifdef OLD
- const size_t dec32table[] = {0, 3, 2, 3, 0, 0, 0, 0}; /* static reduces
speed for LZ4_decompress_safe() on GCC64 */
-#else
- const size_t dec32table[] = {4-0, 4-3, 4-2, 4-3, 4-0, 4-0, 4-0, 4-0}; /*
static reduces speed for LZ4_decompress_safe() on GCC64 */
-#endif
- static const size_t dec64table[] = {0, 0, 0, (size_t)-1, 0, 1, 2, 3};
+ const size_t dec32table[] = {4, 1, 2, 1, 4, 4, 4, 4};
+ const size_t dec64table[] = {0, 0, 0, (size_t)-1, 0, 1, 2, 3};
- const int checkOffset = (endOnInput) && (dictSize < (int)(64 KB));
+ const int safeDecode = (endOnInput==endOnInputSize);
+ const int checkOffset = ((safeDecode) && (dictSize < (int)(64 KB)));
/* Special cases */
- if ((partialDecoding) && (oexit> oend-MFLIMIT)) oexit = oend-MFLIMIT;
/* targetOutputSize too high => decode everything */
- if ((endOnInput) && (unlikely(outputSize==0))) return ((inputSize==1) &&
(*ip==0)) ? 0 : -1; /* Empty output buffer */
+ if ((partialDecoding) && (oexit> oend-MFLIMIT)) oexit = oend-MFLIMIT;
/* targetOutputSize too high => decode everything */
+ if ((endOnInput) && (unlikely(outputSize==0))) return ((inputSize==1) &&
(*ip==0)) ? 0 : -1; /* Empty output buffer */
if ((!endOnInput) && (unlikely(outputSize==0))) return (*ip==0?1:-1);
@@ -910,8 +983,9 @@ FORCE_INLINE int LZ4_decompress_generic(
{
unsigned token;
size_t length;
+ const BYTE* match;
- /* get runlength */
+ /* get literal length */
token = *ip++;
if ((length=(token>>ML_BITS)) == RUN_MASK)
{
@@ -922,9 +996,8 @@ FORCE_INLINE int LZ4_decompress_generic(
length += s;
}
while (likely((endOnInput)?ip<iend-RUN_MASK:1) && (s==255));
- //if ((sizeof(void*)==4) && unlikely(length>LZ4_MAX_INPUT_SIZE))
goto _output_error; /* overflow detection */
- if ((sizeof(void*)==4) &&
unlikely((size_t)(op+length)<(size_t)(op))) goto _output_error; /* quickfix
issue 134 */
- if ((endOnInput) && (sizeof(void*)==4) &&
unlikely((size_t)(ip+length)<(size_t)(ip))) goto _output_error; /* quickfix
issue 134 */
+ if ((safeDecode) && unlikely((size_t)(op+length)<(size_t)(op)))
goto _output_error; /* overflow detection */
+ if ((safeDecode) && unlikely((size_t)(ip+length)<(size_t)(ip)))
goto _output_error; /* overflow detection */
}
/* copy literals */
@@ -945,16 +1018,18 @@ FORCE_INLINE int LZ4_decompress_generic(
memcpy(op, ip, length);
ip += length;
op += length;
- break; /* Necessarily EOF,
due to parsing restrictions */
+ break; /* Necessarily EOF, due to parsing restrictions */
}
- LZ4_WILDCOPY(op, ip, cpy); ip -= (op-cpy); op = cpy;
+ LZ4_wildCopy(op, ip, cpy);
+ ip += length; op = cpy;
/* get offset */
- LZ4_READ_LITTLEENDIAN_16(ref,cpy,ip); ip+=2;
- if ((checkOffset) && (unlikely(ref < lowLimit))) goto _output_error;
/* Error : offset outside destination buffer */
+ match = cpy - LZ4_readLE16(ip); ip+=2;
+ if ((checkOffset) && (unlikely(match < lowLimit))) goto _output_error;
/* Error : offset outside destination buffer */
/* get matchlength */
- if ((length=(token&ML_MASK)) == ML_MASK)
+ length = token & ML_MASK;
+ if (length == ML_MASK)
{
unsigned s;
do
@@ -963,36 +1038,38 @@ FORCE_INLINE int LZ4_decompress_generic(
s = *ip++;
length += s;
} while (s==255);
- //if ((sizeof(void*)==4) && unlikely(length>LZ4_MAX_INPUT_SIZE))
goto _output_error; /* overflow detection */
- if ((sizeof(void*)==4) &&
unlikely((size_t)(op+length)<(size_t)op)) goto _output_error; /* quickfix
issue 134 */
+ if ((safeDecode) && unlikely((size_t)(op+length)<(size_t)op)) goto
_output_error; /* overflow detection */
}
+ length += MINMATCH;
/* check external dictionary */
- if ((dict==usingExtDict) && (ref < (BYTE* const)dest))
+ if ((dict==usingExtDict) && (match < lowPrefix))
{
- if (unlikely(op+length+MINMATCH > oend-LASTLITERALS)) goto
_output_error;
+ if (unlikely(op+length > oend-LASTLITERALS)) goto _output_error;
/* doesn't respect parsing restriction */
- if (length+MINMATCH <= (size_t)(dest-(char*)ref))
+ if (length <= (size_t)(lowPrefix-match))
{
- ref = dictEnd - (dest-(char*)ref);
- memcpy(op, ref, length+MINMATCH);
- op += length+MINMATCH;
+ /* match can be copied as a single segment from external
dictionary */
+ match = dictEnd - (lowPrefix-match);
+ memcpy(op, match, length);
+ op += length;
}
else
{
- size_t copySize = (size_t)(dest-(char*)ref);
+ /* match encompass external dictionary and current segment */
+ size_t copySize = (size_t)(lowPrefix-match);
memcpy(op, dictEnd - copySize, copySize);
op += copySize;
- copySize = length+MINMATCH - copySize;
- if (copySize > (size_t)((char*)op-dest)) /* overlap */
+ copySize = length - copySize;
+ if (copySize > (size_t)(op-lowPrefix)) /* overlap within
current segment */
{
- BYTE* const cpy = op + copySize;
- const BYTE* ref = (BYTE*)dest;
- while (op < cpy) *op++ = *ref++;
+ BYTE* const endOfMatch = op + copySize;
+ const BYTE* copyFrom = lowPrefix;
+ while (op < endOfMatch) *op++ = *copyFrom++;
}
else
{
- memcpy(op, dest, copySize);
+ memcpy(op, lowPrefix, copySize);
op += copySize;
}
}
@@ -1000,34 +1077,32 @@ FORCE_INLINE int LZ4_decompress_generic(
}
/* copy repeated sequence */
- if (unlikely((op-ref)<(int)STEPSIZE))
+ cpy = op + length;
+ if (unlikely((op-match)<8))
{
- const size_t dec64 = dec64table[(sizeof(void*)==4) ? 0 : op-ref];
- op[0] = ref[0];
- op[1] = ref[1];
- op[2] = ref[2];
- op[3] = ref[3];
-#ifdef OLD
- op += 4, ref += 4; ref -= dec32table[op-ref];
- A32(op) = A32(ref);
- op += STEPSIZE-4; ref -= dec64;
-#else
- ref += dec32table[op-ref];
- A32(op+4) = A32(ref);
- op += STEPSIZE; ref -= dec64;
-#endif
- } else { LZ4_COPYSTEP(op,ref); }
- cpy = op + length - (STEPSIZE-4);
-
- if (unlikely(cpy>oend-COPYLENGTH-(STEPSIZE-4)))
+ const size_t dec64 = dec64table[op-match];
+ op[0] = match[0];
+ op[1] = match[1];
+ op[2] = match[2];
+ op[3] = match[3];
+ match += dec32table[op-match];
+ LZ4_copy4(op+4, match);
+ op += 8; match -= dec64;
+ } else { LZ4_copy8(op, match); op+=8; match+=8; }
+
+ if (unlikely(cpy>oend-12))
{
- if (cpy > oend-LASTLITERALS) goto _output_error; /* Error :
last 5 bytes must be literals */
- if (op<oend-COPYLENGTH) LZ4_WILDCOPY(op, ref, (oend-COPYLENGTH));
- while(op<cpy) *op++=*ref++;
- op=cpy;
- continue;
+ if (cpy > oend-LASTLITERALS) goto _output_error; /* Error :
last LASTLITERALS bytes must be literals */
+ if (op < oend-8)
+ {
+ LZ4_wildCopy(op, match, oend-8);
+ match += (oend-8) - op;
+ op = oend-8;
+ }
+ while (op<cpy) *op++ = *match++;
}
- LZ4_WILDCOPY(op, ref, cpy);
+ else
+ LZ4_wildCopy(op, match, cpy);
op=cpy; /* correction */
}
@@ -1043,30 +1118,30 @@ _output_error:
}
-int LZ4_decompress_safe(const char* source, char* dest, int compressedSize,
int maxOutputSize)
+int LZ4_decompress_safe(const char* source, char* dest, int compressedSize,
int maxDecompressedSize)
{
- return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
endOnInputSize, full, 0, noDict, NULL, 0);
+ return LZ4_decompress_generic(source, dest, compressedSize,
maxDecompressedSize, endOnInputSize, full, 0, noDict, (BYTE*)dest, NULL, 0);
}
-int LZ4_decompress_safe_partial(const char* source, char* dest, int
compressedSize, int targetOutputSize, int maxOutputSize)
+int LZ4_decompress_safe_partial(const char* source, char* dest, int
compressedSize, int targetOutputSize, int maxDecompressedSize)
{
- return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
endOnInputSize, partial, targetOutputSize, noDict, NULL, 0);
+ return LZ4_decompress_generic(source, dest, compressedSize,
maxDecompressedSize, endOnInputSize, partial, targetOutputSize, noDict,
(BYTE*)dest, NULL, 0);
}
int LZ4_decompress_fast(const char* source, char* dest, int originalSize)
{
- return LZ4_decompress_generic(source, dest, 0, originalSize,
endOnOutputSize, full, 0, withPrefix64k, NULL, 0);
+ return LZ4_decompress_generic(source, dest, 0, originalSize,
endOnOutputSize, full, 0, withPrefix64k, (BYTE*)(dest - 64 KB), NULL, 64 KB);
}
+
/* streaming decompression functions */
-//#define LZ4_STREAMDECODESIZE_U32 4
-//#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U32 * sizeof(unsigned
int))
-//typedef struct { unsigned int table[LZ4_STREAMDECODESIZE_U32]; }
LZ4_streamDecode_t;
typedef struct
{
- const char* dictionary;
- int dictSize;
+ BYTE* externalDict;
+ size_t extDictSize;
+ BYTE* prefixEnd;
+ size_t prefixSize;
} LZ4_streamDecode_t_internal;
/*
@@ -1074,25 +1149,32 @@ typedef struct
* LZ4_createStreamDecode()
* provides a pointer (void*) towards an initialized LZ4_streamDecode_t
structure.
*/
-void* LZ4_createStreamDecode()
+LZ4_streamDecode_t* LZ4_createStreamDecode(void)
{
- void* lz4s = ALLOCATOR(sizeof(U32), LZ4_STREAMDECODESIZE_U32);
- MEM_INIT(lz4s, 0, LZ4_STREAMDECODESIZE);
+ LZ4_streamDecode_t* lz4s = (LZ4_streamDecode_t*) ALLOCATOR(1,
sizeof(LZ4_streamDecode_t));
return lz4s;
}
+int LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream)
+{
+ FREEMEM(LZ4_stream);
+ return 0;
+}
+
/*
- * LZ4_setDictDecode
+ * LZ4_setStreamDecode
* Use this function to instruct where to find the dictionary
* This function is not necessary if previous data is still available where it
was decoded.
* Loading a size of 0 is allowed (same effect as no dictionary).
* Return : 1 if OK, 0 if error
*/
-int LZ4_setDictDecode (void* LZ4_streamDecode, const char* dictionary, int
dictSize)
+int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const char*
dictionary, int dictSize)
{
LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*)
LZ4_streamDecode;
- lz4sd->dictionary = dictionary;
- lz4sd->dictSize = dictSize;
+ lz4sd->prefixSize = (size_t) dictSize;
+ lz4sd->prefixEnd = (BYTE*) dictionary + dictSize;
+ lz4sd->externalDict = NULL;
+ lz4sd->extDictSize = 0;
return 1;
}
@@ -1101,43 +1183,61 @@ int LZ4_setDictDecode (void* LZ4_streamDecode, const
char* dictionary, int dictS
These decoding functions allow decompression of multiple blocks in
"streaming" mode.
Previously decoded blocks must still be available at the memory position
where they were decoded.
If it's not possible, save the relevant part of decoded data into a safe
buffer,
- and indicate where it stands using LZ4_setDictDecode()
+ and indicate where it stands using LZ4_setStreamDecode()
*/
-int LZ4_decompress_safe_continue (void* LZ4_streamDecode, const char* source,
char* dest, int compressedSize, int maxOutputSize)
+int LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode, const
char* source, char* dest, int compressedSize, int maxOutputSize)
{
LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*)
LZ4_streamDecode;
int result;
- result = LZ4_decompress_generic(source, dest, compressedSize,
maxOutputSize, endOnInputSize, full, 0, usingExtDict, lz4sd->dictionary,
lz4sd->dictSize);
- if (result <= 0) return result;
- if (lz4sd->dictionary + lz4sd->dictSize == dest)
+ if (lz4sd->prefixEnd == (BYTE*)dest)
{
- lz4sd->dictSize += result;
+ result = LZ4_decompress_generic(source, dest, compressedSize,
maxOutputSize,
+ endOnInputSize, full, 0,
+ usingExtDict, lz4sd->prefixEnd -
lz4sd->prefixSize, lz4sd->externalDict, lz4sd->extDictSize);
+ if (result <= 0) return result;
+ lz4sd->prefixSize += result;
+ lz4sd->prefixEnd += result;
}
else
{
- lz4sd->dictionary = dest;
- lz4sd->dictSize = result;
+ lz4sd->extDictSize = lz4sd->prefixSize;
+ lz4sd->externalDict = lz4sd->prefixEnd - lz4sd->extDictSize;
+ result = LZ4_decompress_generic(source, dest, compressedSize,
maxOutputSize,
+ endOnInputSize, full, 0,
+ usingExtDict, (BYTE*)dest,
lz4sd->externalDict, lz4sd->extDictSize);
+ if (result <= 0) return result;
+ lz4sd->prefixSize = result;
+ lz4sd->prefixEnd = (BYTE*)dest + result;
}
return result;
}
-int LZ4_decompress_fast_continue (void* LZ4_streamDecode, const char* source,
char* dest, int originalSize)
+int LZ4_decompress_fast_continue (LZ4_streamDecode_t* LZ4_streamDecode, const
char* source, char* dest, int originalSize)
{
LZ4_streamDecode_t_internal* lz4sd = (LZ4_streamDecode_t_internal*)
LZ4_streamDecode;
int result;
- result = LZ4_decompress_generic(source, dest, 0, originalSize,
endOnOutputSize, full, 0, usingExtDict, lz4sd->dictionary, lz4sd->dictSize);
- if (result <= 0) return result;
- if (lz4sd->dictionary + lz4sd->dictSize == dest)
+ if (lz4sd->prefixEnd == (BYTE*)dest)
{
- lz4sd->dictSize += result;
+ result = LZ4_decompress_generic(source, dest, 0, originalSize,
+ endOnOutputSize, full, 0,
+ usingExtDict, lz4sd->prefixEnd -
lz4sd->prefixSize, lz4sd->externalDict, lz4sd->extDictSize);
+ if (result <= 0) return result;
+ lz4sd->prefixSize += originalSize;
+ lz4sd->prefixEnd += originalSize;
}
else
{
- lz4sd->dictionary = dest;
- lz4sd->dictSize = result;
+ lz4sd->extDictSize = lz4sd->prefixSize;
+ lz4sd->externalDict = (BYTE*)dest - lz4sd->extDictSize;
+ result = LZ4_decompress_generic(source, dest, 0, originalSize,
+ endOnOutputSize, full, 0,
+ usingExtDict, (BYTE*)dest,
lz4sd->externalDict, lz4sd->extDictSize);
+ if (result <= 0) return result;
+ lz4sd->prefixSize = originalSize;
+ lz4sd->prefixEnd = (BYTE*)dest + originalSize;
}
return result;
@@ -1151,19 +1251,38 @@ Advanced decoding functions :
the dictionary must be explicitly provided within parameters
*/
+FORCE_INLINE int LZ4_decompress_usingDict_generic(const char* source, char*
dest, int compressedSize, int maxOutputSize, int safe, const char* dictStart,
int dictSize)
+{
+ if (dictSize==0)
+ return LZ4_decompress_generic(source, dest, compressedSize,
maxOutputSize, safe, full, 0, noDict, (BYTE*)dest, NULL, 0);
+ if (dictStart+dictSize == dest)
+ {
+ if (dictSize >= (int)(64 KB - 1))
+ return LZ4_decompress_generic(source, dest, compressedSize,
maxOutputSize, safe, full, 0, withPrefix64k, (BYTE*)dest-64 KB, NULL, 0);
+ return LZ4_decompress_generic(source, dest, compressedSize,
maxOutputSize, safe, full, 0, noDict, (BYTE*)dest-dictSize, NULL, 0);
+ }
+ return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
safe, full, 0, usingExtDict, (BYTE*)dest, (BYTE*)dictStart, dictSize);
+}
+
int LZ4_decompress_safe_usingDict(const char* source, char* dest, int
compressedSize, int maxOutputSize, const char* dictStart, int dictSize)
{
- return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
endOnInputSize, full, 0, usingExtDict, dictStart, dictSize);
+ return LZ4_decompress_usingDict_generic(source, dest, compressedSize,
maxOutputSize, 1, dictStart, dictSize);
}
int LZ4_decompress_fast_usingDict(const char* source, char* dest, int
originalSize, const char* dictStart, int dictSize)
{
- return LZ4_decompress_generic(source, dest, 0, originalSize,
endOnOutputSize, full, 0, usingExtDict, dictStart, dictSize);
+ return LZ4_decompress_usingDict_generic(source, dest, 0, originalSize, 0,
dictStart, dictSize);
+}
+
+/* debug function */
+int LZ4_decompress_safe_forceExtDict(const char* source, char* dest, int
compressedSize, int maxOutputSize, const char* dictStart, int dictSize)
+{
+ return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
endOnInputSize, full, 0, usingExtDict, (BYTE*)dest, (BYTE*)dictStart, dictSize);
}
/***************************************************
- Obsolete Functions
+* Obsolete Functions
***************************************************/
/*
These function names are deprecated and should no longer be used.
@@ -1179,7 +1298,7 @@ int LZ4_uncompress_unknownOutputSize (const char* source,
char* dest, int isize,
int LZ4_sizeofStreamState() { return LZ4_STREAMSIZE; }
-void LZ4_init(LZ4_stream_t_internal* lz4ds, const BYTE* base)
+static void LZ4_init(LZ4_stream_t_internal* lz4ds, const BYTE* base)
{
MEM_INIT(lz4ds, 0, LZ4_STREAMSIZE);
lz4ds->bufferStart = base;
@@ -1194,18 +1313,16 @@ int LZ4_resetStreamState(void* state, const char*
inputBuffer)
void* LZ4_create (const char* inputBuffer)
{
- void* lz4ds = ALLOCATOR(4, LZ4_STREAMSIZE_U32);
+ void* lz4ds = ALLOCATOR(8, LZ4_STREAMSIZE_U64);
LZ4_init ((LZ4_stream_t_internal*)lz4ds, (const BYTE*)inputBuffer);
return lz4ds;
}
char* LZ4_slideInputBuffer (void* LZ4_Data)
{
- LZ4_stream_t_internal* lz4ds = (LZ4_stream_t_internal*)LZ4_Data;
-
- LZ4_saveDict((LZ4_stream_t*)LZ4_Data, (char*)lz4ds->bufferStart, 64 KB);
-
- return (char*)(lz4ds->bufferStart + 64 KB);
+ LZ4_stream_t_internal* ctx = (LZ4_stream_t_internal*)LZ4_Data;
+ int dictSize = LZ4_saveDict((LZ4_stream_t*)LZ4_Data,
(char*)ctx->bufferStart, 64 KB);
+ return (char*)(ctx->bufferStart + dictSize);
}
/* Obsolete compresson functions using User-allocated state */
@@ -1217,10 +1334,10 @@ int LZ4_compress_withState (void* state, const char*
source, char* dest, int inp
if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned
on 4-bytes boundary */
MEM_INIT(state, 0, LZ4_STREAMSIZE);
- if (inputSize < (int)LZ4_64KLIMIT)
+ if (inputSize < LZ4_64Klimit)
return LZ4_compress_generic(state, source, dest, inputSize, 0,
notLimited, byU16, noDict, noDictIssue);
else
- return LZ4_compress_generic(state, source, dest, inputSize, 0,
notLimited, (sizeof(void*)==8) ? byU32 : byPtr, noDict, noDictIssue);
+ return LZ4_compress_generic(state, source, dest, inputSize, 0,
notLimited, LZ4_64bits() ? byU32 : byPtr, noDict, noDictIssue);
}
int LZ4_compress_limitedOutput_withState (void* state, const char* source,
char* dest, int inputSize, int maxOutputSize)
@@ -1228,20 +1345,23 @@ int LZ4_compress_limitedOutput_withState (void* state,
const char* source, char*
if (((size_t)(state)&3) != 0) return 0; /* Error : state is not aligned
on 4-bytes boundary */
MEM_INIT(state, 0, LZ4_STREAMSIZE);
- if (inputSize < (int)LZ4_64KLIMIT)
+ if (inputSize < LZ4_64Klimit)
return LZ4_compress_generic(state, source, dest, inputSize,
maxOutputSize, limitedOutput, byU16, noDict, noDictIssue);
else
- return LZ4_compress_generic(state, source, dest, inputSize,
maxOutputSize, limitedOutput, (sizeof(void*)==8) ? byU32 : byPtr, noDict,
noDictIssue);
+ return LZ4_compress_generic(state, source, dest, inputSize,
maxOutputSize, limitedOutput, LZ4_64bits() ? byU32 : byPtr, noDict,
noDictIssue);
}
/* Obsolete streaming decompression functions */
int LZ4_decompress_safe_withPrefix64k(const char* source, char* dest, int
compressedSize, int maxOutputSize)
{
- return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
endOnInputSize, full, 0, withPrefix64k, NULL, 64 KB);
+ return LZ4_decompress_generic(source, dest, compressedSize, maxOutputSize,
endOnInputSize, full, 0, withPrefix64k, (BYTE*)dest - 64 KB, NULL, 64 KB);
}
int LZ4_decompress_fast_withPrefix64k(const char* source, char* dest, int
originalSize)
{
- return LZ4_decompress_generic(source, dest, 0, originalSize,
endOnOutputSize, full, 0, withPrefix64k, NULL, 64 KB);
+ return LZ4_decompress_generic(source, dest, 0, originalSize,
endOnOutputSize, full, 0, withPrefix64k, (BYTE*)dest - 64 KB, NULL, 64 KB);
}
+
+#endif /* LZ4_COMMONDEFS_ONLY */
+
diff --git a/src/static_libs/lz4/lz4.h b/src/static_libs/lz4/lz4.h
index 1064fa1..de43fc0 100644
--- a/src/static_libs/lz4/lz4.h
+++ b/src/static_libs/lz4/lz4.h
@@ -1,7 +1,8 @@
/*
LZ4 - Fast LZ compression algorithm
Header File
- Copyright (C) 2011-2014, Yann Collet.
+ Copyright (C) 2011-2015, Yann Collet.
+
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
@@ -28,7 +29,7 @@
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- - LZ4 source repository : http://code.google.com/p/lz4/
+ - LZ4 source repository : https://github.com/Cyan4973/lz4
- LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
*/
#pragma once
@@ -37,17 +38,23 @@
extern "C" {
#endif
+/*
+ * lz4.h provides block compression functions, for optimal performance.
+ * If you need to generate inter-operable compressed data (respecting LZ4
frame specification),
+ * please use lz4frame.h instead.
+*/
/**************************************
- Version
+* Version
**************************************/
-#define LZ4_VERSION_MAJOR 1 /* for major interface/format changes */
-#define LZ4_VERSION_MINOR 2 /* for minor interface/format changes */
+#define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */
+#define LZ4_VERSION_MINOR 6 /* for new (non-breaking) interface
capabilities */
#define LZ4_VERSION_RELEASE 0 /* for tweaks, bug-fixes, or development */
-
+#define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR
*100 + LZ4_VERSION_RELEASE)
+int LZ4_versionNumber (void);
/**************************************
- Tuning parameter
+* Tuning parameter
**************************************/
/*
* LZ4_MEMORY_USAGE :
@@ -60,52 +67,45 @@ extern "C" {
/**************************************
- Simple Functions
+* Simple Functions
**************************************/
-int LZ4_compress (const char* source, char* dest, int inputSize);
-int LZ4_decompress_safe (const char* source, char* dest, int compressedSize,
int maxOutputSize);
+int LZ4_compress (const char* source, char* dest, int sourceSize);
+int LZ4_decompress_safe (const char* source, char* dest, int compressedSize,
int maxDecompressedSize);
/*
LZ4_compress() :
- Compresses 'inputSize' bytes from 'source' into 'dest'.
+ Compresses 'sourceSize' bytes from 'source' into 'dest'.
Destination buffer must be already allocated,
and must be sized to handle worst cases situations (input data not
compressible)
Worst case size evaluation is provided by function LZ4_compressBound()
- inputSize : Max supported value is LZ4_MAX_INPUT_VALUE
+ inputSize : Max supported value is LZ4_MAX_INPUT_SIZE
return : the number of bytes written in buffer dest
or 0 if the compression fails
LZ4_decompress_safe() :
compressedSize : is obviously the source size
- maxOutputSize : is the size of the destination buffer, which must be
already allocated.
- return : the number of bytes decoded in the destination buffer
(necessarily <= maxOutputSize)
+ maxDecompressedSize : is the size of the destination buffer, which must be
already allocated.
+ return : the number of bytes decompressed into the destination buffer
(necessarily <= maxDecompressedSize)
If the destination buffer is not large enough, decoding will stop
and output an error code (<0).
If the source stream is detected malformed, the function will
stop decoding and return a negative result.
- This function is protected against buffer overflow exploits :
- it never writes outside of output buffer, and never reads outside
of input buffer.
- Therefore, it is protected against malicious data packets.
-*/
-
-
-/*
-Note :
- Should you prefer to explicitly allocate compression-table memory using
your own allocation method,
- use the streaming functions provided below, simply reset the memory area
between each call to LZ4_compress_continue()
+ This function is protected against buffer overflow exploits,
+ and never writes outside of output buffer, nor reads outside of
input buffer.
+ It is also protected against malicious data packets.
*/
/**************************************
- Advanced Functions
+* Advanced Functions
**************************************/
#define LZ4_MAX_INPUT_SIZE 0x7E000000 /* 2 113 929 216 bytes */
#define LZ4_COMPRESSBOUND(isize) ((unsigned int)(isize) > (unsigned
int)LZ4_MAX_INPUT_SIZE ? 0 : (isize) + ((isize)/255) + 16)
/*
LZ4_compressBound() :
- Provides the maximum size that LZ4 may output in a "worst case" scenario
(input data not compressible)
- primarily useful for memory allocation of output buffer.
- macro is also provided when result needs to be evaluated at compilation
(such as stack memory allocation).
+ Provides the maximum size that LZ4 compression may output in a "worst
case" scenario (input data not compressible)
+ This function is primarily useful for memory allocation purposes (output
buffer size).
+ Macro LZ4_COMPRESSBOUND() is also provided for compilation-time evaluation
(stack memory allocation for example).
isize : is the input size. Max supported value is LZ4_MAX_INPUT_SIZE
return : maximum output size in a "worst case" scenario
@@ -116,28 +116,40 @@ int LZ4_compressBound(int isize);
/*
LZ4_compress_limitedOutput() :
- Compress 'inputSize' bytes from 'source' into an output buffer 'dest' of
maximum size 'maxOutputSize'.
+ Compress 'sourceSize' bytes from 'source' into an output buffer 'dest' of
maximum size 'maxOutputSize'.
If it cannot achieve it, compression will stop, and result of the function
will be zero.
+ This saves time and memory on detecting non-compressible (or barely
compressible) data.
This function never writes outside of provided output buffer.
- inputSize : Max supported value is LZ4_MAX_INPUT_VALUE
+ sourceSize : Max supported value is LZ4_MAX_INPUT_VALUE
maxOutputSize : is the size of the destination buffer (which must be
already allocated)
return : the number of bytes written in buffer 'dest'
- or 0 if the compression fails
+ or 0 if compression fails
*/
-int LZ4_compress_limitedOutput (const char* source, char* dest, int inputSize,
int maxOutputSize);
+int LZ4_compress_limitedOutput (const char* source, char* dest, int
sourceSize, int maxOutputSize);
+
+
+/*
+LZ4_compress_withState() :
+ Same compression functions, but using an externally allocated memory space
to store compression state.
+ Use LZ4_sizeofState() to know how much memory must be allocated,
+ and then, provide it as 'void* state' to compression functions.
+*/
+int LZ4_sizeofState(void);
+int LZ4_compress_withState (void* state, const char* source,
char* dest, int inputSize);
+int LZ4_compress_limitedOutput_withState (void* state, const char* source,
char* dest, int inputSize, int maxOutputSize);
/*
LZ4_decompress_fast() :
originalSize : is the original and therefore uncompressed size
return : the number of bytes read from the source buffer (in other words,
the compressed size)
- If the source stream is malformed, the function will stop
decoding and return a negative result.
+ If the source stream is detected malformed, the function will
stop decoding and return a negative result.
Destination buffer must be already allocated. Its size must be a
minimum of 'originalSize' bytes.
- note : This function is a bit faster than LZ4_decompress_safe()
- It provides fast decompression and fully respect memory boundaries
for properly formed compressed data.
- It does not provide full protection against intentionnally modified
data stream.
- Use this function in a trusted environment (data to decode comes
from a trusted source).
+ note : This function fully respect memory boundaries for properly formed
compressed data.
+ It is a bit faster than LZ4_decompress_safe().
+ However, it does not provide any protection against intentionally
modified data stream (malicious input).
+ Use this function in trusted environment only (data to decode comes
from a trusted source).
*/
int LZ4_decompress_fast (const char* source, char* dest, int originalSize);
@@ -145,117 +157,120 @@ int LZ4_decompress_fast (const char* source, char*
dest, int originalSize);
/*
LZ4_decompress_safe_partial() :
This function decompress a compressed block of size 'compressedSize' at
position 'source'
- into output buffer 'dest' of size 'maxOutputSize'.
+ into destination buffer 'dest' of size 'maxDecompressedSize'.
The function tries to stop decompressing operation as soon as
'targetOutputSize' has been reached,
reducing decompression time.
- return : the number of bytes decoded in the destination buffer
(necessarily <= maxOutputSize)
+ return : the number of bytes decoded in the destination buffer
(necessarily <= maxDecompressedSize)
Note : this number can be < 'targetOutputSize' should the compressed
block to decode be smaller.
Always control how many bytes were decoded.
If the source stream is detected malformed, the function will
stop decoding and return a negative result.
This function never writes outside of output buffer, and never
reads outside of input buffer. It is therefore protected against malicious data
packets
*/
-int LZ4_decompress_safe_partial (const char* source, char* dest, int
compressedSize, int targetOutputSize, int maxOutputSize);
+int LZ4_decompress_safe_partial (const char* source, char* dest, int
compressedSize, int targetOutputSize, int maxDecompressedSize);
/***********************************************
- Experimental Streaming Compression Functions
+* Streaming Compression Functions
***********************************************/
-#define LZ4_STREAMSIZE_U32 ((1 << (LZ4_MEMORY_USAGE-2)) + 8)
-#define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U32 * sizeof(unsigned int))
+#define LZ4_STREAMSIZE_U64 ((1 << (LZ4_MEMORY_USAGE-3)) + 4)
+#define LZ4_STREAMSIZE (LZ4_STREAMSIZE_U64 * sizeof(long long))
/*
* LZ4_stream_t
* information structure to track an LZ4 stream.
- * important : set this structure content to zero before first use !
+ * important : init this structure content before first use !
+ * note : only allocated directly the structure if you are statically linking
LZ4
+ * If you are using liblz4 as a DLL, please use below construction
methods instead.
*/
-typedef struct { unsigned int table[LZ4_STREAMSIZE_U32]; } LZ4_stream_t;
+typedef struct { long long table[LZ4_STREAMSIZE_U64]; } LZ4_stream_t;
/*
- * If you prefer dynamic allocation methods,
- * LZ4_createStream
- * provides a pointer (void*) towards an initialized LZ4_stream_t structure.
- * LZ4_free just frees it.
+ * LZ4_resetStream
+ * Use this function to init an allocated LZ4_stream_t structure
*/
-void* LZ4_createStream();
-int LZ4_free (void* LZ4_stream);
+void LZ4_resetStream (LZ4_stream_t* LZ4_streamPtr);
+/*
+ * LZ4_createStream will allocate and initialize an LZ4_stream_t structure
+ * LZ4_freeStream releases its memory.
+ * In the context of a DLL (liblz4), please use these methods rather than the
static struct.
+ * They are more future proof, in case of a change of LZ4_stream_t size.
+ */
+LZ4_stream_t* LZ4_createStream(void);
+int LZ4_freeStream (LZ4_stream_t* LZ4_streamPtr);
/*
* LZ4_loadDict
* Use this function to load a static dictionary into LZ4_stream.
* Any previous data will be forgotten, only 'dictionary' will remain in
memory.
- * Loading a size of 0 is allowed (same effect as init).
- * Return : 1 if OK, 0 if error
+ * Loading a size of 0 is allowed.
+ * Return : dictionary size, in bytes (necessarily <= 64 KB)
*/
-int LZ4_loadDict (void* LZ4_stream, const char* dictionary, int dictSize);
+int LZ4_loadDict (LZ4_stream_t* LZ4_streamPtr, const char* dictionary, int
dictSize);
/*
* LZ4_compress_continue
* Compress data block 'source', using blocks compressed before as dictionary
to improve compression ratio
* Previous data blocks are assumed to still be present at their previous
location.
+ * dest buffer must be already allocated, and sized to at least
LZ4_compressBound(inputSize)
*/
-int LZ4_compress_continue (void* LZ4_stream, const char* source, char* dest,
int inputSize);
+int LZ4_compress_continue (LZ4_stream_t* LZ4_streamPtr, const char* source,
char* dest, int inputSize);
/*
* LZ4_compress_limitedOutput_continue
* Same as before, but also specify a maximum target compressed size
(maxOutputSize)
* If objective cannot be met, compression exits, and returns a zero.
*/
-int LZ4_compress_limitedOutput_continue (void* LZ4_stream, const char* source,
char* dest, int inputSize, int maxOutputSize);
+int LZ4_compress_limitedOutput_continue (LZ4_stream_t* LZ4_streamPtr, const
char* source, char* dest, int inputSize, int maxOutputSize);
/*
* LZ4_saveDict
- * If previously compressed data block is not guaranteed to remain at its
previous memory location
- * save it into a safe place (char* safeBuffer)
+ * If previously compressed data block is not guaranteed to remain available
at its memory location
+ * save it into a safer place (char* safeBuffer)
* Note : you don't need to call LZ4_loadDict() afterwards,
* dictionary is immediately usable, you can therefore call again
LZ4_compress_continue()
- * Return : 1 if OK, 0 if error
- * Note : any dictSize > 64 KB will be interpreted as 64KB.
+ * Return : saved dictionary size in bytes (necessarily <= dictSize), or 0 if
error
*/
-int LZ4_saveDict (void* LZ4_stream, char* safeBuffer, int dictSize);
+int LZ4_saveDict (LZ4_stream_t* LZ4_streamPtr, char* safeBuffer, int dictSize);
/************************************************
- Experimental Streaming Decompression Functions
+* Streaming Decompression Functions
************************************************/
-#define LZ4_STREAMDECODESIZE_U32 4
-#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U32 * sizeof(unsigned
int))
+#define LZ4_STREAMDECODESIZE_U64 4
+#define LZ4_STREAMDECODESIZE (LZ4_STREAMDECODESIZE_U64 * sizeof(unsigned
long long))
+typedef struct { unsigned long long table[LZ4_STREAMDECODESIZE_U64]; }
LZ4_streamDecode_t;
/*
* LZ4_streamDecode_t
* information structure to track an LZ4 stream.
- * important : set this structure content to zero before first use !
+ * init this structure content using LZ4_setStreamDecode or memset() before
first use !
+ *
+ * In the context of a DLL (liblz4) please prefer usage of construction
methods below.
+ * They are more future proof, in case of a change of LZ4_streamDecode_t size
in the future.
+ * LZ4_createStreamDecode will allocate and initialize an LZ4_streamDecode_t
structure
+ * LZ4_freeStreamDecode releases its memory.
*/
-typedef struct { unsigned int table[LZ4_STREAMDECODESIZE_U32]; }
LZ4_streamDecode_t;
+LZ4_streamDecode_t* LZ4_createStreamDecode(void);
+int LZ4_freeStreamDecode (LZ4_streamDecode_t* LZ4_stream);
/*
- * If you prefer dynamic allocation methods,
- * LZ4_createStreamDecode()
- * provides a pointer (void*) towards an initialized LZ4_streamDecode_t
structure.
- * LZ4_free just frees it.
+ * LZ4_setStreamDecode
+ * Use this function to instruct where to find the dictionary.
+ * Setting a size of 0 is allowed (same effect as reset).
+ * Return : 1 if OK, 0 if error
*/
-void* LZ4_createStreamDecode();
-int LZ4_free (void* LZ4_stream); /* yes, it's the same one as for
compression */
+int LZ4_setStreamDecode (LZ4_streamDecode_t* LZ4_streamDecode, const char*
dictionary, int dictSize);
/*
*_continue() :
These decoding functions allow decompression of multiple blocks in
"streaming" mode.
- Previously decoded blocks must still be available at the memory position
where they were decoded.
- If it's not possible, save the relevant part of decoded data into a safe
buffer,
- and indicate where it stands using LZ4_setDictDecode()
+ Previously decoded blocks *must* remain available at the memory position
where they were decoded (up to 64 KB)
+ If this condition is not possible, save the relevant part of decoded data
into a safe buffer,
+ and indicate where is its new address using LZ4_setStreamDecode()
*/
-int LZ4_decompress_safe_continue (void* LZ4_streamDecode, const char* source,
char* dest, int compressedSize, int maxOutputSize);
-int LZ4_decompress_fast_continue (void* LZ4_streamDecode, const char* source,
char* dest, int originalSize);
-
-/*
- * LZ4_setDictDecode
- * Use this function to instruct where to find the dictionary.
- * This function can be used to specify a static dictionary,
- * or to instruct where to find some previously decoded data saved into a
different memory space.
- * Setting a size of 0 is allowed (same effect as no dictionary).
- * Return : 1 if OK, 0 if error
- */
-int LZ4_setDictDecode (void* LZ4_streamDecode, const char* dictionary, int
dictSize);
+int LZ4_decompress_safe_continue (LZ4_streamDecode_t* LZ4_streamDecode, const
char* source, char* dest, int compressedSize, int maxDecompressedSize);
+int LZ4_decompress_fast_continue (LZ4_streamDecode_t* LZ4_streamDecode, const
char* source, char* dest, int originalSize);
/*
@@ -263,17 +278,15 @@ Advanced decoding functions :
*_usingDict() :
These decoding functions work the same as
a combination of LZ4_setDictDecode() followed by
LZ4_decompress_x_continue()
- all together into a single function call.
- It doesn't use nor update an LZ4_streamDecode_t structure.
+ They are stand-alone and don't use nor update an LZ4_streamDecode_t
structure.
*/
-int LZ4_decompress_safe_usingDict (const char* source, char* dest, int
compressedSize, int maxOutputSize, const char* dictStart, int dictSize);
+int LZ4_decompress_safe_usingDict (const char* source, char* dest, int
compressedSize, int maxDecompressedSize, const char* dictStart, int dictSize);
int LZ4_decompress_fast_usingDict (const char* source, char* dest, int
originalSize, const char* dictStart, int dictSize);
-
/**************************************
- Obsolete Functions
+* Obsolete Functions
**************************************/
/*
Obsolete decompression functions
@@ -281,14 +294,11 @@ These function names are deprecated and should no longer
be used.
They are only provided here for compatibility with older user programs.
- LZ4_uncompress is the same as LZ4_decompress_fast
- LZ4_uncompress_unknownOutputSize is the same as LZ4_decompress_safe
-*/
-int LZ4_uncompress (const char* source, char* dest, int outputSize);
-int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int
isize, int maxOutputSize);
+These function prototypes are now disabled; uncomment them if you really need
them.
+It is highly recommended to stop using these functions and migrate to newer
ones */
+/* int LZ4_uncompress (const char* source, char* dest, int outputSize); */
+/* int LZ4_uncompress_unknownOutputSize (const char* source, char* dest, int
isize, int maxOutputSize); */
-/* Obsolete functions for externally allocated state; use streaming interface
instead */
-int LZ4_sizeofState(void);
-int LZ4_compress_withState (void* state, const char* source,
char* dest, int inputSize);
-int LZ4_compress_limitedOutput_withState (void* state, const char* source,
char* dest, int inputSize, int maxOutputSize);
/* Obsolete streaming functions; use new streaming interface whenever possible
*/
void* LZ4_create (const char* inputBuffer);
diff --git a/src/static_libs/lz4/lz4hc.c b/src/static_libs/lz4/lz4hc.c
index 6086749..c7a94a0 100644
--- a/src/static_libs/lz4/lz4hc.c
+++ b/src/static_libs/lz4/lz4hc.c
@@ -1,34 +1,35 @@
/*
- LZ4 HC - High Compression Mode of LZ4
- Copyright (C) 2011-2014, Yann Collet.
- BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
-
- Redistribution and use in source and binary forms, with or without
- modification, are permitted provided that the following conditions are
- met:
-
- * Redistributions of source code must retain the above copyright
- notice, this list of conditions and the following disclaimer.
- * Redistributions in binary form must reproduce the above
- copyright notice, this list of conditions and the following disclaimer
- in the documentation and/or other materials provided with the
- distribution.
-
- THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
- "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
- LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
- A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
- OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
- SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
- LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
- DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
- THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
- (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
- OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
-
- You can contact the author at :
- - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html
- - LZ4 source repository : http://code.google.com/p/lz4/
+LZ4 HC - High Compression Mode of LZ4
+Copyright (C) 2011-2015, Yann Collet.
+
+BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
+
+Redistribution and use in source and binary forms, with or without
+modification, are permitted provided that the following conditions are
+met:
+
+* Redistributions of source code must retain the above copyright
+notice, this list of conditions and the following disclaimer.
+* Redistributions in binary form must reproduce the above
+copyright notice, this list of conditions and the following disclaimer
+in the documentation and/or other materials provided with the
+distribution.
+
+THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+
+You can contact the author at :
+ - LZ4 source repository : https://github.com/Cyan4973/lz4
+ - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
*/
@@ -36,213 +37,48 @@
/**************************************
Tuning Parameter
**************************************/
-#define LZ4HC_DEFAULT_COMPRESSIONLEVEL 8
-
-
-/**************************************
- Memory routines
-**************************************/
-#include <stdlib.h> /* calloc, free */
-#define ALLOCATOR(s) calloc(1,s)
-#define FREEMEM free
-#include <string.h> /* memset, memcpy */
-#define MEM_INIT memset
-
-
-/**************************************
- CPU Feature Detection
-**************************************/
-/* 32 or 64 bits ? */
-#if (defined(__x86_64__) || defined(_M_X64) || defined(_WIN64) \
- || defined(__powerpc64__) || defined(__powerpc64le__) \
- || defined(__ppc64__) || defined(__ppc64le__) \
- || defined(__PPC64__) || defined(__PPC64LE__) \
- || defined(__ia64) || defined(__itanium__) || defined(_M_IA64) ) /*
Detects 64 bits mode */
-# define LZ4_ARCH64 1
-#else
-# define LZ4_ARCH64 0
-#endif
-
-/*
- * Little Endian or Big Endian ?
- * Overwrite the #define below if you know your architecture endianess
- */
-#include <stdlib.h> /* Apparently required to detect endianess */
-#if defined (__GLIBC__)
-# include <endian.h>
-# if (__BYTE_ORDER == __BIG_ENDIAN)
-# define LZ4_BIG_ENDIAN 1
-# endif
-#elif (defined(__BIG_ENDIAN__) || defined(__BIG_ENDIAN) ||
defined(_BIG_ENDIAN)) && !(defined(__LITTLE_ENDIAN__) ||
defined(__LITTLE_ENDIAN) || defined(_LITTLE_ENDIAN))
-# define LZ4_BIG_ENDIAN 1
-#elif defined(__sparc) || defined(__sparc__) \
- || defined(__powerpc__) || defined(__ppc__) || defined(__PPC__) \
- || defined(__hpux) || defined(__hppa) \
- || defined(_MIPSEB) || defined(__s390__)
-# define LZ4_BIG_ENDIAN 1
-#else
-/* Little Endian assumed. PDP Endian and other very rare endian format are
unsupported. */
-#endif
-
-/*
- * Unaligned memory access is automatically enabled for "common" CPU, such as
x86.
- * For others CPU, the compiler will be more cautious, and insert extra code
to ensure aligned access is respected
- * If you know your target CPU supports unaligned memory access, you want to
force this option manually to improve performance
- */
-#if defined(__ARM_FEATURE_UNALIGNED)
-# define LZ4_FORCE_UNALIGNED_ACCESS 1
-#endif
-
-/* Define this parameter if your target system or compiler does not support
hardware bit count */
-#if defined(_MSC_VER) && defined(_WIN32_WCE) /* Visual Studio for
Windows CE does not support Hardware bit count */
-# define LZ4_FORCE_SW_BITCOUNT
-#endif
-
-
-/**************************************
- Compiler Options
-**************************************/
-#if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
-/* "restrict" is a known keyword */
-#else
-# define restrict /* Disable restrict */
-#endif
-
-#ifdef _MSC_VER /* Visual Studio */
-# define FORCE_INLINE static __forceinline
-# include <intrin.h> /* For Visual 2005 */
-# if LZ4_ARCH64 /* 64-bits */
-# pragma intrinsic(_BitScanForward64) /* For Visual 2005 */
-# pragma intrinsic(_BitScanReverse64) /* For Visual 2005 */
-# else /* 32-bits */
-# pragma intrinsic(_BitScanForward) /* For Visual 2005 */
-# pragma intrinsic(_BitScanReverse) /* For Visual 2005 */
-# endif
-# pragma warning(disable : 4127) /* disable: C4127: conditional
expression is constant */
-# pragma warning(disable : 4701) /* disable: C4701: potentially
uninitialized local variable used */
-#else
-# ifdef __GNUC__
-# define FORCE_INLINE static inline __attribute__((always_inline))
-# else
-# define FORCE_INLINE static inline
-# endif
-#endif
-
-#ifdef _MSC_VER /* Visual Studio */
-# define lz4_bswap16(x) _byteswap_ushort(x)
-#else
-# define lz4_bswap16(x) ((unsigned short int) ((((x) >> 8) & 0xffu) | (((x)
& 0xffu) << 8)))
-#endif
+static const int LZ4HC_compressionLevel_default = 9;
/**************************************
Includes
**************************************/
#include "lz4hc.h"
-#include "lz4.h"
/**************************************
- Basic Types
+ Local Compiler Options
**************************************/
-#if defined (__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) /* C99 */
-# include <stdint.h>
- typedef uint8_t BYTE;
- typedef uint16_t U16;
- typedef uint32_t U32;
- typedef int32_t S32;
- typedef uint64_t U64;
-#else
- typedef unsigned char BYTE;
- typedef unsigned short U16;
- typedef unsigned int U32;
- typedef signed int S32;
- typedef unsigned long long U64;
+#if defined(__GNUC__)
+# pragma GCC diagnostic ignored "-Wunused-function"
#endif
-#if defined(__GNUC__) && !defined(LZ4_FORCE_UNALIGNED_ACCESS)
-# define _PACKED __attribute__ ((packed))
-#else
-# define _PACKED
+#if defined (__clang__)
+# pragma clang diagnostic ignored "-Wunused-function"
#endif
-#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
-# ifdef __IBMC__
-# pragma pack(1)
-# else
-# pragma pack(push, 1)
-# endif
-#endif
-
-typedef struct _U16_S { U16 v; } _PACKED U16_S;
-typedef struct _U32_S { U32 v; } _PACKED U32_S;
-typedef struct _U64_S { U64 v; } _PACKED U64_S;
-
-#if !defined(LZ4_FORCE_UNALIGNED_ACCESS) && !defined(__GNUC__)
-# pragma pack(pop)
-#endif
-#define A64(x) (((U64_S *)(x))->v)
-#define A32(x) (((U32_S *)(x))->v)
-#define A16(x) (((U16_S *)(x))->v)
+/**************************************
+ Common LZ4 definition
+**************************************/
+#define LZ4_COMMONDEFS_ONLY
+#include "lz4.c"
/**************************************
- Constants
+ Local Constants
**************************************/
-#define MINMATCH 4
-
#define DICTIONARY_LOGSIZE 16
#define MAXD (1<<DICTIONARY_LOGSIZE)
#define MAXD_MASK ((U32)(MAXD - 1))
-#define MAX_DISTANCE (MAXD - 1)
#define HASH_LOG (DICTIONARY_LOGSIZE-1)
#define HASHTABLESIZE (1 << HASH_LOG)
#define HASH_MASK (HASHTABLESIZE - 1)
-#define ML_BITS 4
-#define ML_MASK (size_t)((1U<<ML_BITS)-1)
-#define RUN_BITS (8-ML_BITS)
-#define RUN_MASK ((1U<<RUN_BITS)-1)
-
-#define COPYLENGTH 8
-#define LASTLITERALS 5
-#define MFLIMIT (COPYLENGTH+MINMATCH)
-#define MINLENGTH (MFLIMIT+1)
#define OPTIMAL_ML (int)((ML_MASK-1)+MINMATCH)
-#define KB *(1U<<10)
-#define MB *(1U<<20)
-#define GB *(1U<<30)
-
-
-/**************************************
- Architecture-specific macros
-**************************************/
-#if LZ4_ARCH64 /* 64-bit */
-# define STEPSIZE 8
-# define LZ4_COPYSTEP(s,d) A64(d) = A64(s); d+=8; s+=8;
-# define LZ4_COPYPACKET(s,d) LZ4_COPYSTEP(s,d)
-# define AARCH A64
-# define HTYPE U32
-# define INITBASE(b,s) const BYTE* const b = s
-#else /* 32-bit */
-# define STEPSIZE 4
-# define LZ4_COPYSTEP(s,d) A32(d) = A32(s); d+=4; s+=4;
-# define LZ4_COPYPACKET(s,d) LZ4_COPYSTEP(s,d); LZ4_COPYSTEP(s,d);
-# define AARCH A32
-# define HTYPE U32
-# define INITBASE(b,s) const BYTE* const b = s
-#endif
-
-#if defined(LZ4_BIG_ENDIAN)
-# define LZ4_READ_LITTLEENDIAN_16(d,s,p) { U16 v = A16(p); v =
lz4_bswap16(v); d = (s) - v; }
-# define LZ4_WRITE_LITTLEENDIAN_16(p,i) { U16 v = (U16)(i); v =
lz4_bswap16(v); A16(p) = v; p+=2; }
-#else /* Little Endian */
-# define LZ4_READ_LITTLEENDIAN_16(d,s,p) { d = (s) - A16(p); }
-# define LZ4_WRITE_LITTLEENDIAN_16(p,v) { A16(p) = v; p+=2; }
-#endif
+static const int g_maxCompressionLevel = 16;
/**************************************
@@ -250,312 +86,195 @@ typedef struct _U64_S { U64 v; } _PACKED U64_S;
**************************************/
typedef struct
{
- const BYTE* inputBuffer;
- const BYTE* base;
- const BYTE* end;
- HTYPE hashTable[HASHTABLESIZE];
- U16 chainTable[MAXD];
- const BYTE* nextToUpdate;
+ U32 hashTable[HASHTABLESIZE];
+ U16 chainTable[MAXD];
+ const BYTE* end; /* next block here to continue on current prefix */
+ const BYTE* base; /* All index relative to this position */
+ const BYTE* dictBase; /* alternate base for extDict */
+ const BYTE* inputBuffer;/* deprecated */
+ U32 dictLimit; /* below that point, need extDict */
+ U32 lowLimit; /* below that point, no more dict */
+ U32 nextToUpdate;
+ U32 compressionLevel;
} LZ4HC_Data_Structure;
/**************************************
- Macros
+ Local Macros
**************************************/
-#define LZ4_WILDCOPY(s,d,e) do { LZ4_COPYPACKET(s,d) } while (d<e);
-#define LZ4_BLINDCOPY(s,d,l) { BYTE* e=d+l; LZ4_WILDCOPY(s,d,e); d=e; }
#define HASH_FUNCTION(i) (((i) * 2654435761U) >> ((MINMATCH*8)-HASH_LOG))
-#define HASH_VALUE(p) HASH_FUNCTION(A32(p))
-#define HASH_POINTER(p) (HashTable[HASH_VALUE(p)] + base)
#define DELTANEXT(p) chainTable[(size_t)(p) & MAXD_MASK]
#define GETNEXT(p) ((p) - (size_t)DELTANEXT(p))
+static U32 LZ4HC_hashPtr(const void* ptr) { return
HASH_FUNCTION(LZ4_read32(ptr)); }
-/**************************************
- Private functions
-**************************************/
-#if LZ4_ARCH64
-
-FORCE_INLINE int LZ4_NbCommonBytes (register U64 val)
-{
-#if defined(LZ4_BIG_ENDIAN)
-# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
- unsigned long r = 0;
- _BitScanReverse64( &r, val );
- return (int)(r>>3);
-# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
- return (__builtin_clzll(val) >> 3);
-# else
- int r;
- if (!(val>>32)) { r=4; } else { r=0; val>>=32; }
- if (!(val>>16)) { r+=2; val>>=8; } else { val>>=24; }
- r += (!val);
- return r;
-# endif
-#else
-# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
- unsigned long r = 0;
- _BitScanForward64( &r, val );
- return (int)(r>>3);
-# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
- return (__builtin_ctzll(val) >> 3);
-# else
- static const int DeBruijnBytePos[64] = { 0, 0, 0, 0, 0, 1, 1, 2, 0, 3, 1,
3, 1, 4, 2, 7, 0, 2, 3, 6, 1, 5, 3, 5, 1, 3, 4, 4, 2, 5, 6, 7, 7, 0, 1, 2, 3,
3, 4, 6, 2, 6, 5, 5, 3, 4, 5, 6, 7, 1, 2, 4, 6, 4, 4, 5, 7, 2, 6, 5, 7, 6, 7, 7
};
- return DeBruijnBytePos[((U64)((val & -val) * 0x0218A392CDABBD3F)) >> 58];
-# endif
-#endif
-}
-
-#else
-FORCE_INLINE int LZ4_NbCommonBytes (register U32 val)
-{
-#if defined(LZ4_BIG_ENDIAN)
-# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
- unsigned long r;
- _BitScanReverse( &r, val );
- return (int)(r>>3);
-# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
- return (__builtin_clz(val) >> 3);
-# else
- int r;
- if (!(val>>16)) { r=2; val>>=8; } else { r=0; val>>=24; }
- r += (!val);
- return r;
-# endif
-#else
-# if defined(_MSC_VER) && !defined(LZ4_FORCE_SW_BITCOUNT)
- unsigned long r;
- _BitScanForward( &r, val );
- return (int)(r>>3);
-# elif defined(__GNUC__) && ((__GNUC__ * 100 + __GNUC_MINOR__) >= 304) &&
!defined(LZ4_FORCE_SW_BITCOUNT)
- return (__builtin_ctz(val) >> 3);
-# else
- static const int DeBruijnBytePos[32] = { 0, 0, 3, 0, 3, 1, 3, 0, 3, 2, 2,
1, 3, 2, 0, 1, 3, 3, 1, 2, 2, 2, 2, 0, 3, 1, 2, 0, 1, 0, 1, 1 };
- return DeBruijnBytePos[((U32)((val & -(S32)val) * 0x077CB531U)) >> 27];
-# endif
-#endif
-}
-#endif
-
-
-int LZ4_sizeofStreamStateHC()
-{
- return sizeof(LZ4HC_Data_Structure);
-}
-
-FORCE_INLINE void LZ4_initHC (LZ4HC_Data_Structure* hc4, const BYTE* base)
+/**************************************
+ HC Compression
+**************************************/
+static void LZ4HC_init (LZ4HC_Data_Structure* hc4, const BYTE* start)
{
MEM_INIT((void*)hc4->hashTable, 0, sizeof(hc4->hashTable));
MEM_INIT(hc4->chainTable, 0xFF, sizeof(hc4->chainTable));
- hc4->nextToUpdate = base + 1;
- hc4->base = base;
- hc4->inputBuffer = base;
- hc4->end = base;
-}
-
-int LZ4_resetStreamStateHC(void* state, const char* inputBuffer)
-{
- if ((((size_t)state) & (sizeof(void*)-1)) != 0) return 1; /* Error :
pointer is not aligned for pointer (32 or 64 bits) */
- LZ4_initHC((LZ4HC_Data_Structure*)state, (const BYTE*)inputBuffer);
- return 0;
-}
-
-
-void* LZ4_createHC (const char* inputBuffer)
-{
- void* hc4 = ALLOCATOR(sizeof(LZ4HC_Data_Structure));
- LZ4_initHC ((LZ4HC_Data_Structure*)hc4, (const BYTE*)inputBuffer);
- return hc4;
-}
-
-
-int LZ4_freeHC (void* LZ4HC_Data)
-{
- FREEMEM(LZ4HC_Data);
- return (0);
+ hc4->nextToUpdate = 64 KB;
+ hc4->base = start - 64 KB;
+ hc4->inputBuffer = start;
+ hc4->end = start;
+ hc4->dictBase = start - 64 KB;
+ hc4->dictLimit = 64 KB;
+ hc4->lowLimit = 64 KB;
}
/* Update chains up to ip (excluded) */
FORCE_INLINE void LZ4HC_Insert (LZ4HC_Data_Structure* hc4, const BYTE* ip)
{
- U16* chainTable = hc4->chainTable;
- HTYPE* HashTable = hc4->hashTable;
- INITBASE(base,hc4->base);
+ U16* chainTable = hc4->chainTable;
+ U32* HashTable = hc4->hashTable;
+ const BYTE* const base = hc4->base;
+ const U32 target = (U32)(ip - base);
+ U32 idx = hc4->nextToUpdate;
- while(hc4->nextToUpdate < ip)
+ while(idx < target)
{
- const BYTE* const p = hc4->nextToUpdate;
- size_t delta = (p) - HASH_POINTER(p);
+ U32 h = LZ4HC_hashPtr(base+idx);
+ size_t delta = idx - HashTable[h];
if (delta>MAX_DISTANCE) delta = MAX_DISTANCE;
- DELTANEXT(p) = (U16)delta;
- HashTable[HASH_VALUE(p)] = (HTYPE)((p) - base);
- hc4->nextToUpdate++;
+ chainTable[idx & 0xFFFF] = (U16)delta;
+ HashTable[h] = idx;
+ idx++;
}
-}
-
-char* LZ4_slideInputBufferHC(void* LZ4HC_Data)
-{
- LZ4HC_Data_Structure* hc4 = (LZ4HC_Data_Structure*)LZ4HC_Data;
- U32 distance = (U32)(hc4->end - hc4->inputBuffer) - 64 KB;
- distance = (distance >> 16) << 16; /* Must be a multiple of 64 KB */
- LZ4HC_Insert(hc4, hc4->end - MINMATCH);
- memcpy((void*)(hc4->end - 64 KB - distance), (const void*)(hc4->end - 64
KB), 64 KB);
- hc4->nextToUpdate -= distance;
- hc4->base -= distance;
- if ((U32)(hc4->inputBuffer - hc4->base) > 1 GB + 64 KB) /* Avoid
overflow */
- {
- int i;
- hc4->base += 1 GB;
- for (i=0; i<HASHTABLESIZE; i++) hc4->hashTable[i] -= 1 GB;
- }
- hc4->end -= distance;
- return (char*)(hc4->end);
+ hc4->nextToUpdate = target;
}
-FORCE_INLINE size_t LZ4HC_CommonLength (const BYTE* p1, const BYTE* p2, const
BYTE* const matchlimit)
-{
- const BYTE* p1t = p1;
-
- while (p1t<matchlimit-(STEPSIZE-1))
- {
- size_t diff = AARCH(p2) ^ AARCH(p1t);
- if (!diff) { p1t+=STEPSIZE; p2+=STEPSIZE; continue; }
- p1t += LZ4_NbCommonBytes(diff);
- return (p1t - p1);
- }
- if (LZ4_ARCH64) if ((p1t<(matchlimit-3)) && (A32(p2) == A32(p1t))) {
p1t+=4; p2+=4; }
- if ((p1t<(matchlimit-1)) && (A16(p2) == A16(p1t))) { p1t+=2; p2+=2; }
- if ((p1t<matchlimit) && (*p2 == *p1t)) p1t++;
- return (p1t - p1);
-}
-
-
-FORCE_INLINE int LZ4HC_InsertAndFindBestMatch (LZ4HC_Data_Structure* hc4,
const BYTE* ip, const BYTE* const matchlimit, const BYTE** matchpos, const int
maxNbAttempts)
+FORCE_INLINE int LZ4HC_InsertAndFindBestMatch (LZ4HC_Data_Structure* hc4, /*
Index table will be updated */
+ const BYTE* ip, const BYTE*
const iLimit,
+ const BYTE** matchpos,
+ const int maxNbAttempts)
{
U16* const chainTable = hc4->chainTable;
- HTYPE* const HashTable = hc4->hashTable;
- const BYTE* ref;
- INITBASE(base,hc4->base);
+ U32* const HashTable = hc4->hashTable;
+ const BYTE* const base = hc4->base;
+ const BYTE* const dictBase = hc4->dictBase;
+ const U32 dictLimit = hc4->dictLimit;
+ const U32 lowLimit = (hc4->lowLimit + 64 KB > (U32)(ip-base)) ?
hc4->lowLimit : (U32)(ip - base) - (64 KB - 1);
+ U32 matchIndex;
+ const BYTE* match;
int nbAttempts=maxNbAttempts;
- size_t repl=0, ml=0;
- U16 delta=0; /* useless assignment, to remove an uninitialization warning
*/
+ size_t ml=0;
/* HC4 match finder */
LZ4HC_Insert(hc4, ip);
- ref = HASH_POINTER(ip);
-
-#define REPEAT_OPTIMIZATION
-#ifdef REPEAT_OPTIMIZATION
- /* Detect repetitive sequences of length <= 4 */
- if ((U32)(ip-ref) <= 4) /* potential repetition */
- {
- if (A32(ref) == A32(ip)) /* confirmed */
- {
- delta = (U16)(ip-ref);
- repl = ml = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH,
matchlimit) + MINMATCH;
- *matchpos = ref;
- }
- ref = GETNEXT(ref);
- }
-#endif
+ matchIndex = HashTable[LZ4HC_hashPtr(ip)];
- while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts))
+ while ((matchIndex>=lowLimit) && (nbAttempts))
{
nbAttempts--;
- if (*(ref+ml) == *(ip+ml))
- if (A32(ref) == A32(ip))
+ if (matchIndex >= dictLimit)
{
- size_t mlt = LZ4HC_CommonLength(ip+MINMATCH, ref+MINMATCH,
matchlimit) + MINMATCH;
- if (mlt > ml) { ml = mlt; *matchpos = ref; }
+ match = base + matchIndex;
+ if (*(match+ml) == *(ip+ml)
+ && (LZ4_read32(match) == LZ4_read32(ip)))
+ {
+ size_t mlt = LZ4_count(ip+MINMATCH, match+MINMATCH, iLimit) +
MINMATCH;
+ if (mlt > ml) { ml = mlt; *matchpos = match; }
+ }
}
- ref = GETNEXT(ref);
- }
-
-#ifdef REPEAT_OPTIMIZATION
- /* Complete table */
- if (repl)
- {
- const BYTE* ptr = ip;
- const BYTE* end;
-
- end = ip + repl - (MINMATCH-1);
- while(ptr < end-delta)
+ else
{
- DELTANEXT(ptr) = delta; /* Pre-Load */
- ptr++;
+ match = dictBase + matchIndex;
+ if (LZ4_read32(match) == LZ4_read32(ip))
+ {
+ size_t mlt;
+ const BYTE* vLimit = ip + (dictLimit - matchIndex);
+ if (vLimit > iLimit) vLimit = iLimit;
+ mlt = LZ4_count(ip+MINMATCH, match+MINMATCH, vLimit) +
MINMATCH;
+ if ((ip+mlt == vLimit) && (vLimit < iLimit))
+ mlt += LZ4_count(ip+mlt, base+dictLimit, iLimit);
+ if (mlt > ml) { ml = mlt; *matchpos = base + matchIndex; }
/* virtual matchpos */
+ }
}
- do
- {
- DELTANEXT(ptr) = delta;
- HashTable[HASH_VALUE(ptr)] = (HTYPE)((ptr) - base); /* Head of
chain */
- ptr++;
- } while(ptr < end);
- hc4->nextToUpdate = end;
+ matchIndex -= chainTable[matchIndex & 0xFFFF];
}
-#endif
return (int)ml;
}
-FORCE_INLINE int LZ4HC_InsertAndGetWiderMatch (LZ4HC_Data_Structure* hc4,
const BYTE* ip, const BYTE* startLimit, const BYTE* matchlimit, int longest,
const BYTE** matchpos, const BYTE** startpos, const int maxNbAttempts)
+FORCE_INLINE int LZ4HC_InsertAndGetWiderMatch (
+ LZ4HC_Data_Structure* hc4,
+ const BYTE* const ip,
+ const BYTE* const iLowLimit,
+ const BYTE* const iHighLimit,
+ int longest,
+ const BYTE** matchpos,
+ const BYTE** startpos,
+ const int maxNbAttempts)
{
- U16* const chainTable = hc4->chainTable;
- HTYPE* const HashTable = hc4->hashTable;
- INITBASE(base,hc4->base);
- const BYTE* ref;
+ U16* const chainTable = hc4->chainTable;
+ U32* const HashTable = hc4->hashTable;
+ const BYTE* const base = hc4->base;
+ const U32 dictLimit = hc4->dictLimit;
+ const BYTE* const lowPrefixPtr = base + dictLimit;
+ const U32 lowLimit = (hc4->lowLimit + 64 KB > (U32)(ip-base)) ?
hc4->lowLimit : (U32)(ip - base) - (64 KB - 1);
+ const BYTE* const dictBase = hc4->dictBase;
+ U32 matchIndex;
int nbAttempts = maxNbAttempts;
- int delta = (int)(ip-startLimit);
+ int delta = (int)(ip-iLowLimit);
+
/* First Match */
LZ4HC_Insert(hc4, ip);
- ref = HASH_POINTER(ip);
+ matchIndex = HashTable[LZ4HC_hashPtr(ip)];
- while (((U32)(ip-ref) <= MAX_DISTANCE) && (nbAttempts))
+ while ((matchIndex>=lowLimit) && (nbAttempts))
{
nbAttempts--;
- if (*(startLimit + longest) == *(ref - delta + longest))
- if (A32(ref) == A32(ip))
+ if (matchIndex >= dictLimit)
{
-#if 1
- const BYTE* reft = ref+MINMATCH;
- const BYTE* ipt = ip+MINMATCH;
- const BYTE* startt = ip;
+ const BYTE* matchPtr = base + matchIndex;
+ if (*(iLowLimit + longest) == *(matchPtr - delta + longest))
+ if (LZ4_read32(matchPtr) == LZ4_read32(ip))
+ {
+ int mlt = MINMATCH + LZ4_count(ip+MINMATCH,
matchPtr+MINMATCH, iHighLimit);
+ int back = 0;
- while (ipt<matchlimit-(STEPSIZE-1))
- {
- size_t diff = AARCH(reft) ^ AARCH(ipt);
- if (!diff) { ipt+=STEPSIZE; reft+=STEPSIZE; continue; }
- ipt += LZ4_NbCommonBytes(diff);
- goto _endCount;
- }
- if (LZ4_ARCH64) if ((ipt<(matchlimit-3)) && (A32(reft) ==
A32(ipt))) { ipt+=4; reft+=4; }
- if ((ipt<(matchlimit-1)) && (A16(reft) == A16(ipt))) { ipt+=2;
reft+=2; }
- if ((ipt<matchlimit) && (*reft == *ipt)) ipt++;
-_endCount:
- reft = ref;
-#else
- /* Easier for code maintenance, but unfortunately slower too */
- const BYTE* startt = ip;
- const BYTE* reft = ref;
- const BYTE* ipt = ip + MINMATCH + LZ4HC_CommonLength(ip+MINMATCH,
ref+MINMATCH, matchlimit);
-#endif
+ while ((ip+back>iLowLimit)
+ && (matchPtr+back > lowPrefixPtr)
+ && (ip[back-1] == matchPtr[back-1]))
+ back--;
- while ((startt>startLimit) && (reft > hc4->inputBuffer) &&
(startt[-1] == reft[-1])) {startt--; reft--;}
+ mlt -= back;
- if ((ipt-startt) > longest)
+ if (mlt > longest)
+ {
+ longest = (int)mlt;
+ *matchpos = matchPtr+back;
+ *startpos = ip+back;
+ }
+ }
+ }
+ else
+ {
+ const BYTE* matchPtr = dictBase + matchIndex;
+ if (LZ4_read32(matchPtr) == LZ4_read32(ip))
{
- longest = (int)(ipt-startt);
- *matchpos = reft;
- *startpos = startt;
+ size_t mlt;
+ int back=0;
+ const BYTE* vLimit = ip + (dictLimit - matchIndex);
+ if (vLimit > iHighLimit) vLimit = iHighLimit;
+ mlt = LZ4_count(ip+MINMATCH, matchPtr+MINMATCH, vLimit) +
MINMATCH;
+ if ((ip+mlt == vLimit) && (vLimit < iHighLimit))
+ mlt += LZ4_count(ip+mlt, base+dictLimit, iHighLimit);
+ while ((ip+back > iLowLimit) && (matchIndex+back > lowLimit)
&& (ip[back-1] == matchPtr[back-1])) back--;
+ mlt -= back;
+ if ((int)mlt > longest) { longest = (int)mlt; *matchpos = base
+ matchIndex + back; *startpos = ip+back; }
}
}
- ref = GETNEXT(ref);
+ matchIndex -= chainTable[matchIndex & 0xFFFF];
}
return longest;
@@ -564,34 +283,44 @@ _endCount:
typedef enum { noLimit = 0, limitedOutput = 1 } limitedOutput_directive;
+#define LZ4HC_DEBUG 0
+#if LZ4HC_DEBUG
+static unsigned debug = 0;
+#endif
+
FORCE_INLINE int LZ4HC_encodeSequence (
- const BYTE** ip,
- BYTE** op,
- const BYTE** anchor,
- int matchLength,
- const BYTE* ref,
- limitedOutput_directive limitedOutputBuffer,
- BYTE* oend)
+ const BYTE** ip,
+ BYTE** op,
+ const BYTE** anchor,
+ int matchLength,
+ const BYTE* const match,
+ limitedOutput_directive limitedOutputBuffer,
+ BYTE* oend)
{
int length;
BYTE* token;
+#if LZ4HC_DEBUG
+ if (debug) printf("literal : %u -- match : %u -- offset : %u\n",
(U32)(*ip - *anchor), (U32)matchLength, (U32)(*ip-match));
+#endif
+
/* Encode Literal length */
length = (int)(*ip - *anchor);
token = (*op)++;
- if ((limitedOutputBuffer) && ((*op + length + (2 + 1 + LASTLITERALS) +
(length>>8)) > oend)) return 1; /* Check output limit */
+ if ((limitedOutputBuffer) && ((*op + (length>>8) + length + (2 + 1 +
LASTLITERALS)) > oend)) return 1; /* Check output limit */
if (length>=(int)RUN_MASK) { int len; *token=(RUN_MASK<<ML_BITS); len =
length-RUN_MASK; for(; len > 254 ; len-=255) *(*op)++ = 255; *(*op)++ =
(BYTE)len; }
else *token = (BYTE)(length<<ML_BITS);
/* Copy Literals */
- LZ4_BLINDCOPY(*anchor, *op, length);
+ LZ4_wildCopy(*op, *anchor, (*op) + length);
+ *op += length;
/* Encode Offset */
- LZ4_WRITE_LITTLEENDIAN_16(*op,(U16)(*ip-ref));
+ LZ4_writeLE16(*op, (U16)(*ip-match)); *op += 2;
/* Encode MatchLength */
length = (int)(matchLength-MINMATCH);
- if ((limitedOutputBuffer) && (*op + (1 + LASTLITERALS) + (length>>8) >
oend)) return 1; /* Check output limit */
+ if ((limitedOutputBuffer) && (*op + (length>>8) + (1 + LASTLITERALS) >
oend)) return 1; /* Check output limit */
if (length>=(int)ML_MASK) { *token+=ML_MASK; length-=ML_MASK; for(; length
> 509 ; length-=510) { *(*op)++ = 255; *(*op)++ = 255; } if (length > 254) {
length-=255; *(*op)++ = 255; } *(*op)++ = (BYTE)length; }
else *token += (BYTE)(length);
@@ -603,16 +332,15 @@ FORCE_INLINE int LZ4HC_encodeSequence (
}
-#define MAX_COMPRESSION_LEVEL 16
static int LZ4HC_compress_generic (
- void* ctxvoid,
- const char* source,
- char* dest,
- int inputSize,
- int maxOutputSize,
- int compressionLevel,
- limitedOutput_directive limit
- )
+ void* ctxvoid,
+ const char* source,
+ char* dest,
+ int inputSize,
+ int maxOutputSize,
+ int compressionLevel,
+ limitedOutput_directive limit
+ )
{
LZ4HC_Data_Structure* ctx = (LZ4HC_Data_Structure*) ctxvoid;
const BYTE* ip = (const BYTE*) source;
@@ -624,7 +352,7 @@ static int LZ4HC_compress_generic (
BYTE* op = (BYTE*) dest;
BYTE* const oend = op + maxOutputSize;
- const int maxNbAttempts = compressionLevel > MAX_COMPRESSION_LEVEL ? 1 <<
MAX_COMPRESSION_LEVEL : compressionLevel ? 1<<(compressionLevel-1) :
1<<LZ4HC_DEFAULT_COMPRESSIONLEVEL;
+ unsigned maxNbAttempts;
int ml, ml2, ml3, ml0;
const BYTE* ref=NULL;
const BYTE* start2=NULL;
@@ -635,8 +363,10 @@ static int LZ4HC_compress_generic (
const BYTE* ref0;
- /* Ensure blocks follow each other */
- if (ip != ctx->end) return 0;
+ /* init */
+ if (compressionLevel > g_maxCompressionLevel) compressionLevel =
g_maxCompressionLevel;
+ if (compressionLevel < 1) compressionLevel =
LZ4HC_compressionLevel_default;
+ maxNbAttempts = 1 << (compressionLevel-1);
ctx->end += inputSize;
ip++;
@@ -684,10 +414,10 @@ _Search2:
_Search3:
/*
- * Currently we have :
- * ml2 > ml1, and
- * ip1+3 <= ip2 (usually < ip1+ml1)
- */
+ * Currently we have :
+ * ml2 > ml1, and
+ * ip1+3 <= ip2 (usually < ip1+ml1)
+ */
if ((start2 - ip) < OPTIMAL_ML)
{
int correction;
@@ -755,9 +485,9 @@ _Search3:
}
/*
- * OK, now we have 3 ascending matches; let's write at least the first
one
- * ip & ref are known; Now for ml
- */
+ * OK, now we have 3 ascending matches; let's write at least the first
one
+ * ip & ref are known; Now for ml
+ */
if (start2 < ip+ml)
{
if ((start2 - ip) < (int)ML_MASK)
@@ -789,7 +519,6 @@ _Search3:
ml2 = ml3;
goto _Search3;
-
}
/* Encode Last Literals */
@@ -809,28 +538,18 @@ _Search3:
int LZ4_compressHC2(const char* source, char* dest, int inputSize, int
compressionLevel)
{
- void* ctx = LZ4_createHC(source);
- int result;
- if (ctx==NULL) return 0;
-
- result = LZ4HC_compress_generic (ctx, source, dest, inputSize, 0,
compressionLevel, noLimit);
-
- LZ4_freeHC(ctx);
- return result;
+ LZ4HC_Data_Structure ctx;
+ LZ4HC_init(&ctx, (const BYTE*)source);
+ return LZ4HC_compress_generic (&ctx, source, dest, inputSize, 0,
compressionLevel, noLimit);
}
int LZ4_compressHC(const char* source, char* dest, int inputSize) { return
LZ4_compressHC2(source, dest, inputSize, 0); }
int LZ4_compressHC2_limitedOutput(const char* source, char* dest, int
inputSize, int maxOutputSize, int compressionLevel)
{
- void* ctx = LZ4_createHC(source);
- int result;
- if (ctx==NULL) return 0;
-
- result = LZ4HC_compress_generic (ctx, source, dest, inputSize,
maxOutputSize, compressionLevel, limitedOutput);
-
- LZ4_freeHC(ctx);
- return result;
+ LZ4HC_Data_Structure ctx;
+ LZ4HC_init(&ctx, (const BYTE*)source);
+ return LZ4HC_compress_generic (&ctx, source, dest, inputSize,
maxOutputSize, compressionLevel, limitedOutput);
}
int LZ4_compressHC_limitedOutput(const char* source, char* dest, int
inputSize, int maxOutputSize)
@@ -840,15 +559,15 @@ int LZ4_compressHC_limitedOutput(const char* source,
char* dest, int inputSize,
/*****************************
- Using external allocation
-*****************************/
-int LZ4_sizeofStateHC() { return sizeof(LZ4HC_Data_Structure); }
+ * Using external allocation
+ * ***************************/
+int LZ4_sizeofStateHC(void) { return sizeof(LZ4HC_Data_Structure); }
int LZ4_compressHC2_withStateHC (void* state, const char* source, char* dest,
int inputSize, int compressionLevel)
{
if (((size_t)(state)&(sizeof(void*)-1)) != 0) return 0; /* Error : state
is not aligned for pointers (32 or 64 bits) */
- LZ4_initHC ((LZ4HC_Data_Structure*)state, (const BYTE*)source);
+ LZ4HC_init ((LZ4HC_Data_Structure*)state, (const BYTE*)source);
return LZ4HC_compress_generic (state, source, dest, inputSize, 0,
compressionLevel, noLimit);
}
@@ -859,7 +578,7 @@ int LZ4_compressHC_withStateHC (void* state, const char*
source, char* dest, int
int LZ4_compressHC2_limitedOutput_withStateHC (void* state, const char*
source, char* dest, int inputSize, int maxOutputSize, int compressionLevel)
{
if (((size_t)(state)&(sizeof(void*)-1)) != 0) return 0; /* Error : state
is not aligned for pointers (32 or 64 bits) */
- LZ4_initHC ((LZ4HC_Data_Structure*)state, (const BYTE*)source);
+ LZ4HC_init ((LZ4HC_Data_Structure*)state, (const BYTE*)source);
return LZ4HC_compress_generic (state, source, dest, inputSize,
maxOutputSize, compressionLevel, limitedOutput);
}
@@ -867,26 +586,171 @@ int LZ4_compressHC_limitedOutput_withStateHC (void*
state, const char* source, c
{ return LZ4_compressHC2_limitedOutput_withStateHC (state, source, dest,
inputSize, maxOutputSize, 0); }
-/****************************
- Stream functions
-****************************/
-int LZ4_compressHC_continue (void* LZ4HC_Data, const char* source, char* dest,
int inputSize)
+/**************************************
+ * Streaming Functions
+ * ************************************/
+/* allocation */
+LZ4_streamHC_t* LZ4_createStreamHC(void) { return
(LZ4_streamHC_t*)malloc(sizeof(LZ4_streamHC_t)); }
+int LZ4_freeStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr) {
free(LZ4_streamHCPtr); return 0; }
+
+
+/* initialization */
+void LZ4_resetStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr, int compressionLevel)
{
- return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, 0, 0,
noLimit);
+ LZ4_STATIC_ASSERT(sizeof(LZ4HC_Data_Structure) <= sizeof(LZ4_streamHC_t));
/* if compilation fails here, LZ4_STREAMHCSIZE must be increased */
+ ((LZ4HC_Data_Structure*)LZ4_streamHCPtr)->base = NULL;
+ ((LZ4HC_Data_Structure*)LZ4_streamHCPtr)->compressionLevel =
(unsigned)compressionLevel;
}
-int LZ4_compressHC2_continue (void* LZ4HC_Data, const char* source, char*
dest, int inputSize, int compressionLevel)
+int LZ4_loadDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, const char* dictionary,
int dictSize)
{
- return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, 0,
compressionLevel, noLimit);
+ LZ4HC_Data_Structure* ctxPtr = (LZ4HC_Data_Structure*) LZ4_streamHCPtr;
+ if (dictSize > 64 KB)
+ {
+ dictionary += dictSize - 64 KB;
+ dictSize = 64 KB;
+ }
+ LZ4HC_init (ctxPtr, (const BYTE*)dictionary);
+ if (dictSize >= 4) LZ4HC_Insert (ctxPtr, (const BYTE*)dictionary
+(dictSize-3));
+ ctxPtr->end = (const BYTE*)dictionary + dictSize;
+ return dictSize;
}
+
+/* compression */
+
+static void LZ4HC_setExternalDict(LZ4HC_Data_Structure* ctxPtr, const BYTE*
newBlock)
+{
+ if (ctxPtr->end >= ctxPtr->base + 4)
+ LZ4HC_Insert (ctxPtr, ctxPtr->end-3); /* Referencing remaining
dictionary content */
+ /* Only one memory segment for extDict, so any previous extDict is lost at
this stage */
+ ctxPtr->lowLimit = ctxPtr->dictLimit;
+ ctxPtr->dictLimit = (U32)(ctxPtr->end - ctxPtr->base);
+ ctxPtr->dictBase = ctxPtr->base;
+ ctxPtr->base = newBlock - ctxPtr->dictLimit;
+ ctxPtr->end = newBlock;
+ ctxPtr->nextToUpdate = ctxPtr->dictLimit; /* match referencing will
resume from there */
+}
+
+static int LZ4_compressHC_continue_generic (LZ4HC_Data_Structure* ctxPtr,
+ const char* source, char* dest,
+ int inputSize, int maxOutputSize,
limitedOutput_directive limit)
+{
+ /* auto-init if forgotten */
+ if (ctxPtr->base == NULL)
+ LZ4HC_init (ctxPtr, (const BYTE*) source);
+
+ /* Check overflow */
+ if ((size_t)(ctxPtr->end - ctxPtr->base) > 2 GB)
+ {
+ size_t dictSize = (size_t)(ctxPtr->end - ctxPtr->base) -
ctxPtr->dictLimit;
+ if (dictSize > 64 KB) dictSize = 64 KB;
+
+ LZ4_loadDictHC((LZ4_streamHC_t*)ctxPtr, (const char*)(ctxPtr->end) -
dictSize, (int)dictSize);
+ }
+
+ /* Check if blocks follow each other */
+ if ((const BYTE*)source != ctxPtr->end) LZ4HC_setExternalDict(ctxPtr,
(const BYTE*)source);
+
+ /* Check overlapping input/dictionary space */
+ {
+ const BYTE* sourceEnd = (const BYTE*) source + inputSize;
+ const BYTE* dictBegin = ctxPtr->dictBase + ctxPtr->lowLimit;
+ const BYTE* dictEnd = ctxPtr->dictBase + ctxPtr->dictLimit;
+ if ((sourceEnd > dictBegin) && ((BYTE*)source < dictEnd))
+ {
+ if (sourceEnd > dictEnd) sourceEnd = dictEnd;
+ ctxPtr->lowLimit = (U32)(sourceEnd - ctxPtr->dictBase);
+ if (ctxPtr->dictLimit - ctxPtr->lowLimit < 4) ctxPtr->lowLimit =
ctxPtr->dictLimit;
+ }
+ }
+
+ return LZ4HC_compress_generic (ctxPtr, source, dest, inputSize,
maxOutputSize, ctxPtr->compressionLevel, limit);
+}
+
+int LZ4_compressHC_continue (LZ4_streamHC_t* LZ4_streamHCPtr, const char*
source, char* dest, int inputSize)
+{
+ return LZ4_compressHC_continue_generic
((LZ4HC_Data_Structure*)LZ4_streamHCPtr, source, dest, inputSize, 0, noLimit);
+}
+
+int LZ4_compressHC_limitedOutput_continue (LZ4_streamHC_t* LZ4_streamHCPtr,
const char* source, char* dest, int inputSize, int maxOutputSize)
+{
+ return LZ4_compressHC_continue_generic
((LZ4HC_Data_Structure*)LZ4_streamHCPtr, source, dest, inputSize,
maxOutputSize, limitedOutput);
+}
+
+
+/* dictionary saving */
+
+int LZ4_saveDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, char* safeBuffer, int
dictSize)
+{
+ LZ4HC_Data_Structure* streamPtr = (LZ4HC_Data_Structure*)LZ4_streamHCPtr;
+ int prefixSize = (int)(streamPtr->end - (streamPtr->base +
streamPtr->dictLimit));
+ if (dictSize > 64 KB) dictSize = 64 KB;
+ if (dictSize < 4) dictSize = 0;
+ if (dictSize > prefixSize) dictSize = prefixSize;
+ memcpy(safeBuffer, streamPtr->end - dictSize, dictSize);
+ {
+ U32 endIndex = (U32)(streamPtr->end - streamPtr->base);
+ streamPtr->end = (const BYTE*)safeBuffer + dictSize;
+ streamPtr->base = streamPtr->end - endIndex;
+ streamPtr->dictLimit = endIndex - dictSize;
+ streamPtr->lowLimit = endIndex - dictSize;
+ if (streamPtr->nextToUpdate < streamPtr->dictLimit)
streamPtr->nextToUpdate = streamPtr->dictLimit;
+ }
+ return dictSize;
+}
+
+
+/***********************************
+ * Deprecated Functions
+ ***********************************/
+int LZ4_sizeofStreamStateHC(void) { return LZ4_STREAMHCSIZE; }
+
+int LZ4_resetStreamStateHC(void* state, const char* inputBuffer)
+{
+ if ((((size_t)state) & (sizeof(void*)-1)) != 0) return 1; /* Error :
pointer is not aligned for pointer (32 or 64 bits) */
+ LZ4HC_init((LZ4HC_Data_Structure*)state, (const BYTE*)inputBuffer);
+ return 0;
+}
+
+void* LZ4_createHC (const char* inputBuffer)
+{
+ void* hc4 = ALLOCATOR(1, sizeof(LZ4HC_Data_Structure));
+ LZ4HC_init ((LZ4HC_Data_Structure*)hc4, (const BYTE*)inputBuffer);
+ return hc4;
+}
+
+int LZ4_freeHC (void* LZ4HC_Data)
+{
+ FREEMEM(LZ4HC_Data);
+ return (0);
+}
+
+/*
+int LZ4_compressHC_continue (void* LZ4HC_Data, const char* source, char* dest,
int inputSize)
+{
+return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, 0, 0,
noLimit);
+}
int LZ4_compressHC_limitedOutput_continue (void* LZ4HC_Data, const char*
source, char* dest, int inputSize, int maxOutputSize)
{
- return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize,
maxOutputSize, 0, limitedOutput);
+return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize,
maxOutputSize, 0, limitedOutput);
+}
+*/
+
+int LZ4_compressHC2_continue (void* LZ4HC_Data, const char* source, char*
dest, int inputSize, int compressionLevel)
+{
+ return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize, 0,
compressionLevel, noLimit);
}
int LZ4_compressHC2_limitedOutput_continue (void* LZ4HC_Data, const char*
source, char* dest, int inputSize, int maxOutputSize, int compressionLevel)
{
return LZ4HC_compress_generic (LZ4HC_Data, source, dest, inputSize,
maxOutputSize, compressionLevel, limitedOutput);
}
+
+char* LZ4_slideInputBufferHC(void* LZ4HC_Data)
+{
+ LZ4HC_Data_Structure* hc4 = (LZ4HC_Data_Structure*)LZ4HC_Data;
+ int dictSize = LZ4_saveDictHC((LZ4_streamHC_t*)LZ4HC_Data,
(char*)(hc4->inputBuffer), 64 KB);
+ return (char*)(hc4->inputBuffer + dictSize);
+}
diff --git a/src/static_libs/lz4/lz4hc.h b/src/static_libs/lz4/lz4hc.h
index deb2394..4a05845 100644
--- a/src/static_libs/lz4/lz4hc.h
+++ b/src/static_libs/lz4/lz4hc.h
@@ -1,7 +1,7 @@
/*
LZ4 HC - High Compression Mode of LZ4
Header File
- Copyright (C) 2011-2014, Yann Collet.
+ Copyright (C) 2011-2015, Yann Collet.
BSD 2-Clause License (http://www.opensource.org/licenses/bsd-license.php)
Redistribution and use in source and binary forms, with or without
@@ -28,8 +28,8 @@
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
You can contact the author at :
- - LZ4 homepage : http://fastcompression.blogspot.com/p/lz4.html
- - LZ4 source repository : http://code.google.com/p/lz4/
+ - LZ4 source repository : https://github.com/Cyan4973/lz4
+ - LZ4 public forum : https://groups.google.com/forum/#!forum/lz4c
*/
#pragma once
@@ -74,12 +74,12 @@ int LZ4_compressHC2_limitedOutput (const char* source,
char* dest, int inputSize
*/
/* Note :
-Decompression functions are provided within LZ4 source code (see "lz4.h") (BSD
license)
+ Decompression functions are provided within LZ4 source code (see "lz4.h")
(BSD license)
*/
/**************************************
- Using an external allocation
+* Using an external allocation
**************************************/
int LZ4_sizeofStateHC(void);
int LZ4_compressHC_withStateHC (void* state, const char* source,
char* dest, int inputSize);
@@ -95,79 +95,86 @@ int LZ4_sizeofStateHC();
Note that tables must be aligned for pointer (32 or 64 bits), otherwise
compression will fail (return code 0).
-The allocated memory can be provided to the compressions functions using
'void* state' parameter.
+The allocated memory can be provided to the compression functions using 'void*
state' parameter.
LZ4_compress_withStateHC() and LZ4_compress_limitedOutput_withStateHC() are
equivalent to previously described functions.
-They just use the externally allocated memory area instead of allocating their
own (on stack, or on heap).
+They just use the externally allocated memory for state instead of allocating
their own (on stack, or on heap).
*/
-/**************************************
- Streaming Functions
-**************************************/
-/* Note : these streaming functions still follows the older model */
-void* LZ4_createHC (const char* inputBuffer);
-int LZ4_compressHC_continue (void* LZ4HC_Data, const char* source, char*
dest, int inputSize);
-int LZ4_compressHC_limitedOutput_continue (void* LZ4HC_Data, const char*
source, char* dest, int inputSize, int maxOutputSize);
-char* LZ4_slideInputBufferHC (void* LZ4HC_Data);
-int LZ4_freeHC (void* LZ4HC_Data);
-int LZ4_compressHC2_continue (void* LZ4HC_Data, const char* source, char*
dest, int inputSize, int compressionLevel);
-int LZ4_compressHC2_limitedOutput_continue (void* LZ4HC_Data, const char*
source, char* dest, int inputSize, int maxOutputSize, int compressionLevel);
+/*****************************
+* Includes
+*****************************/
+#include <stddef.h> /* size_t */
+
+/**************************************
+* Experimental Streaming Functions
+**************************************/
+#define LZ4_STREAMHCSIZE 262192
+#define LZ4_STREAMHCSIZE_SIZET (LZ4_STREAMHCSIZE / sizeof(size_t))
+typedef struct { size_t table[LZ4_STREAMHCSIZE_SIZET]; } LZ4_streamHC_t;
/*
-These functions allow the compression of dependent blocks, where each block
benefits from prior 64 KB within preceding blocks.
-In order to achieve this, it is necessary to start creating the LZ4HC Data
Structure, thanks to the function :
+LZ4_streamHC_t
+This structure allows static allocation of LZ4 HC streaming state.
+State must then be initialized using LZ4_resetStreamHC() before first use.
-void* LZ4_createHC (const char* inputBuffer);
-The result of the function is the (void*) pointer on the LZ4HC Data Structure.
-This pointer will be needed in all other functions.
-If the pointer returned is NULL, then the allocation has failed, and
compression must be aborted.
-The only parameter 'const char* inputBuffer' must, obviously, point at the
beginning of input buffer.
-The input buffer must be already allocated, and size at least 192KB.
-'inputBuffer' will also be the 'const char* source' of the first block.
-
-All blocks are expected to lay next to each other within the input buffer,
starting from 'inputBuffer'.
-To compress each block, use either LZ4_compressHC_continue() or
LZ4_compressHC_limitedOutput_continue().
-Their behavior are identical to LZ4_compressHC() or
LZ4_compressHC_limitedOutput(),
-but require the LZ4HC Data Structure as their first argument, and check that
each block starts right after the previous one.
-If next block does not begin immediately after the previous one, the
compression will fail (return 0).
-
-When it's no longer possible to lay the next block after the previous one (not
enough space left into input buffer), a call to :
-char* LZ4_slideInputBufferHC(void* LZ4HC_Data);
-must be performed. It will typically copy the latest 64KB of input at the
beginning of input buffer.
-Note that, for this function to work properly, minimum size of an input buffer
must be 192KB.
-==> The memory position where the next input data block must start is provided
as the result of the function.
-
-Compression can then resume, using LZ4_compressHC_continue() or
LZ4_compressHC_limitedOutput_continue(), as usual.
-
-When compression is completed, a call to LZ4_freeHC() will release the memory
used by the LZ4HC Data Structure.
+Static allocation should only be used with statically linked library.
+If you want to use LZ4 as a DLL, please use construction functions below,
which are more future-proof.
*/
-int LZ4_sizeofStreamStateHC(void);
-int LZ4_resetStreamStateHC(void* state, const char* inputBuffer);
+LZ4_streamHC_t* LZ4_createStreamHC(void);
+int LZ4_freeStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr);
/*
-These functions achieve the same result as :
-void* LZ4_createHC (const char* inputBuffer);
+These functions create and release memory for LZ4 HC streaming state.
+Newly created states are already initialized.
+Existing state space can be re-used anytime using LZ4_resetStreamHC().
+If you use LZ4 as a DLL, please use these functions instead of direct struct
allocation,
+to avoid size mismatch between different versions.
+*/
+
+void LZ4_resetStreamHC (LZ4_streamHC_t* LZ4_streamHCPtr, int compressionLevel);
+int LZ4_loadDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, const char* dictionary,
int dictSize);
-They are provided here to allow the user program to allocate memory using its
own routines.
+int LZ4_compressHC_continue (LZ4_streamHC_t* LZ4_streamHCPtr, const char*
source, char* dest, int inputSize);
+int LZ4_compressHC_limitedOutput_continue (LZ4_streamHC_t* LZ4_streamHCPtr,
const char* source, char* dest, int inputSize, int maxOutputSize);
-To know how much space must be allocated, use LZ4_sizeofStreamStateHC();
-Note also that space must be aligned for pointers (32 or 64 bits).
+int LZ4_saveDictHC (LZ4_streamHC_t* LZ4_streamHCPtr, char* safeBuffer, int
maxDictSize);
-Once space is allocated, you must initialize it using :
LZ4_resetStreamStateHC(void* state, const char* inputBuffer);
-void* state is a pointer to the space allocated.
-It must be aligned for pointers (32 or 64 bits), and be large enough.
-The parameter 'const char* inputBuffer' must, obviously, point at the
beginning of input buffer.
-The input buffer must be already allocated, and size at least 192KB.
-'inputBuffer' will also be the 'const char* source' of the first block.
+/*
+These functions compress data in successive blocks of any size, using previous
blocks as dictionary.
+One key assumption is that each previous block will remain read-accessible
while compressing next block.
+
+Before starting compression, state must be properly initialized, using
LZ4_resetStreamHC().
+A first "fictional block" can then be designated as initial dictionary, using
LZ4_loadDictHC() (Optional).
-The same space can be re-used multiple times, just by initializing it each
time with LZ4_resetStreamState().
-return value of LZ4_resetStreamStateHC() must be 0 is OK.
-Any other value means there was an error (typically, state is not aligned for
pointers (32 or 64 bits)).
+Then, use LZ4_compressHC_continue() or LZ4_compressHC_limitedOutput_continue()
to compress each successive block.
+They work like usual LZ4_compressHC() or LZ4_compressHC_limitedOutput(), but
use previous memory blocks to improve compression.
+Previous memory blocks (including initial dictionary when present) must remain
accessible and unmodified during compression.
+
+If, for any reason, previous data block can't be preserved in memory during
next compression block,
+you must save it to a safer memory space,
+using LZ4_saveDictHC().
*/
+
+/**************************************
+ * Deprecated Streaming Functions
+ * ************************************/
+/* Note : these streaming functions follows the older model, and should no
longer be used */
+void* LZ4_createHC (const char* inputBuffer);
+char* LZ4_slideInputBufferHC (void* LZ4HC_Data);
+int LZ4_freeHC (void* LZ4HC_Data);
+
+int LZ4_compressHC2_continue (void* LZ4HC_Data, const char* source, char*
dest, int inputSize, int compressionLevel);
+int LZ4_compressHC2_limitedOutput_continue (void* LZ4HC_Data, const char*
source, char* dest, int inputSize, int maxOutputSize, int compressionLevel);
+
+int LZ4_sizeofStreamStateHC(void);
+int LZ4_resetStreamStateHC(void* state, const char* inputBuffer);
+
+
#if defined (__cplusplus)
}
#endif
--
