?????? Using snappy compresscodec in hive
Hi Gopal,
Thanks for your reply! One more question, does the effect of using pure-java
version is the same as that of using SnappyCodec? Or, in other words, is there
any difference between these two methods, about the compression result and
effect?
Looking forward to your reply and help.
Best,
Zhefu Peng
-- --
??: "Gopal Vijayaraghavan";
: 2018??7??24??(??) 10:53
??: "user@hive.apache.org";
????: Re: Using snappy compresscodec in hive
> "TBLPROPERTIES ("orc.compress"="Snappy"); "
That doesn't use the Hadoop SnappyCodec, but uses a pure-java version (which is
slower, but always works).
The Hadoop snappyCodec needs libsnappy installed on all hosts.
Cheers,
Gopal
?????? Using snappy compresscodec in hive
Hi Gopal,
Thanks for your reply! One more question, does the effect of using pure-java
version is the same as that of using SnappyCodec? Or, in other words, is there
any difference between these two methods, about the compression result and
effect?
Looking forward to your reply and help.
Best,
Zhefu Peng
-- --
??: "Gopal Vijayaraghavan";
: 2018??7??24??(??) 10:53
??: "user@hive.apache.org";
????: Re: Using snappy compresscodec in hive
> "TBLPROPERTIES ("orc.compress"="Snappy"); "
That doesn't use the Hadoop SnappyCodec, but uses a pure-java version (which is
slower, but always works).
The Hadoop snappyCodec needs libsnappy installed on all hosts.
Cheers,
Gopal
Re: Using snappy compresscodec in hive
> "TBLPROPERTIES ("orc.compress"="Snappy"); "
That doesn't use the Hadoop SnappyCodec, but uses a pure-java version (which is
slower, but always works).
The Hadoop snappyCodec needs libsnappy installed on all hosts.
Cheers,
Gopal
Using snappy compresscodec in hive
Hi,
Here is a confusion I encountered these days: I don't install or build snappy
on my hadoop cluster, but when I tested and compared about the compression
ratio of Parquet and ORC storage format. During the test, I can set the way of
compression for two storage format, for example, using "TBLPROPERTIES
("orc.compress"="Snappy"); " or "set parquet.compression=snappy;", both these
commands would work. However, when I just want to compress the textfile format
with snappy compression, it says that "can not find or access the snappy
library".
I wonder why this situation happen, and, I really doubt that whether the ORC or
Parquet file using "Snappy" compression. But, the storage really becomes
smaller, and diff from the "gzip" or "zlib" compression.
Looking forward to your reply and help.
Best,
Zhefu Peng
Using snappy compresscodec in hive
Hi,
Here is a confusion I encounter these days: I don't install or build snappy on
my hadoop cluster, but when I tested and compared about the compression ratio
of Parquet and ORC storage format. During the test, I can set the way of
compression for two storage format, for example, using "TBLPROPERTIES
("orc.compress"="Snappy"); " or "set parquet.compression=snappy;", both these
commands would work. However, when I just want to compress the textfile format
with snappy compression, it says that "can not find or access the snappy
library".
I wonder why this situation happen, and, I really doubt that whether the ORC or
Parquet file using "Snappy" compression. But, the storage really becomes
smaller, and diff from the "gzip" or "zlib" compression.
Looking forward to your reply and help.
Best,
Zhefu Peng
Using snappy compresscodec in hive
Hi,
Here is a confusion I encounter these days: I don't install or build snappy on
my hadoop cluster, but when I tested and compared about the compression ratio
of Parquet and ORC storage format. During the test, I can set the way of
compression for two storage format, for example, using "TBLPROPERTIES
("orc.compress"="Snappy"); " or "set parquet.compression=snappy;", both these
commands would work. However, when I just want to compress the textfile format
with snappy compression, it says that "can not find or access the snappy
library".
I wonder why this situation happen, and, I really doubt that whether the ORC or
Parquet file using "Snappy" compression. But, the storage really becomes
smaller, and diff from the "gzip" or "zlib" compression.
Looking forward to your reply and help.
Best,
Zhefu Peng
