Yup, that property can be changed on the fly, and future compactions
will use the new codec (all of the queued compactions will see it). It
will not actively seek out and re-write files which are not compressed
with the current codec.
And, yes, depending on the distro of Hadoop you're using, there might be
some extra bits to install. I believe that your normal Hadoop-2.x.y
release from Apache will have the native libs available for you and
might just to install libsnappy via your OS's package manager.
On 5/6/14, 11:52 AM, David Medinets wrote:
Can that property be changed on the fly? And Snappy needs to be
installed throughout the cluster before making the configuration change?
On Tue, May 6, 2014 at 12:43 AM, Josh Elser <[email protected]
<mailto:[email protected]>> wrote:
Depending on the CPU/IO ratio for your system, switching to a
different compression codec might help. Snappy tends to be a bit
quicker writing out data as opposed to gzip at the cost of being
larger on disk. The increase in final size on disk might be
prohibitive depending on your requirements though.
I forget the table property off hand, but, if you haven't changed
this already, it will be the property with a defauly value of 'gz' :)
On May 5, 2014 6:43 PM, "Dickson, Matt MR"
<[email protected] <mailto:[email protected]>>
wrote:
__
*UNOFFICIAL*
I'm trying to compact a table and have had a queue of 60,000
compactions with only 800 running at a time. This has now been
running for 4 days and only decreased the queue by 8,000.
I've increased tserver.compaction.major.concurrent.max=12 and
stopped all ingest but not seen a change in progress. Are there
other accumulo settings I can alter to improve this? I also saw
tserver.compaction.major.thread.files.open.max=10 should this be
increased?
Thanks in advance,
Matt
