[
https://issues.apache.org/jira/browse/ATLAS-4941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pinal Shah updated ATLAS-4941:
------------------------------
Description:
*Issue:* After compression type is set to SNAPPY, When Atlas is build locally,
it fails giving below console exception
*Exception:*
{code:java}
java.lang.RuntimeException: org.apache.atlas.AtlasException:
org.apache.hadoop.hbase.DoNotRetryIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z Set
hbase.table.sanity.checks to false at conf or table descriptor if you want to
bypass sanity checks
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:318)
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:33
Caused by: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method)
at
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
at
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136)
{code}
*Workaround:*
Ensure Snappy compression libraries are available.
If not already installed, download and configure the Snappy library on your
system. (below was tried on Linux OS)
1. For hbase 2.3.3 download
[https://downloads.apache.org/hadoop/common/hadoop-3.2.4/hadoop-3.2.4.tar.gz]
2. mv hadoop-3.2.4.tar.gz /usr/local/
3. tar -zxvf hadoop-3.2.4.tar.gz
4. sudo apt install libsnappy-dev
5. add below configs in "hbase/conf/hbase-env.sh"
export HADOOP_HOME=/usr/local/hadoop-3.2.4
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native
*Aproach:*
This patch makes GZ as a default compression algorithm, SNAPPY can be set using
below property in atlas-application.properties
atlas.graph.storage.hbase.compression-algorithm=SNAPPY
was:
**Issue:** After compression type is set to SNAPPY, When Atlas is build
locally, it fails giving below console exception
**Exception:**
{code:java}
java.lang.RuntimeException: org.apache.atlas.AtlasException:
org.apache.hadoop.hbase.DoNotRetryIOException:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z Set
hbase.table.sanity.checks to false at conf or table descriptor if you want to
bypass sanity checks
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:318)
at
org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:33
Caused by: java.lang.UnsatisfiedLinkError:
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at
org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method)
at
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
at
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136)
{code}
**Workaround:**
Ensure Snappy compression libraries are available.
If not already installed, download and configure the Snappy library on your
system. (below was tried on Linux OS)
1. For hbase 2.3.3 download
https://downloads.apache.org/hadoop/common/hadoop-3.2.4/hadoop-3.2.4.tar.gz
2. mv hadoop-3.2.4.tar.gz /usr/local/
3. tar -zxvf hadoop-3.2.4.tar.gz
4. sudo apt install libsnappy-dev
5. add below configs in "hbase/conf/hbase-env.sh"
export HADOOP_HOME=/usr/local/hadoop-3.2.4
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native
**Aproach:**
This patch makes GZ as a default compression algorithm, SNAPPY can be set using
below property in atlas-application.properties
atlas.graph.storage.hbase.compression-algorithm=SNAPPY
> Atlas fails to startup locally after compression type has changed to SNAPPY
> ----------------------------------------------------------------------------
>
> Key: ATLAS-4941
> URL: https://issues.apache.org/jira/browse/ATLAS-4941
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
> Reporter: Pinal Shah
> Assignee: Pinal Shah
> Priority: Major
> Fix For: 3.0.0, 2.4.0
>
>
> *Issue:* After compression type is set to SNAPPY, When Atlas is build
> locally, it fails giving below console exception
> *Exception:*
> {code:java}
> java.lang.RuntimeException: org.apache.atlas.AtlasException:
> org.apache.hadoop.hbase.DoNotRetryIOException:
> org.apache.hadoop.hbase.DoNotRetryIOException:
> java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z Set
> hbase.table.sanity.checks to false at conf or table descriptor if you want to
> bypass sanity checks
> at
> org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:318)
> at
> org.apache.hadoop.hbase.util.TableDescriptorChecker.warnOrThrowExceptionForFailure(TableDescriptorChecker.java:33
>
> Caused by: java.lang.UnsatisfiedLinkError:
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy()Z at
> org.apache.hadoop.util.NativeCodeLoader.buildSupportsSnappy(Native Method)
> at
> org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:63)
> at
> org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:136)
> {code}
> *Workaround:*
> Ensure Snappy compression libraries are available.
> If not already installed, download and configure the Snappy library on your
> system. (below was tried on Linux OS)
> 1. For hbase 2.3.3 download
> [https://downloads.apache.org/hadoop/common/hadoop-3.2.4/hadoop-3.2.4.tar.gz]
> 2. mv hadoop-3.2.4.tar.gz /usr/local/
> 3. tar -zxvf hadoop-3.2.4.tar.gz
> 4. sudo apt install libsnappy-dev
> 5. add below configs in "hbase/conf/hbase-env.sh"
> export HADOOP_HOME=/usr/local/hadoop-3.2.4
> export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HADOOP_HOME/lib/native
> *Aproach:*
> This patch makes GZ as a default compression algorithm, SNAPPY can be set
> using below property in atlas-application.properties
> atlas.graph.storage.hbase.compression-algorithm=SNAPPY
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
