[
https://issues.apache.org/jira/browse/TRAFODION-2475?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15858808#comment-15858808
]
ASF GitHub Bot commented on TRAFODION-2475:
-------------------------------------------
Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/949#discussion_r100210604
--- Diff: core/sql/src/main/java/org/trafodion/sql/HBaseClient.java ---
@@ -294,15 +296,27 @@ private ChangeFlags setDescriptors(Object[]
tableOptions,
break ;
case HBASE_COMPRESSION:
if (tableOption.equalsIgnoreCase("GZ"))
+ { // throws IOException
+ CompressionTest.testCompression(Algorithm.GZ);
colDesc.setCompressionType(Algorithm.GZ);
+ }
else if (tableOption.equalsIgnoreCase("LZ4"))
+ { // throws IOException
+ CompressionTest.testCompression(Algorithm.LZ4);
colDesc.setCompressionType(Algorithm.LZ4);
+ }
else if (tableOption.equalsIgnoreCase("LZO"))
+ { // throws IOException
+ CompressionTest.testCompression(Algorithm.LZO);
colDesc.setCompressionType(Algorithm.LZO);
+ }
else if (tableOption.equalsIgnoreCase("NONE"))
colDesc.setCompressionType(Algorithm.NONE);
else if (tableOption.equalsIgnoreCase("SNAPPY"))
+ { // throws IOException
+ CompressionTest.testCompression(Algorithm.SNAPPY);
colDesc.setCompressionType(Algorithm.SNAPPY);
+ }
--- End diff --
Good catch Dave. I was confident that invalid compression options like
"SNAPY" would be caught earlier in the parse/bind stage. Now I see that
validation of input arguments is not thorough. I will add a commit to this PR
by tomorrow with that change. Much validation can be done on the C++ side
before we call the create()/alter() Java function. For an allowed compression
setting usable by HBase, the only verification I know is the Java one provided
by HBase itself. Therefore this validation could be done once in Java side and
results stored in C++ side or we can do as this PR does, validate and raise
error in Java side when necessary. The method provided by HBase uses static
variables to store results of previous validations, so we will not be fully
validating the same compression algorithm twice in a given session.
This is how the error message looks like now.
create table test5 (a int not null primary key, b int) hbase_options
(compression = 'SNAPPY') salt using 2 partitions ;
*** ERROR[8448] Unable to access Hbase interface. Call to
ExpHbaseInterface::create() returned error HBASE_CREATE_ERROR(701). Cause:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.RuntimeException:
native snappy library not available: this version of libhadoop was built
without snappy support.
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:102)
org.trafodion.sql.HBaseClient.setDescriptors(HBaseClient.java:317)
org.trafodion.sql.HBaseClient.createk(HBaseClient.java:490) Caused by
java.lang.RuntimeException: native snappy library not available: this
version of libhadoop was built without snappy support.
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:64)
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:133)
> Create table/index or alter does not work with unavailable compression options
> ------------------------------------------------------------------------------
>
> Key: TRAFODION-2475
> URL: https://issues.apache.org/jira/browse/TRAFODION-2475
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-general
> Affects Versions: any
> Reporter: Suresh Subbiah
> Assignee: Suresh Subbiah
> Fix For: 2.0-incubating
>
>
> Create table/index option with HBASE_OPTIONS (COMPRESSION = '<val>') does not
> work well when <val> is a compression option that is not installed or enabled
> for HBase on the cluster. The 5 allowed values are GZ, LZO, LZ4, NONE,
> SNAPPY. If one of the 4 allowed compression settings is either not installed
> or available to HBase, then while creating a table it can either
> a) hang or take a long time to error out OR
> b) succeed, but have problems while inserting rows later.
> Non-existent compression library should be detected early and CREATE
> statement should error out. Same is true for ALTER table.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
