Github user sureshsubbiah commented on a diff in the pull request:
https://github.com/apache/incubator-trafodion/pull/949#discussion_r100210604
--- Diff: core/sql/src/main/java/org/trafodion/sql/HBaseClient.java ---
@@ -294,15 +296,27 @@ private ChangeFlags setDescriptors(Object[]
tableOptions,
break ;
case HBASE_COMPRESSION:
if (tableOption.equalsIgnoreCase("GZ"))
+ { // throws IOException
+ CompressionTest.testCompression(Algorithm.GZ);
colDesc.setCompressionType(Algorithm.GZ);
+ }
else if (tableOption.equalsIgnoreCase("LZ4"))
+ { // throws IOException
+ CompressionTest.testCompression(Algorithm.LZ4);
colDesc.setCompressionType(Algorithm.LZ4);
+ }
else if (tableOption.equalsIgnoreCase("LZO"))
+ { // throws IOException
+ CompressionTest.testCompression(Algorithm.LZO);
colDesc.setCompressionType(Algorithm.LZO);
+ }
else if (tableOption.equalsIgnoreCase("NONE"))
colDesc.setCompressionType(Algorithm.NONE);
else if (tableOption.equalsIgnoreCase("SNAPPY"))
+ { // throws IOException
+ CompressionTest.testCompression(Algorithm.SNAPPY);
colDesc.setCompressionType(Algorithm.SNAPPY);
+ }
--- End diff --
Good catch Dave. I was confident that invalid compression options like
"SNAPY" would be caught earlier in the parse/bind stage. Now I see that
validation of input arguments is not thorough. I will add a commit to this PR
by tomorrow with that change. Much validation can be done on the C++ side
before we call the create()/alter() Java function. For an allowed compression
setting usable by HBase, the only verification I know is the Java one provided
by HBase itself. Therefore this validation could be done once in Java side and
results stored in C++ side or we can do as this PR does, validate and raise
error in Java side when necessary. The method provided by HBase uses static
variables to store results of previous validations, so we will not be fully
validating the same compression algorithm twice in a given session.
This is how the error message looks like now.
create table test5 (a int not null primary key, b int) hbase_options
(compression = 'SNAPPY') salt using 2 partitions ;
*** ERROR[8448] Unable to access Hbase interface. Call to
ExpHbaseInterface::create() returned error HBASE_CREATE_ERROR(701). Cause:
org.apache.hadoop.hbase.DoNotRetryIOException: java.lang.RuntimeException:
native snappy library not available: this version of libhadoop was built
without snappy support.
org.apache.hadoop.hbase.util.CompressionTest.testCompression(CompressionTest.java:102)
org.trafodion.sql.HBaseClient.setDescriptors(HBaseClient.java:317)
org.trafodion.sql.HBaseClient.createk(HBaseClient.java:490) Caused by
java.lang.RuntimeException: native snappy library not available: this
version of libhadoop was built without snappy support.
org.apache.hadoop.io.compress.SnappyCodec.checkNativeCodeLoaded(SnappyCodec.java:64)
org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:133)
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
