[
https://issues.apache.org/jira/browse/TRAFODION-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15926409#comment-15926409
]
ASF GitHub Bot commented on TRAFODION-2535:
-------------------------------------------
Github user asfgit closed the pull request at:
https://github.com/apache/incubator-trafodion/pull/1008
> Create index succeeds but resulting index is empty, if dop > 32
> ---------------------------------------------------------------
>
> Key: TRAFODION-2535
> URL: https://issues.apache.org/jira/browse/TRAFODION-2535
> Project: Apache Trafodion
> Issue Type: Bug
> Components: sql-exe
> Affects Versions: any
> Reporter: Suresh Subbiah
> Assignee: Suresh Subbiah
> Fix For: 2.2-incubating
>
>
> If more than 32 HFiles are created for a single region of an index or table
> during bulkload, then LOAD COMPLETE step will fail. However if snapshot
> recovery is not enabled the error is silently ignored and LOAD (or CREATE
> INDEX which uses LOAD) statement appears to succeed. But the table or index
> will be empty leading to wrong results for subsequent queries.
> To reproduce
> create table test1 (a int not null primary key, b int not null) salt using 40
> partitions;
> insert into test1 values
> (1,1),(2,2),(3,3),(4,4),(5,5),(6,6),(7,7),(8,8),(9,9),(10,10) ;
> upsert using load into test1 select a+10, b+10 from test1;
> upsert using load into test1 select a+20, b+20 from test1;
> upsert using load into test1 select a+40, b+40 from test1;
> upsert using load into test1 select a+80, b+80 from test1;
> upsert using load into test1 select a+160, b+160 from test1;
> upsert using load into test1 select a+320, b+320 from test1;
> upsert using load into test1 select a+640, b+640 from test1;
> upsert using load into test1 select a+1280, b+1280 from test1;
> upsert using load into test1 select a+2560, b+2560 from test1;
> upsert using load into test1 select a+5120, b+5120 from test1;
> upsert using load into test1 select a+10240, b+10240 from test1;
> upsert using load into test1 select a+20480, b+20480 from test1;
> upsert using load into test1 select a+40960, b+40960 from test1;
> upsert using load into test1 select a+81920, b+81920 from test1;
> exit ;
> from HBase shell flush 'TRAFODION.SCH.TEST1'
> -- new sqlci
> cqd HBASE_MIN_BYTES_PER_ESP_PARTITION '10' ;
> cqd parallel_num_esps '40' ;
> create index ix1 on test1(b) ;
> set parserflags 1 ;
> select * from table(index_table ix1) ;
> -- will have 0 rows when this problem is present.
> in logs directory (cdl), trafodion.hdfs.log will have a similar message.
> 2017-03-14 01:28:29,242 ,ERROR ,mapreduce.LoadIncrementalHFiles ,Trying to
> load more than 32 hfiles to family #1 of region with start key
> With the fix we get this message in sqlci and index is not created.
> >>create index ix1 on test1(b) ;
> *** ERROR[8448] Unable to access Hbase interface. Call to
> ExpHbaseInterface::doBulkLoad returned error HBASE_DOBULK_LOAD_ERROR(-715).
> Cause: java.io.IOException: Trying to load more than 32 hfiles to one family
> of one region
> org.apache.hadoop.hbase.mapreduce.LoadIncrementalHFiles.doBulkLoad(LoadIncrementalHFiles.java:377)
> org.trafodion.sql.HBulkLoadClient.doSnapshotNBulkLoad(HBulkLoadClient.java:381)
> org.trafodion.sql.HBulkLoadClient.doBulkLoad(HBulkLoadClient.java:454).
> *** ERROR[1081] Loading of index TRAFODION.SCH.IX1 failed unexpectedly.
> *** ERROR[8811] Trying to close a statement that is either not in the open
> state or has not reached EOF.
> --- SQL operation failed with errors.
> >>showddl test1 ;
> CREATE TABLE TRAFODION.SCH.TEST1
> (
> A INT NO DEFAULT NOT NULL NOT DROPPABLE NOT
> SERIALIZED
> , B INT NO DEFAULT NOT NULL NOT DROPPABLE NOT
> SERIALIZED
> , PRIMARY KEY (A ASC)
> )
> SALT USING 40 PARTITIONS
> ATTRIBUTES ALIGNED FORMAT
> ;
> -- GRANT SELECT, INSERT, DELETE, UPDATE, REFERENCES ON TRAFODION.SCH.TEST1 TO
> DB__ROOT WITH GRANT OPTION;
> --- SQL operation complete.
> >>exit ;
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
