jang jin kyung created PHOENIX-4985:
---------------------------------------
Summary: IndexBuildingFailureException
Key: PHOENIX-4985
URL: https://issues.apache.org/jira/browse/PHOENIX-4985
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.7.0
Environment: 3 Node HBase cluster running version 1.2.0-cdh5.15
Phoenix version 4.7.0
OS: CentOS release 6.8 (Final)
MEMORY : 32G
Reporter: jang jin kyung
Attachments: hbase-site.xml
I started to experience an issue trying to insert some data into our HBase
cluster using phoenix. This seems to have start occurring after I added a
secondary index on one of my columns.
h6. CREATE TABLE TEST_3 (
SEQ BIGINT,
LOG_DATE VARCHAR,
ID VARCHAR,
IP VARCHAR,
MAC VARCHAR,
HDD VARCHAR,
UUID VARCHAR,
VALUE1 VARCHAR,
VALUE2 VARCHAR, .....
h6. VALUE100 VARCHAR,
CONSTRAINT pk PRIMARY KEY (SEQ) ) ;
h6. CREATE LOCAL INDEX TEST3_IDX1 ON TEST_3 ( ID) ;
h6. CREATE LOCAL INDEX TEST3_IDX2 ON TEST_3 ( IP) ;
h6. CREATE LOCAL INDEX TEST3_IDX3 ON TEST_3 ( MAC) ;
h6. CREATE LOCAL INDEX TEST3_IDX4 ON TEST_3 ( UUID) ;
h6. CREATE LOCAL INDEX TEST3_IDX5 ON TEST_3 ( LOG_DATE) ;
Reads a file with 60000000 rows and executes executeBatch () and commit () with
1000 rows.
I got an error during the process.
Caused by: org.apache.phoenix.execute.CommitException:
org.apache.hadoop.hbase.client.RetriesExhaustedWithDetailsException: Failed
1000 actions:
org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed to
build index for unexpected reason!
at
org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:180)
at org.apache.phoenix.hbase.index.Indexer.preBatchMutate(Indexer.java:205)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$35.call(RegionCoprocessorHost.java:1007)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1749)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1705)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:1003)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3173)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2944)
at org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2886)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:765)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:716)
at
org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2146)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:33656)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2191)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:183)
at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:163)
Caused by: java.util.concurrent.RejectedExecutionException: Executor already
shutdown
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.startTask(MoreExecutors.java:327)
at
com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:251)
at
com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:56)
at
org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submit(BaseTaskRunner.java:58)
at
org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submitUninterruptible(BaseTaskRunner.java:99)
at
org.apache.phoenix.hbase.index.builder.IndexBuildManager.getIndexUpdate(IndexBuildManager.java:143)
at
org.apache.phoenix.hbase.index.Indexer.preBatchMutateWithExceptions(Indexer.java:273)
at org.apache.phoenix.hbase.index.Indexer.preBatchMutate(Indexer.java:202)
... 16 more
: 1000 times,
at org.apache.phoenix.execute.MutationState.send(MutationState.java:984)
at org.apache.phoenix.execute.MutationState.send(MutationState.java:1317)
at org.apache.phoenix.execute.MutationState.commit(MutationState.java:1149)
at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:520)
at org.apache.phoenix.jdbc.PhoenixConnection$3.call(PhoenixConnection.java:517)
at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
at org.apache.phoenix.jdbc.PhoenixConnection.commit(PhoenixConnection.java:517)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
