[jira] [Commented] (PHOENIX-4591) Possible IndexOutOfBoundsException with delete query on bigger table

2018-02-12 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16360562#comment-16360562
 ] 

Hudson commented on PHOENIX-4591:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1933 (See 
[https://builds.apache.org/job/Phoenix-master/1933/])
PHOENIX-4591 Possible IndexOutOfBoundsException with delete query on 
(rajeshbabu: rev a6bf7350d311e42402d330b07cb412b422eeaae9)
* (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/DeleteIT.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/compile/DeleteCompiler.java


> Possible IndexOutOfBoundsException with delete query on bigger table
> 
>
> Key: PHOENIX-4591
> URL: https://issues.apache.org/jira/browse/PHOENIX-4591
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 5.0.0, 4.14.0
>
> Attachments: PHOENIX-4591.patch
>
>
> Here is the schema and inserted around 5million records.
> {noformat}
> 0: jdbc:phoenix:localhost> CREATE TABLE IF NOT EXISTS testTable ( ID BIGINT 
> PRIMARY KEY , FirstName VARCHAR(30) , SecondName VARCHAR(30) , City 
> VARCHAR(30) );
> No rows affected (2.283 seconds)
> 0: jdbc:phoenix:localhost> CREATE INDEX secondary_index ON testTable 
> (FirstName);
> No rows affected (7.275 seconds)
> {noformat}
> Then delete from the table is throwing java.lang.IndexOutOfBoundsException
> {noformat}
> 0: jdbc:phoenix:localhost> delete from testtable;
> Error: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 
> (state=08000,code=101)
> org.apache.phoenix.exception.PhoenixIOException: 
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at 
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:122)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:970)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:912)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.getIterators(RoundRobinResultIterator.java:176)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.next(RoundRobinResultIterator.java:91)
>   at 
> org.apache.phoenix.compile.DeleteCompiler$5.execute(DeleteCompiler.java:796)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:394)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:377)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:376)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:364)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1738)
>   at sqlline.Commands.execute(Commands.java:822)
>   at sqlline.Commands.sql(Commands.java:732)
>   at sqlline.SqlLine.dispatch(SqlLine.java:813)
>   at sqlline.SqlLine.begin(SqlLine.java:686)
>   at sqlline.SqlLine.start(SqlLine.java:398)
>   at sqlline.SqlLine.main(SqlLine.java:291)
> Caused by: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:206)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:965)
>   ... 16 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>   at java.util.ArrayList.get(ArrayList.java:429)
>   at 
> org.apache.phoenix.compile.DeleteCompiler.deleteRows(DeleteCompiler.java:225)
>   at 
> org.apache.phoenix.compile.DeleteCompiler.access$000(DeleteCompiler.java:98)
>   at 
> org.apache.phoenix.compile.DeleteCompiler$DeletingParallelIteratorFactory.mutate(DeleteCompiler.java:276)
>   at 
> org.apache.phoenix.compile.MutatingParallelIteratorFactory.newIterator(MutatingParallelIteratorFactory.java:59)
>   at 
> org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:121)
>   at 
> org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:113)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run(JobManager.java:183)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA

[jira] [Commented] (PHOENIX-4591) Possible IndexOutOfBoundsException with delete query on bigger table

2018-02-09 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358867#comment-16358867
 ] 

James Taylor commented on PHOENIX-4591:
---

+1. Thanks, [~rajeshbabu]! Nice work.

> Possible IndexOutOfBoundsException with delete query on bigger table
> 
>
> Key: PHOENIX-4591
> URL: https://issues.apache.org/jira/browse/PHOENIX-4591
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 5.0.0, 4.14.0
>
> Attachments: PHOENIX-4591.patch
>
>
> Here is the schema and inserted around 5million records.
> {noformat}
> 0: jdbc:phoenix:localhost> CREATE TABLE IF NOT EXISTS testTable ( ID BIGINT 
> PRIMARY KEY , FirstName VARCHAR(30) , SecondName VARCHAR(30) , City 
> VARCHAR(30) );
> No rows affected (2.283 seconds)
> 0: jdbc:phoenix:localhost> CREATE INDEX secondary_index ON testTable 
> (FirstName);
> No rows affected (7.275 seconds)
> {noformat}
> Then delete from the table is throwing java.lang.IndexOutOfBoundsException
> {noformat}
> 0: jdbc:phoenix:localhost> delete from testtable;
> Error: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 
> (state=08000,code=101)
> org.apache.phoenix.exception.PhoenixIOException: 
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at 
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:122)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:970)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:912)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.getIterators(RoundRobinResultIterator.java:176)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.next(RoundRobinResultIterator.java:91)
>   at 
> org.apache.phoenix.compile.DeleteCompiler$5.execute(DeleteCompiler.java:796)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:394)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:377)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:376)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:364)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1738)
>   at sqlline.Commands.execute(Commands.java:822)
>   at sqlline.Commands.sql(Commands.java:732)
>   at sqlline.SqlLine.dispatch(SqlLine.java:813)
>   at sqlline.SqlLine.begin(SqlLine.java:686)
>   at sqlline.SqlLine.start(SqlLine.java:398)
>   at sqlline.SqlLine.main(SqlLine.java:291)
> Caused by: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:206)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:965)
>   ... 16 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>   at java.util.ArrayList.get(ArrayList.java:429)
>   at 
> org.apache.phoenix.compile.DeleteCompiler.deleteRows(DeleteCompiler.java:225)
>   at 
> org.apache.phoenix.compile.DeleteCompiler.access$000(DeleteCompiler.java:98)
>   at 
> org.apache.phoenix.compile.DeleteCompiler$DeletingParallelIteratorFactory.mutate(DeleteCompiler.java:276)
>   at 
> org.apache.phoenix.compile.MutatingParallelIteratorFactory.newIterator(MutatingParallelIteratorFactory.java:59)
>   at 
> org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:121)
>   at 
> org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:113)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run(JobManager.java:183)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4591) Possible IndexOutOfBoundsException with delete query on bigger table

2018-02-09 Thread Rajeshbabu Chintaguntla (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358413#comment-16358413
 ] 

Rajeshbabu Chintaguntla commented on PHOENIX-4591:
--

[~jamestaylor] Please review.
This is applicable to for tables with immutable indexes only as deletes on 
mutable and local indexes should be handled at server. PHOENIX-4531 fixed the 
issue to avoiding client side deletes on mutable and local indexes which is not 
present in 5.x branch. Need to sync 5.x with master.


> Possible IndexOutOfBoundsException with delete query on bigger table
> 
>
> Key: PHOENIX-4591
> URL: https://issues.apache.org/jira/browse/PHOENIX-4591
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 5.0.0, 4.14.0
>
> Attachments: PHOENIX-4591.patch
>
>
> Here is the schema and inserted around 5million records.
> {noformat}
> 0: jdbc:phoenix:localhost> CREATE TABLE IF NOT EXISTS testTable ( ID BIGINT 
> PRIMARY KEY , FirstName VARCHAR(30) , SecondName VARCHAR(30) , City 
> VARCHAR(30) );
> No rows affected (2.283 seconds)
> 0: jdbc:phoenix:localhost> CREATE INDEX secondary_index ON testTable 
> (FirstName);
> No rows affected (7.275 seconds)
> {noformat}
> Then delete from the table is throwing java.lang.IndexOutOfBoundsException
> {noformat}
> 0: jdbc:phoenix:localhost> delete from testtable;
> Error: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 
> (state=08000,code=101)
> org.apache.phoenix.exception.PhoenixIOException: 
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at 
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:122)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:970)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:912)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.getIterators(RoundRobinResultIterator.java:176)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.next(RoundRobinResultIterator.java:91)
>   at 
> org.apache.phoenix.compile.DeleteCompiler$5.execute(DeleteCompiler.java:796)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:394)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:377)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:376)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:364)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1738)
>   at sqlline.Commands.execute(Commands.java:822)
>   at sqlline.Commands.sql(Commands.java:732)
>   at sqlline.SqlLine.dispatch(SqlLine.java:813)
>   at sqlline.SqlLine.begin(SqlLine.java:686)
>   at sqlline.SqlLine.start(SqlLine.java:398)
>   at sqlline.SqlLine.main(SqlLine.java:291)
> Caused by: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:206)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:965)
>   ... 16 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>   at java.util.ArrayList.get(ArrayList.java:429)
>   at 
> org.apache.phoenix.compile.DeleteCompiler.deleteRows(DeleteCompiler.java:225)
>   at 
> org.apache.phoenix.compile.DeleteCompiler.access$000(DeleteCompiler.java:98)
>   at 
> org.apache.phoenix.compile.DeleteCompiler$DeletingParallelIteratorFactory.mutate(DeleteCompiler.java:276)
>   at 
> org.apache.phoenix.compile.MutatingParallelIteratorFactory.newIterator(MutatingParallelIteratorFactory.java:59)
>   at 
> org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:121)
>   at 
> org.apache.phoenix.iterate.ParallelIterators$1.call(ParallelIterators.java:113)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> org.apache.phoenix.job.JobManager$InstrumentedJobFutureTask.run(JobManager.java:183)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
>   at java.lang.Thread.run(Thread.java:745)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (PHOENIX-4591) Possible IndexOutOfBoundsException with delete query on bigger table

2018-02-09 Thread Rajeshbabu Chintaguntla (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16358291#comment-16358291
 ] 

Rajeshbabu Chintaguntla commented on PHOENIX-4591:
--

This is coming when once after writing rows equal to batch size we 50 we 
are clearing the indexMutations list. so next after committing we try to get 
the element from the empty list so IndexOutOfBoundsException is getting thrown. 
Instead of clearing the indexMutations list we need clear the maps holding in 
the list to avoid this.
{noformat}
// Commit a batch if auto commit is true and we're at our batch 
size
if (isAutoCommit && rowCount % batchSize == 0) {
MutationState state = new MutationState(tableRef, 
mutations, 0, maxSize, maxSizeBytes, connection);
connection.getMutationState().join(state);
for (int i = 0; i < otherTableRefs.size(); i++) {
MutationState indexState = new 
MutationState(otherTableRefs.get(i), indexMutations.get(i), 0, maxSize, 
maxSizeBytes, connection);
connection.getMutationState().join(indexState);
}
connection.getMutationState().send();
mutations.clear();
if (indexMutations != null) {
indexMutations.clear();
}
}
{noformat}

> Possible IndexOutOfBoundsException with delete query on bigger table
> 
>
> Key: PHOENIX-4591
> URL: https://issues.apache.org/jira/browse/PHOENIX-4591
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Romil Choksi
>Assignee: Rajeshbabu Chintaguntla
>Priority: Major
> Fix For: 5.0.0, 4.14.0
>
>
> Here is the schema and inserted around 5million records.
> {noformat}
> 0: jdbc:phoenix:localhost> CREATE TABLE IF NOT EXISTS testTable ( ID BIGINT 
> PRIMARY KEY , FirstName VARCHAR(30) , SecondName VARCHAR(30) , City 
> VARCHAR(30) );
> No rows affected (2.283 seconds)
> 0: jdbc:phoenix:localhost> CREATE INDEX secondary_index ON testTable 
> (FirstName);
> No rows affected (7.275 seconds)
> {noformat}
> Then delete from the table is throwing java.lang.IndexOutOfBoundsException
> {noformat}
> 0: jdbc:phoenix:localhost> delete from testtable;
> Error: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0 
> (state=08000,code=101)
> org.apache.phoenix.exception.PhoenixIOException: 
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at 
> org.apache.phoenix.util.ServerUtil.parseServerException(ServerUtil.java:122)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:970)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:912)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.getIterators(RoundRobinResultIterator.java:176)
>   at 
> org.apache.phoenix.iterate.RoundRobinResultIterator.next(RoundRobinResultIterator.java:91)
>   at 
> org.apache.phoenix.compile.DeleteCompiler$5.execute(DeleteCompiler.java:796)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:394)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement$2.call(PhoenixStatement.java:377)
>   at org.apache.phoenix.call.CallRunner.run(CallRunner.java:53)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:376)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.executeMutation(PhoenixStatement.java:364)
>   at 
> org.apache.phoenix.jdbc.PhoenixStatement.execute(PhoenixStatement.java:1738)
>   at sqlline.Commands.execute(Commands.java:822)
>   at sqlline.Commands.sql(Commands.java:732)
>   at sqlline.SqlLine.dispatch(SqlLine.java:813)
>   at sqlline.SqlLine.begin(SqlLine.java:686)
>   at sqlline.SqlLine.start(SqlLine.java:398)
>   at sqlline.SqlLine.main(SqlLine.java:291)
> Caused by: java.util.concurrent.ExecutionException: 
> java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at java.util.concurrent.FutureTask.report(FutureTask.java:122)
>   at java.util.concurrent.FutureTask.get(FutureTask.java:206)
>   at 
> org.apache.phoenix.iterate.BaseResultIterators.getIterators(BaseResultIterators.java:965)
>   ... 16 more
> Caused by: java.lang.IndexOutOfBoundsException: Index: 0, Size: 0
>   at java.util.ArrayList.rangeCheck(ArrayList.java:653)
>   at java.util.ArrayList.get(ArrayList.java:429)
>   at 
> org.apache.phoenix.compile.DeleteCompiler.deleteRows(DeleteCompiler.java:225)
>   at 
> org.apache.phoenix.compile.DeleteCompiler.access$000(DeleteCompiler.java:98)
>   at 
>