[jira] [Commented] (PHOENIX-3603) Fix compilation errors against hbase 1.3.1 release
[
https://issues.apache.org/jira/browse/PHOENIX-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984128#comment-15984128
]
Hudson commented on PHOENIX-3603:
-
FAILURE: Integrated in Jenkins build Phoenix-master #1603 (See
[https://builds.apache.org/job/Phoenix-master/1603/])
PHOENIX-3603 Fix compilation errors against hbase 1.3 (apurtell: rev
5b099014446865c12779f3882fd8b407496717ea)
* (edit) phoenix-hive/pom.xml
* (edit) phoenix-assembly/pom.xml
* (edit) phoenix-spark/pom.xml
* (edit) phoenix-queryserver-client/pom.xml
* (edit)
phoenix-core/src/it/java/org/apache/hadoop/hbase/regionserver/wal/WALReplayWithIndexWritesAndCompressedWALIT.java
* (edit)
phoenix-core/src/main/java/org/apache/phoenix/execute/DelegateHTable.java
* (edit) phoenix-flume/pom.xml
* (edit) phoenix-server/pom.xml
* (edit) phoenix-client/pom.xml
* (edit) phoenix-kafka/pom.xml
* (edit) phoenix-queryserver/pom.xml
* (edit)
phoenix-core/src/test/java/org/apache/phoenix/hbase/index/write/recovery/TestPerRegionIndexWriteCache.java
* (edit) pom.xml
* (edit)
phoenix-core/src/test/java/org/apache/hadoop/hbase/ipc/PhoenixIndexRpcSchedulerTest.java
* (edit) phoenix-tracing-webapp/pom.xml
* (edit) phoenix-pherf/pom.xml
* (edit)
phoenix-core/src/main/java/org/apache/hadoop/hbase/ipc/PhoenixRpcScheduler.java
* (edit) phoenix-pig/pom.xml
* (edit) phoenix-core/pom.xml
> Fix compilation errors against hbase 1.3.1 release
> --
>
> Key: PHOENIX-3603
> URL: https://issues.apache.org/jira/browse/PHOENIX-3603
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Zach York
> Fix For: 4.11.0
>
> Attachments: PHOENIX-3603.patch
>
>
> hbase 1.3.0 has been released.
> I saw the following when compiling master branch against 1.3.0
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile)
> on project phoenix-core: Compilation failure: Compilation failure:
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/phoenix/execute/DelegateHTable.java:[49,8]
> org.apache.phoenix.execute.DelegateHTable is not abstract and does not
> override abstract method getRpcTimeout() in
> org.apache.hadoop.hbase.client.Table
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/hadoop/hbase/ipc/PhoenixRpcScheduler.java:[32,8]
> org.apache.hadoop.hbase.ipc.PhoenixRpcScheduler is not abstract and does not
> override abstract method getNumLifoModeSwitches() in
> org.apache.hadoop.hbase.ipc.RpcScheduler
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (PHOENIX-3603) Fix compilation errors against hbase 1.3.1 release
[
https://issues.apache.org/jira/browse/PHOENIX-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15984091#comment-15984091
]
James Taylor commented on PHOENIX-3603:
---
If you haven't already done so, please create a 4.x-HBase-1.2 branch off of
master at the commit prior to this one. We'll need to continue with releases
that support 1.2 still IMHO. If you could send out a note on the dev list to
remind the community of these changes, that'd be much appreciated.
> Fix compilation errors against hbase 1.3.1 release
> --
>
> Key: PHOENIX-3603
> URL: https://issues.apache.org/jira/browse/PHOENIX-3603
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Zach York
> Fix For: 4.11.0
>
> Attachments: PHOENIX-3603.patch
>
>
> hbase 1.3.0 has been released.
> I saw the following when compiling master branch against 1.3.0
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile)
> on project phoenix-core: Compilation failure: Compilation failure:
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/phoenix/execute/DelegateHTable.java:[49,8]
> org.apache.phoenix.execute.DelegateHTable is not abstract and does not
> override abstract method getRpcTimeout() in
> org.apache.hadoop.hbase.client.Table
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/hadoop/hbase/ipc/PhoenixRpcScheduler.java:[32,8]
> org.apache.hadoop.hbase.ipc.PhoenixRpcScheduler is not abstract and does not
> override abstract method getNumLifoModeSwitches() in
> org.apache.hadoop.hbase.ipc.RpcScheduler
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Resolved] (PHOENIX-3603) Fix compilation errors against hbase 1.3.1 release
[
https://issues.apache.org/jira/browse/PHOENIX-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell resolved PHOENIX-3603.
-
Resolution: Fixed
Committed to master
> Fix compilation errors against hbase 1.3.1 release
> --
>
> Key: PHOENIX-3603
> URL: https://issues.apache.org/jira/browse/PHOENIX-3603
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Zach York
> Fix For: 4.11.0
>
> Attachments: PHOENIX-3603.patch
>
>
> hbase 1.3.0 has been released.
> I saw the following when compiling master branch against 1.3.0
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile)
> on project phoenix-core: Compilation failure: Compilation failure:
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/phoenix/execute/DelegateHTable.java:[49,8]
> org.apache.phoenix.execute.DelegateHTable is not abstract and does not
> override abstract method getRpcTimeout() in
> org.apache.hadoop.hbase.client.Table
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/hadoop/hbase/ipc/PhoenixRpcScheduler.java:[32,8]
> org.apache.hadoop.hbase.ipc.PhoenixRpcScheduler is not abstract and does not
> override abstract method getNumLifoModeSwitches() in
> org.apache.hadoop.hbase.ipc.RpcScheduler
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Assigned] (PHOENIX-3808) Implement chaos tests using HBase's hbase-it facility
[ https://issues.apache.org/jira/browse/PHOENIX-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Purtell reassigned PHOENIX-3808: --- Assignee: Andrew Purtell > Implement chaos tests using HBase's hbase-it facility > - > > Key: PHOENIX-3808 > URL: https://issues.apache.org/jira/browse/PHOENIX-3808 > Project: Phoenix > Issue Type: Improvement >Reporter: Andrew Purtell >Assignee: Andrew Purtell > > Implement chaos tests using HBase's hbase-it facility. Especially, > correctness testing with an active server killing monkey policy. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (PHOENIX-3603) Fix compilation errors against hbase 1.3.1 release
[
https://issues.apache.org/jira/browse/PHOENIX-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Andrew Purtell updated PHOENIX-3603:
Attachment: PHOENIX-3603.patch
I have updated the patch to use HBase release 1.3.1 and have been running the
test suite all afternoon. The unit tests and short running IT tests all pass.
The only issue I've found so far with the long running IT tests is MapReduceIT
will run out of heap, which does not appear related to this change. Unless
something drastic happens between now and when the test run completes, I plan
to commit the attached patch on master, moving up from HBase 1.2 to HBase 1.3.
Please advise if you'd like me to hold off.
> Fix compilation errors against hbase 1.3.1 release
> --
>
> Key: PHOENIX-3603
> URL: https://issues.apache.org/jira/browse/PHOENIX-3603
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Zach York
> Fix For: 4.11.0
>
> Attachments: PHOENIX-3603.patch
>
>
> hbase 1.3.0 has been released.
> I saw the following when compiling master branch against 1.3.0
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile)
> on project phoenix-core: Compilation failure: Compilation failure:
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/phoenix/execute/DelegateHTable.java:[49,8]
> org.apache.phoenix.execute.DelegateHTable is not abstract and does not
> override abstract method getRpcTimeout() in
> org.apache.hadoop.hbase.client.Table
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/hadoop/hbase/ipc/PhoenixRpcScheduler.java:[32,8]
> org.apache.hadoop.hbase.ipc.PhoenixRpcScheduler is not abstract and does not
> override abstract method getNumLifoModeSwitches() in
> org.apache.hadoop.hbase.ipc.RpcScheduler
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (PHOENIX-3808) Implement chaos tests using HBase's hbase-it facility
[ https://issues.apache.org/jira/browse/PHOENIX-3808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983920#comment-15983920 ] Andrew Purtell commented on PHOENIX-3808: - [~mujtabachohan] points out that Phoenix IT tests can be executed against a cluster with https://github.com/apache/phoenix/blob/master/phoenix-core/src/it/java/org/apache/phoenix/end2end/End2EndTestDriver.java. End2EndTestDriver is pretty much a clone of HBase IntegrationTestsDriver. End2EndTestDriver extends AbstractHBaseTool. If it instead extended HBase's IntegrationTestBase (which extends AbstractHBaseTool) then we get hbase-it monkey factory integration. > Implement chaos tests using HBase's hbase-it facility > - > > Key: PHOENIX-3808 > URL: https://issues.apache.org/jira/browse/PHOENIX-3808 > Project: Phoenix > Issue Type: Improvement >Reporter: Andrew Purtell > > Implement chaos tests using HBase's hbase-it facility. Especially, > correctness testing with an active server killing monkey policy. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Assigned] (PHOENIX-3812) Explore using HBase snapshots in async index building M/R job
[ https://issues.apache.org/jira/browse/PHOENIX-3812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor reassigned PHOENIX-3812: - Assignee: Maddineni Sukumar > Explore using HBase snapshots in async index building M/R job > - > > Key: PHOENIX-3812 > URL: https://issues.apache.org/jira/browse/PHOENIX-3812 > Project: Phoenix > Issue Type: Improvement >Affects Versions: 4.10.0 >Reporter: Maddineni Sukumar >Assignee: Maddineni Sukumar > > As per discussion with James, HBase snapshots makes it lot easier and faster > to operate on existing data. > So explore using HBase snapshots in index building M/R job for async index. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (PHOENIX-3812) Explore using HBase snapshots in async index building M/R job
Maddineni Sukumar created PHOENIX-3812: -- Summary: Explore using HBase snapshots in async index building M/R job Key: PHOENIX-3812 URL: https://issues.apache.org/jira/browse/PHOENIX-3812 Project: Phoenix Issue Type: Improvement Affects Versions: 4.10.0 Reporter: Maddineni Sukumar As per discussion with James, HBase snapshots makes it lot easier and faster to operate on existing data. So explore using HBase snapshots in index building M/R job for async index. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3806) IndexUpdateManager spending a lot of time sorting mutations on Index rebuild
[
https://issues.apache.org/jira/browse/PHOENIX-3806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983858#comment-15983858
]
Samarth Jain commented on PHOENIX-3806:
---
One thing we could do is to look in the HTableDescriptor of the data table and
see what the "versions" attribute is set to and then use the same number of
version when doing raw scan (instead of doing scan.setMaxVersions())
> IndexUpdateManager spending a lot of time sorting mutations on Index rebuild
>
>
> Key: PHOENIX-3806
> URL: https://issues.apache.org/jira/browse/PHOENIX-3806
> Project: Phoenix
> Issue Type: Bug
>Reporter: Lars Hofhansl
>
> Here's the stack trace. The Array contains 50001 Delete Mutations in this
> case.
> It seems the code is sorting this over and over again.
> {code}
> Thread 170 (B.DefaultRpcServer.handler=67,queue=7,port=60020):
> State: RUNNABLE
> Blocked count: 220598
> Waited count: 377933
> Stack:
> java.util.TimSort.binarySort(TimSort.java:296)
> java.util.TimSort.sort(TimSort.java:239)
> java.util.Arrays.sort(Arrays.java:1438)
>
> org.apache.phoenix.hbase.index.covered.update.SortedCollection.iterator(SortedCollection.java:78)
>
> org.apache.phoenix.hbase.index.covered.update.IndexUpdateManager.fixUpCurrentUpdates(IndexUpdateManager.java:128)
>
> org.apache.phoenix.hbase.index.covered.update.IndexUpdateManager.addIndexUpdate(IndexUpdateManager.java:115)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.addCurrentStateMutationsForBatch(NonTxIndexBuilder.java:333)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.addUpdateForGivenTimestamp(NonTxIndexBuilder.java:258)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.addMutationsForBatch(NonTxIndexBuilder.java:231)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.batchMutationAndAddUpdates(NonTxIndexBuilder.java:109)
>
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.getIndexUpdate(NonTxIndexBuilder.java:71)
>
> org.apache.phoenix.hbase.index.builder.IndexBuildManager$1.call(IndexBuildManager.java:137)
>
> org.apache.phoenix.hbase.index.builder.IndexBuildManager$1.call(IndexBuildManager.java:133)
> java.util.concurrent.FutureTask.run(FutureTask.java:266)
>
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:293)
>
> com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:61)
>
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submit(BaseTaskRunner.java:58)
>
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submitUninterruptible(BaseTaskRunner.java:99)
>
> org.apache.phoenix.hbase.index.builder.IndexBuildManager.getIndexUpdate(IndexBuildManager.java:144)
>
> org.apache.phoenix.hbase.index.Indexer.preBatchMutateWithExceptions(Indexer.java:324)
> Thread 169 (B.DefaultRpcServer.handler=66,queue=6,port=60020):
> {code}
> [~jamestaylor]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Comment Edited] (PHOENIX-3571) Potential divide by zero exception in LongDivideExpression
[
https://issues.apache.org/jira/browse/PHOENIX-3571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15954966#comment-15954966
]
Ted Yu edited comment on PHOENIX-3571 at 4/25/17 9:59 PM:
--
Assertion for zero denominator is fine .
was (Author: yuzhih...@gmail.com):
Assertion for zero denominator is fine.
> Potential divide by zero exception in LongDivideExpression
> --
>
> Key: PHOENIX-3571
> URL: https://issues.apache.org/jira/browse/PHOENIX-3571
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
>Reporter: Ted Yu
>Priority: Minor
>
> Running SaltedIndexIT, I saw the following:
> {code}
> ===>
> testExpressionThrowsException(org.apache.phoenix.end2end.index.IndexExpressionIT)
> starts
> 2017-01-05 19:42:48,992 INFO [main] client.HBaseAdmin: Created I
> 2017-01-05 19:42:48,996 INFO [main] schema.MetaDataClient: Created index I
> at 1483645369000
> 2017-01-05 19:42:49,066 WARN [hconnection-0x5a45c218-shared--pool52-t6]
> client.AsyncProcess: #38, table=T, attempt=1/35 failed=1ops, last exception:
> org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException:
> org.apache.phoenix.hbase.index.builder.IndexBuildingFailureException: Failed
> to build index for unexpected reason!
> at
> org.apache.phoenix.hbase.index.util.IndexManagementUtil.rethrowIndexingException(IndexManagementUtil.java:183)
> at org.apache.phoenix.hbase.index.Indexer.preBatchMutate(Indexer.java:204)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$35.call(RegionCoprocessorHost.java:974)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1660)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1734)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1692)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preBatchMutate(RegionCoprocessorHost.java:970)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:3218)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2984)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2926)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doBatchOp(RSRpcServices.java:718)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.doNonAtomicRegionMutation(RSRpcServices.java:680)
> at
> org.apache.hadoop.hbase.regionserver.RSRpcServices.multi(RSRpcServices.java:2065)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32393)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2141)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:112)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:238)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:218)
> Caused by: java.lang.ArithmeticException: / by zero
> at
> org.apache.phoenix.expression.LongDivideExpression.evaluate(LongDivideExpression.java:50)
> at
> org.apache.phoenix.index.IndexMaintainer.buildRowKey(IndexMaintainer.java:521)
> at
> org.apache.phoenix.index.IndexMaintainer.buildUpdateMutation(IndexMaintainer.java:859)
> at
> org.apache.phoenix.index.PhoenixIndexCodec.getIndexUpserts(PhoenixIndexCodec.java:76)
> at
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.addCurrentStateMutationsForBatch(NonTxIndexBuilder.java:288)
> at
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.addUpdateForGivenTimestamp(NonTxIndexBuilder.java:256)
> at
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.addMutationsForBatch(NonTxIndexBuilder.java:222)
> at
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.batchMutationAndAddUpdates(NonTxIndexBuilder.java:109)
> at
> org.apache.phoenix.hbase.index.covered.NonTxIndexBuilder.getIndexUpdate(NonTxIndexBuilder.java:71)
> at
> org.apache.phoenix.hbase.index.builder.IndexBuildManager$1.call(IndexBuildManager.java:136)
> at
> org.apache.phoenix.hbase.index.builder.IndexBuildManager$1.call(IndexBuildManager.java:132)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:253)
> at
> com.google.common.util.concurrent.AbstractListeningExecutorService.submit(AbstractListeningExecutorService.java:56)
> at
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submit(BaseTaskRunner.java:58)
> at
> org.apache.phoenix.hbase.index.parallel.BaseTaskRunner.submitUninterruptible(BaseTaskRunner.java:99)
> at
>
[jira] [Updated] (PHOENIX-3603) Fix compilation errors against hbase 1.3.1 release
[
https://issues.apache.org/jira/browse/PHOENIX-3603?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ted Yu updated PHOENIX-3603:
Summary: Fix compilation errors against hbase 1.3.1 release (was: Fix
compilation errors against hbase 1.3.0 release)
> Fix compilation errors against hbase 1.3.1 release
> --
>
> Key: PHOENIX-3603
> URL: https://issues.apache.org/jira/browse/PHOENIX-3603
> Project: Phoenix
> Issue Type: Bug
>Reporter: Ted Yu
>Assignee: Zach York
> Fix For: 4.11.0
>
>
> hbase 1.3.0 has been released.
> I saw the following when compiling master branch against 1.3.0
> {code}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile)
> on project phoenix-core: Compilation failure: Compilation failure:
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/phoenix/execute/DelegateHTable.java:[49,8]
> org.apache.phoenix.execute.DelegateHTable is not abstract and does not
> override abstract method getRpcTimeout() in
> org.apache.hadoop.hbase.client.Table
> [ERROR]
> /Users/tyu/phoenix/phoenix-core/src/main/java/org/apache/hadoop/hbase/ipc/PhoenixRpcScheduler.java:[32,8]
> org.apache.hadoop.hbase.ipc.PhoenixRpcScheduler is not abstract and does not
> override abstract method getNumLifoModeSwitches() in
> org.apache.hadoop.hbase.ipc.RpcScheduler
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (PHOENIX-3811) Ignore index write failure and let client deal with failure
[ https://issues.apache.org/jira/browse/PHOENIX-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-3811: -- Attachment: PHOENIX-3811-wip2.patch > Ignore index write failure and let client deal with failure > --- > > Key: PHOENIX-3811 > URL: https://issues.apache.org/jira/browse/PHOENIX-3811 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor > Attachments: PHOENIX-3811-wip1.patch, PHOENIX-3811-wip2.patch > > > We should provide a way to configure the system so that the server takes no > specific action when an index write fails. Since we always throw the write > failure back to the client, the client can often deal with failures more > easily than the server since they have the batch of mutations in memory. > Often times, allowing access to an index that may be one batch behind the > data table is better than disabling it given the negative performance that > will occur while the index cannot be written to. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
[ https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983232#comment-15983232 ] churro morales commented on PHOENIX-3534: - that is fine, we resolve columns on read so they can be excluded quite easily. I believe we walk from child to parent always as well so we can just plug that logic into our getTable code. > Support multi region SYSTEM.CATALOG table > - > > Key: PHOENIX-3534 > URL: https://issues.apache.org/jira/browse/PHOENIX-3534 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: churro morales > > Currently Phoenix requires that the SYSTEM.CATALOG table is single region > based on the server-side row locks being held for operations that impact a > table and all of it's views. For example, adding/removing a column from a > base table pushes this change to all views. > As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431), > when a new table is created we can do a lazy cleanup of any rows that may be > left over from a failed DDL call (kudos to [~lhofhansl] for coming up with > this idea). To implement this efficiently, we'd need to also do PHOENIX-2051 > so that we can efficiently find derived views. > The implementation would rely on an optimistic concurrency model based on > checking our sequence numbers for each table/view before/after updating. Each > table/view row would be individually locked for their change (metadata for a > view or table cannot span regions due to our split policy), with the sequence > number being incremented under lock and then returned to the client. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3534) Support multi region SYSTEM.CATALOG table
[
https://issues.apache.org/jira/browse/PHOENIX-3534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15983192#comment-15983192
]
James Taylor commented on PHOENIX-3534:
---
One more corner case: once a view has deleted columns from its parent table(s),
it's considered as "diverged". I think we can detect this if the view has any
excluded columns. In this case, we should no longer add columns from the parent
tables created after this. I think we can base this on the earliest timestamp
of the excluded column key value. This is conceptually equivalent to creating
the view without doing a SELECT *, but instead only selecting a subset of
columns, like this:
{code}
CREATE VIEW v AS SELECT a,c FROM t;
{code}
In this case, columns added to {{t}} should not impact the view {{v}} at all.
> Support multi region SYSTEM.CATALOG table
> -
>
> Key: PHOENIX-3534
> URL: https://issues.apache.org/jira/browse/PHOENIX-3534
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: churro morales
>
> Currently Phoenix requires that the SYSTEM.CATALOG table is single region
> based on the server-side row locks being held for operations that impact a
> table and all of it's views. For example, adding/removing a column from a
> base table pushes this change to all views.
> As an alternative to making the SYSTEM.CATALOG transactional (PHOENIX-2431),
> when a new table is created we can do a lazy cleanup of any rows that may be
> left over from a failed DDL call (kudos to [~lhofhansl] for coming up with
> this idea). To implement this efficiently, we'd need to also do PHOENIX-2051
> so that we can efficiently find derived views.
> The implementation would rely on an optimistic concurrency model based on
> checking our sequence numbers for each table/view before/after updating. Each
> table/view row would be individually locked for their change (metadata for a
> view or table cannot span regions due to our split policy), with the sequence
> number being incremented under lock and then returned to the client.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Updated] (PHOENIX-3811) Ignore index write failure and let client deal with failure
[ https://issues.apache.org/jira/browse/PHOENIX-3811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-3811: -- Attachment: PHOENIX-3811-wip1.patch > Ignore index write failure and let client deal with failure > --- > > Key: PHOENIX-3811 > URL: https://issues.apache.org/jira/browse/PHOENIX-3811 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor > Attachments: PHOENIX-3811-wip1.patch > > > We should provide a way to configure the system so that the server takes no > specific action when an index write fails. Since we always throw the write > failure back to the client, the client can often deal with failures more > easily than the server since they have the batch of mutations in memory. > Often times, allowing access to an index that may be one batch behind the > data table is better than disabling it given the negative performance that > will occur while the index cannot be written to. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (PHOENIX-3710) Cannot use lowername data table name with indextool
[
https://issues.apache.org/jira/browse/PHOENIX-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15982772#comment-15982772
]
Hadoop QA commented on PHOENIX-3710:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12864835/PHOENIX-3710.patch
against master branch at commit 92b951e5387768e084ed09729884a59160cd81d3.
ATTACHMENT ID: 12864835
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
47 warning messages.
{color:green}+1 release audit{color}. The applied patch does not increase
the total number of release audit warnings.
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+? SchemaUtil.getQualifiedTableName(schemaName,
indexTable) : SchemaUtil.normalizeIdentifier(indexTable));
+IndexToolUtil.updateIndexState(connection, dataTableName,
indexTable, PIndexState.ACTIVE);
+final String schemaName =
SchemaUtil.normalizeIdentifier(SchemaUtil.getSchemaNameFromFullName(masterTable));
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexFailureIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.IndexToolForPartialBuildWithNamespaceEnabledIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/839//testReport/
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/839//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/839//console
This message is automatically generated.
> Cannot use lowername data table name with indextool
> ---
>
> Key: PHOENIX-3710
> URL: https://issues.apache.org/jira/browse/PHOENIX-3710
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Matthew Shipton
>Assignee: Sergey Soldatov
>Priority: Minor
> Attachments: PHOENIX-3710.patch, test.sh, test.sql
>
>
> {code}
> hbase org.apache.phoenix.mapreduce.index.IndexTool --data-table
> \"my_lowcase_table\" --index-table INDEX_TABLE --output-path /tmp/some_path
> {code}
> results in:
> {code}
> java.lang.IllegalArgumentException: INDEX_TABLE is not an index table for
> MY_LOWCASE_TABLE
> {code}
> This is despite the data table being explictly lowercased.
> Appears to be referring to the lowcase table, not the uppercase version.
> Workaround exists by changing the tablename, but this is not always feasible.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
[jira] [Commented] (PHOENIX-3710) Cannot use lowername data table name with indextool
[
https://issues.apache.org/jira/browse/PHOENIX-3710?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15982460#comment-15982460
]
Ankit Singhal commented on PHOENIX-3710:
Thanks [~sergey.soldatov], changes look good. But at least to avoid regression,
if we can include below two test cases as
IT(IndexExtendedIT.testSecondaryIndex() can be reused for this)
* IndexTool with both tablename and indexname in lowercase
* To check whether the normalisation is correct for phoenix tableName of type
\"S:T\"
> Cannot use lowername data table name with indextool
> ---
>
> Key: PHOENIX-3710
> URL: https://issues.apache.org/jira/browse/PHOENIX-3710
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
>Reporter: Matthew Shipton
>Assignee: Sergey Soldatov
>Priority: Minor
> Attachments: PHOENIX-3710.patch, test.sh, test.sql
>
>
> {code}
> hbase org.apache.phoenix.mapreduce.index.IndexTool --data-table
> \"my_lowcase_table\" --index-table INDEX_TABLE --output-path /tmp/some_path
> {code}
> results in:
> {code}
> java.lang.IllegalArgumentException: INDEX_TABLE is not an index table for
> MY_LOWCASE_TABLE
> {code}
> This is despite the data table being explictly lowercased.
> Appears to be referring to the lowcase table, not the uppercase version.
> Workaround exists by changing the tablename, but this is not always feasible.
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
