[jira] [Updated] (PHOENIX-4131) UngroupedAggregateRegionObserver.preClose() and doPostScannerOpen() can deadlock
[ https://issues.apache.org/jira/browse/PHOENIX-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Samarth Jain updated PHOENIX-4131: -- Attachment: PHOENIX-4131.patch We need to make sure that we are closing all scanners otherwise it could prevent mini cluster from shutting down. Also, I have slightly relaxed the check we have for checking if number of scans that are involved in writing. With this patch, I saw that my local runs were no longer hanging. I also removed the jvm halt shutdown hook since it was sometimes causing otherwise successful builds to fail. Would be interesting to see test run results. > UngroupedAggregateRegionObserver.preClose() and doPostScannerOpen() can > deadlock > > > Key: PHOENIX-4131 > URL: https://issues.apache.org/jira/browse/PHOENIX-4131 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Attachments: PHOENIX-4131.patch > > > On my local test run I saw that the tests were not completing because the > mini cluster couldn't shut down. So I took a jstack and discovered the > following deadlock: > {code} > "RS:0;samarthjai-wsm4:59006" #16265 prio=5 os_prio=31 tid=0x7fafa6327000 > nid=0x37b3f runnable [0x7000115f5000] >java.lang.Thread.State: RUNNABLE > at java.lang.Object.wait(Native Method) > at > org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.preClose(UngroupedAggregateRegionObserver.java:1201) > - locked <0x00072bc406b8> (a java.lang.Object) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$4.call(RegionCoprocessorHost.java:494) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1749) > at > org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preClose(RegionCoprocessorHost.java:490) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2843) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegionIgnoreErrors(HRegionServer.java:2805) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.closeUserRegions(HRegionServer.java:2423) > at > org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1052) > at > org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:157) > at > org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:110) > at > org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:141) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:360) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1637) > at > org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:334) > at > org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:139) > at java.lang.Thread.run(Thread.java:748) > {code} > {code} > "RpcServer.FifoWFPBQ.default.handler=3,queue=0,port=59006" #16246 daemon > prio=5 os_prio=31 tid=0x7fafae856000 nid=0x1abdb waiting for monitor > entry [0x7000102bc000] >java.lang.Thread.State: BLOCKED (on object monitor) > at > org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:734) > - waiting to lock <0x00072bc406b8> (a java.lang.Object) > at > org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.overrideDelegate(BaseScannerRegionObserver.java:236) > at > org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:281) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2629) > - locked <0x00072b625a90> (a > org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder) > at > org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2833) > at > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950) > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339) > at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188) > at > org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168) > {code} >
[jira] [Commented] (PHOENIX-4141) Fix flapping TableSnapshotReadsMapReduceIT
[ https://issues.apache.org/jira/browse/PHOENIX-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148449#comment-16148449 ] Hudson commented on PHOENIX-4141: - FAILURE: Integrated in Jenkins build Phoenix-master #1760 (See [https://builds.apache.org/job/Phoenix-master/1760/]) PHOENIX-4141 Addendum to fix test failure (samarth: rev 7d8b8430212fae117ac09faf6b7c22bf673e9073) * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/TableSnapshotReadsMapReduceIT.java > Fix flapping TableSnapshotReadsMapReduceIT > -- > > Key: PHOENIX-4141 > URL: https://issues.apache.org/jira/browse/PHOENIX-4141 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.12.0 > > Attachments: PHOENIX-4141.patch > > -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148425#comment-16148425 ] Lars Hofhansl commented on PHOENIX-418: --- You guys kick a**. :) > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3815) Only disable indexes on which write failures occurred
[ https://issues.apache.org/jira/browse/PHOENIX-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148368#comment-16148368 ] Hudson commented on PHOENIX-3815: - FAILURE: Integrated in Jenkins build Phoenix-master #1759 (See [https://builds.apache.org/job/Phoenix-master/1759/]) PHOENIX-3815 Only disable indexes on which write failures occurred (jtaylor: rev 7cf7b3abb10fc6a1ace0cb77615f054a8a6378f7) * (edit) phoenix-core/src/test/java/org/apache/phoenix/hbase/index/write/TestParalleIndexWriter.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexFailurePolicy.java * (delete) phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/ParallelWriterIndexCommitter.java * (edit) phoenix-core/src/test/java/org/apache/phoenix/hbase/index/write/TestIndexWriter.java * (add) phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/TrackingParallelWriterIndexCommitter.java * (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/index/MutableIndexFailureIT.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/RecoveryIndexWriter.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/recovery/StoreFailuresInCachePolicy.java * (edit) phoenix-core/src/test/java/org/apache/phoenix/hbase/index/write/TestParalleWriterIndexCommitter.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/IndexWriter.java * (delete) phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/recovery/TrackingParallelWriterIndexCommitter.java > Only disable indexes on which write failures occurred > - > > Key: PHOENIX-3815 > URL: https://issues.apache.org/jira/browse/PHOENIX-3815 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Vincent Poon > Fix For: 4.12.0 > > Attachments: PHOENIX-3815.0.98.v2.patch, > PHOENIX-3815.master.v2.patch, PHOENIX-3815.v1.patch > > > We currently disable all indexes if any of them fail to be written to. We > really only should disable the one in which the write failed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148366#comment-16148366 ] Hudson commented on PHOENIX-4128: - FAILURE: Integrated in Jenkins build Phoenix-master #1759 (See [https://builds.apache.org/job/Phoenix-master/1759/]) PHOENIX-4128 CNF org/apache/commons/math3/exception/OutOfRangeException (jtaylor: rev c83cec8903fd558777fb4dcd86a8bd67838f46e4) * (edit) phoenix-core/src/main/java/org/apache/phoenix/parse/ConcreteTableNode.java > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > Attachments: PHOENIX-4128-v1.patch > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148356#comment-16148356 ] Ethan Wang commented on PHOENIX-4128: - Thanks for showing me manifestation related to the dependency changes on the server side. I'd like to include this test set that you are running as part of my future local CI flow. Thanks again! [~mujtabachohan]. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > Attachments: PHOENIX-4128-v1.patch > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148306#comment-16148306 ] Hadoop QA commented on PHOENIX-4128: {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12884559/PHOENIX-4128-v1.patch against master branch at commit e6c1f01c5f7d5c9996017714efe90202a95355b2. ATTACHMENT ID: 12884559 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:red}-1 tests included{color}. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 62 warning messages. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TableSnapshotReadsMapReduceIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GroupByIT Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1326//testReport/ Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1326//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1326//console This message is automatically generated. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > Attachments: PHOENIX-4128-v1.patch > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (PHOENIX-3815) Only disable indexes on which write failures occurred
[ https://issues.apache.org/jira/browse/PHOENIX-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor reopened PHOENIX-3815: --- [~vincentpoon] - your patch causes the 0.98 and 1.1 branch to hang when running the unit tests. Would you mind taking a look? I'm using 1.7.0_80. I've reverted you patch for these two branches. > Only disable indexes on which write failures occurred > - > > Key: PHOENIX-3815 > URL: https://issues.apache.org/jira/browse/PHOENIX-3815 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Vincent Poon > Fix For: 4.12.0 > > Attachments: PHOENIX-3815.0.98.v2.patch, > PHOENIX-3815.master.v2.patch, PHOENIX-3815.v1.patch > > > We currently disable all indexes if any of them fail to be written to. We > really only should disable the one in which the write failed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (PHOENIX-3815) Only disable indexes on which write failures occurred
[ https://issues.apache.org/jira/browse/PHOENIX-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor resolved PHOENIX-3815. --- Resolution: Fixed Pushed to 4.x and master branches. Thanks for the fix, [~vincentpoon]. > Only disable indexes on which write failures occurred > - > > Key: PHOENIX-3815 > URL: https://issues.apache.org/jira/browse/PHOENIX-3815 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Vincent Poon > Fix For: 4.12.0 > > Attachments: PHOENIX-3815.0.98.v2.patch, > PHOENIX-3815.master.v2.patch, PHOENIX-3815.v1.patch > > > We currently disable all indexes if any of them fail to be written to. We > really only should disable the one in which the write failed. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148209#comment-16148209 ] James Taylor commented on PHOENIX-4128: --- +1 on the patch. Thanks, [~aertoria] and [~mujtabachohan]! > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > Attachments: PHOENIX-4128-v1.patch > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148188#comment-16148188 ] Mujtaba Chohan commented on PHOENIX-4128: - [~aertoria] with patch, things looks good. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > Attachments: PHOENIX-4128-v1.patch > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148175#comment-16148175 ] Hadoop QA commented on PHOENIX-418: --- {color:red}-1 overall{color}. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12884517/PHOENIX-418-v6-pom.patch against master branch at commit e6c1f01c5f7d5c9996017714efe90202a95355b2. ATTACHMENT ID: 12884517 {color:green}+1 @author{color}. The patch does not contain any @author tags. {color:green}+0 tests included{color}. The patch appears to be a documentation, build, or dev patch that doesn't require tests. {color:green}+1 javac{color}. The applied patch does not increase the total number of javac compiler warnings. {color:red}-1 javadoc{color}. The javadoc tool appears to have generated 62 warning messages. {color:green}+1 release audit{color}. The applied patch does not increase the total number of release audit warnings. {color:green}+1 lineLengths{color}. The patch does not introduce lines longer than 100 {color:red}-1 core tests{color}. The patch failed these unit tests: ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexFailureIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GroupByIT ./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TableSnapshotReadsMapReduceIT Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1325//testReport/ Javadoc warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1325//artifact/patchprocess/patchJavadocWarnings.txt Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/1325//console This message is automatically generated. > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Wang updated PHOENIX-4128: Attachment: PHOENIX-4128-v1.patch > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > Attachments: PHOENIX-4128-v1.patch > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130 ] Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:28 PM: --- [~jamestaylor] I meant the later. Since this is the only one place using OutOfRangeException, we can use alternative exception so that we will not rely on Math3 dependency. I wasn't aware the complexity of adding a extra dependency into phoenix project. throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to: throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); was (Author: aertoria): [~jamestaylor] I meant the later. Since this is the only one place using OutOfRangeException, we can use alternative exception so that we will not rely on Math3 dependency. throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to: throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130 ] Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:27 PM: --- [~jamestaylor] I meant the later. Since this is the only one place using OutOfRangeException, we can use alternative exception so that we will not rely on Math3 dependency. throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to: throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); was (Author: aertoria): [~jamestaylor] I meant the later. Since this is the only one place using OutOfRangeException, we can use alternative exception to get rid of this extra dependency. throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to: throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130 ] Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:27 PM: --- [~jamestaylor] I meant the later. Since this is the only one place using OutOfRangeException, we can use alternative exception to get rid of this extra dependency. throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to: throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); was (Author: aertoria): [~jamestaylor] I meant the later. Since there is only one place using OutofRangeException. We can use alternative exception to get rid of this extra dependency. throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to: throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130 ] Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:20 PM: --- [~jamestaylor] I meant the later. Since there is only one place using OutofRangeException. We can use alternative exception to get rid of this extra dependency. throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to: throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); was (Author: aertoria): [~jamestaylor] I meant the later. Since only there is only one place using OutofRangeException. We can change from: throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to: throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130 ] Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:19 PM: --- [~jamestaylor] I meant the later. Since only there is only one place using OutofRangeException. We can change from: throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to: throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); was (Author: aertoria): [~jamestaylor] I meant changing from throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130 ] Ethan Wang commented on PHOENIX-4128: - [~jamestaylor] I meant changing from throw new org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 100d); to throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 100"); > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148116#comment-16148116 ] James Taylor commented on PHOENIX-4128: --- bq. 2. I will remove the use of OutOfRangeException, replace it with IllegalArguementException. Do you mean changing the lib or are you throwing the exception? You'll want to just include all pom dependencies instead of changing the lib if the former. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4002) Document FETCH NEXT| n ROWS from Cursor
[ https://issues.apache.org/jira/browse/PHOENIX-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biju Nair updated PHOENIX-4002: --- Attachment: PHOENIX-4002-WIP.PATCH > Document FETCH NEXT| n ROWS from Cursor > --- > > Key: PHOENIX-4002 > URL: https://issues.apache.org/jira/browse/PHOENIX-4002 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Biju Nair > Attachments: PHOENIX-4002-WIP.PATCH > > > Now that PHOENIX-3572 is resolved and released, we need to add documentation > for this new functionality on our website. For directions on how to do that, > see http://phoenix.apache.org/building_website.html. I'd recommend adding a > new top level page linked off of our Features menu that explains from a users > perspective how to use it, and also updating our reference grammar here > (which is derived from content in phoenix.csv): > http://phoenix.apache.org/language/index.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4002) Document FETCH NEXT| n ROWS from Cursor
[ https://issues.apache.org/jira/browse/PHOENIX-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biju Nair updated PHOENIX-4002: --- Attachment: PATCH-4002-WIP.PATCH Attaching the {{svn}} diff as a patch. Please let me know if there is another process that need to be followed for doc patches. > Document FETCH NEXT| n ROWS from Cursor > --- > > Key: PHOENIX-4002 > URL: https://issues.apache.org/jira/browse/PHOENIX-4002 > Project: Phoenix > Issue Type: Sub-task >Reporter: James Taylor >Assignee: Biju Nair > Attachments: PATCH-4002-WIP.PATCH > > > Now that PHOENIX-3572 is resolved and released, we need to add documentation > for this new functionality on our website. For directions on how to do that, > see http://phoenix.apache.org/building_website.html. I'd recommend adding a > new top level page linked off of our Features menu that explains from a users > perspective how to use it, and also updating our reference grammar here > (which is derived from content in phoenix.csv): > http://phoenix.apache.org/language/index.html -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148031#comment-16148031 ] Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 8:51 PM: -- Correct [~jamestaylor]. Both are being used. Java.lang.math is being used in TableSamplerPredicate for generating hash org.apache.common.math3.exception.OutOfRangeException is being used in ConcreteTableNode for checking if input tablesample rate is within 0 and 100. if not, throw out OutOfRangeException. After discussed with [~mujtabachohan], two proposal options. 1, include org.apache.common.math3 in to the pom. 2, I will remove the use of OutOfRangeException, replace it with IllegalArguementException. (+1) Meanwhile I'm confirming if hbase 0.98 does not include common math3. was (Author: aertoria): Correct [~jamestaylor]. Both are being used. Java.lang.math is being used in TableSamplerPredicate for generating hash org.apache.common.math3.exception.OutOfRangeException is being used in ConcreteTableNode for checking if input tablesample rate is within 0 and 100. if not, throw out OutOfRangeException. After discussed with [~mujtabachohan], two proposes: 1, include org.apache.common.math3 in to the pom. 2, I will remove the use of OutOfRangeException, replace with IllegalArguementException. (+1) Meanwhile I'm confirming if hbase 0.98 does not include common math3. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148031#comment-16148031 ] Ethan Wang commented on PHOENIX-4128: - Correct [~jamestaylor]. Both are being used. Java.lang.math is being used in TableSamplerPredicate for generating hash org.apache.common.math3.exception.OutOfRangeException is being used in ConcreteTableNode for checking if input tablesample rate is within 0 and 100. if not, throw out OutOfRangeException. After discussed with [~mujtabachohan], two proposes: 1, include org.apache.common.math3 in to the pom. 2, I will remove the use of OutOfRangeException, replace with IllegalArguementException. (+1) Meanwhile I'm confirming if hbase 0.98 does not include common math3. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148022#comment-16148022 ] James Taylor commented on PHOENIX-4128: --- Might be another manifestation of the issue you fixed with your new patch on PHOENIX-418. Can you retest with the patch applied, [~mujtabachohan]? Please go ahead and commit that patch. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor reassigned PHOENIX-4128: - Assignee: Ethan Wang > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Assignee: Ethan Wang >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148007#comment-16148007 ] James Taylor commented on PHOENIX-4128: --- FYI, looks like org.apache.commons.math3 is being used, not java.lang.Math. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4144) Query for latest version during index rebuilding
[ https://issues.apache.org/jira/browse/PHOENIX-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4144: -- Attachment: PHOENIX-4144_wip_v2.patch Actually, I'm not able to repro it. My original repro had an error in the way it was manipulating its clock. The INDEX_FAILURE_HANDLING_REBUILD_OVERLAP_FORWARD_TIME_ATTRIB which is the overlap between when an index is marked INACTIVE versus when the rebuilder starts (by default 18ms) should be long enough to prevent this. Maybe if a client is stuck in an HBase retry loop for a long time (where the client has the index marked as disabled), this could occur. If that's the case, then I'm not sure how we can prevent it. Maybe Phoenix needs to manage the retry loops. If a client writes to the table while the index is disabled *after* the index builder is finished, then the index can get out of sync. Maybe we can leave the index active and let the write be attempted every time. > Query for latest version during index rebuilding > > > Key: PHOENIX-4144 > URL: https://issues.apache.org/jira/browse/PHOENIX-4144 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Attachments: PHOENIX-4144_wip.patch, PHOENIX-4144_wip_v2.patch > > > We're currently setting an upper bound of the scan to find the previous data > table row when rebuilding the index. We need to remove this upper bound, > though, so that we know if there are more recent versions of a data table row > so that we delete the row we've added. We can then process one row past our > current timestamp so we don't miss any deletes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147996#comment-16147996 ] Mujtaba Chohan commented on PHOENIX-4128: - Correct. This is with 0.98.23. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147992#comment-16147992 ] James Taylor commented on PHOENIX-4128: --- [~mujtabachohan] is likely testing on 0.98 branch. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147981#comment-16147981 ] Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 8:20 PM: -- I was not able to reproduce the same error on hbase 1.3.1. during table sampling hash generating process, java.lang.Math is used. I'll investigating the error stack to make sure if it need to be also included into the server side. Thanks for the info [~mujtabachohan] was (Author: aertoria): I was not able to reproduce the same error on hbase 1.3.1. math3 should not be used during table sampling process. I'll investigating the error stack. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147981#comment-16147981 ] Ethan Wang commented on PHOENIX-4128: - I was not able to reproduce the same error on hbase 1.3.1. math3 should not be used during table sampling process. I'll investigating the error stack. > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException
[ https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147958#comment-16147958 ] Mujtaba Chohan commented on PHOENIX-4128: - [~jamestaylor] It's a drop column statement {{ALTER TABLE T DROP COLUMN C1, C2}} This happens after this [commit|https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=0135c10754104b8a5515202edf3c78f72323ef18] for PHOENIX-153 FYI [~aertoria] > CNF org/apache/commons/math3/exception/OutOfRangeException > --- > > Key: PHOENIX-4128 > URL: https://issues.apache.org/jira/browse/PHOENIX-4128 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Mujtaba Chohan >Priority: Minor > Fix For: 4.12.0 > > > {noformat} > Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92) > > at > > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163) > > at > > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243) > > at > > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317) > > at > > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533) > > at > > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515) > > at > > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496) > > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195) > ... > > Caused by: java.lang.NoClassDefFoundError: > > org/apache/commons/math3/exception/OutOfRangeException > > at > > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827) > > at > > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519) > > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108) > > at > > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497) > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-4144) Query for latest version during index rebuilding
[ https://issues.apache.org/jira/browse/PHOENIX-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147878#comment-16147878 ] James Taylor edited comment on PHOENIX-4144 at 8/30/17 7:24 PM: Attach WIP patch with test that repros this potential issue. Will work on fix soon. FYI, [~vincentpoon]. was (Author: jamestaylor): Attach WIP patch with test that repros this potential issue. Will work on fix soon. > Query for latest version during index rebuilding > > > Key: PHOENIX-4144 > URL: https://issues.apache.org/jira/browse/PHOENIX-4144 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Attachments: PHOENIX-4144_wip.patch > > > We're currently setting an upper bound of the scan to find the previous data > table row when rebuilding the index. We need to remove this upper bound, > though, so that we know if there are more recent versions of a data table row > so that we delete the row we've added. We can then process one row past our > current timestamp so we don't miss any deletes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4144) Query for latest version during index rebuilding
[ https://issues.apache.org/jira/browse/PHOENIX-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4144: -- Attachment: PHOENIX-4144_wip.patch Attach WIP patch with test that repros this potential issue. Will work on fix soon. > Query for latest version during index rebuilding > > > Key: PHOENIX-4144 > URL: https://issues.apache.org/jira/browse/PHOENIX-4144 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Attachments: PHOENIX-4144_wip.patch > > > We're currently setting an upper bound of the scan to find the previous data > table row when rebuilding the index. We need to remove this upper bound, > though, so that we know if there are more recent versions of a data table row > so that we delete the row we've added. We can then process one row past our > current timestamp so we don't miss any deletes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (PHOENIX-4144) Query for latest version during index rebuilding
James Taylor created PHOENIX-4144: - Summary: Query for latest version during index rebuilding Key: PHOENIX-4144 URL: https://issues.apache.org/jira/browse/PHOENIX-4144 Project: Phoenix Issue Type: Bug Reporter: James Taylor Assignee: James Taylor We're currently setting an upper bound of the scan to find the previous data table row when rebuilding the index. We need to remove this upper bound, though, so that we know if there are more recent versions of a data table row so that we delete the row we've added. We can then process one row past our current timestamp so we don't miss any deletes. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147867#comment-16147867 ] James Taylor commented on PHOENIX-418: -- +1. Thanks for the quick turnaround, [~aertoria] and for the help [~mujtabachohan]. > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147826#comment-16147826 ] Ethan Wang edited comment on PHOENIX-418 at 8/30/17 7:03 PM: - patch v6-pom, for including dependency com.clearspring.analytics.stream into server side. due to my oversight earlier. Thank you [~mujtabachohan] for helping! was (Author: aertoria): patch v6-pom, for including dependency com.clearspring.analytics.stream into server side. Thank you [~mujtabachohan] for helping! > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147826#comment-16147826 ] Ethan Wang edited comment on PHOENIX-418 at 8/30/17 6:51 PM: - patch v6-pom, for including dependency com.clearspring.analytics.stream into server side. Thank you [~mujtabachohan] for helping! was (Author: aertoria): Thank you [~mujtabachohan] for helping! > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Wang updated PHOENIX-418: --- Attachment: PHOENIX-418-v6-pom.patch Thank you [~mujtabachohan] for helping! > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147704#comment-16147704 ] Ethan Wang edited comment on PHOENIX-418 at 8/30/17 6:22 PM: - I see. I forgot to ask during our discussion, for me the local test to work, I have to copy both the compiled phoenix-server.jar as well as a stream.jar to hbase lib folder. It make sense. I will go prepare another patch was (Author: aertoria): I see. I forgot to mention that, for me the local test to work, I have to copy both the compiled phoenix-server.jar as well as a stream.jar to hbase lib folder. I will go prepare another patch > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147721#comment-16147721 ] James Taylor commented on PHOENIX-418: -- By updating the assembly of the phoenix server jar in phoenix-assembly, you can get the clearspring classes bundled in the jar so that copy isn't necessary. > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147704#comment-16147704 ] Ethan Wang commented on PHOENIX-418: I see. I forgot to mention that, for me the local test to work, I have to copy both the compiled phoenix-server.jar as well as a stream.jar to hbase lib folder. I will go prepare another patch > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4096) Disallow DML operations on connections with CURRENT_SCN set
[ https://issues.apache.org/jira/browse/PHOENIX-4096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147700#comment-16147700 ] James Taylor commented on PHOENIX-4096: --- There are about 20 more tests derived from BaseClientManagedTimeIT classes that need to be converted before we can disallow DML and DDL on connections that set CURRENT_SCN. See my patch for ones I've already converted - pretty straightforward - just use unique table names and get rid of usage of CURRENT_SCN as these tests don't need it. If you can help, then please indicate which tests you're converting so we don't duplicate our efforts, [~vincentpoon], [~samarthjain], [~aertoria], [~tdsilva]. > Disallow DML operations on connections with CURRENT_SCN set > --- > > Key: PHOENIX-4096 > URL: https://issues.apache.org/jira/browse/PHOENIX-4096 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Attachments: PHOENIX-4096_wip.patch > > > We should make a connection read-only if CURRENT_SCN is set. It's really a > bad idea to go back in time and update data and it won't work with secondary > indexing, potentially leading to your index and table getting out of sync. > For testing purposes, where we need to control the timestamp, we should rely > on the EnvironmentEdgeManager instead to control the current time. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (PHOENIX-4143) ConcurrentMutationsIT flaps
Samarth Jain created PHOENIX-4143: - Summary: ConcurrentMutationsIT flaps Key: PHOENIX-4143 URL: https://issues.apache.org/jira/browse/PHOENIX-4143 Project: Phoenix Issue Type: Bug Reporter: Samarth Jain I have seen sometimes that the test flaps with a RegionOpeningException especially on master and 1.x branches. I believe this is likely because we are adding a co-processor by doing an admin.modify() call which is asynchronous. So it's possible that the region would likely be still opening even though the call returned. A potential fix is to poll for the updated table descriptor. Not sure if it will always prevent test from flapping (since the problem is similar to what Rajesh is solving in PHOENIX-3496) but it should reduce the chances. FYI, [~jamestaylor] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147690#comment-16147690 ] James Taylor commented on PHOENIX-418: -- [~aertoria] - please see comment above. Need to get the clearspring library into the server jar. This is controlled by phoenix-assembly. [~mujtabachohan], [~samarthjain], or [~tdsilva] can help if you run into any issues. > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-418: - Priority: Blocker > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang >Priority: Blocker > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-418: - Fix Version/s: 4.12.0 > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Reopened] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor reopened PHOENIX-418: -- > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-418: - Issue Type: New Feature (was: Task) > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: New Feature >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Fix For: 4.12.0 > > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (PHOENIX-4142) GEOMETRY data type and OpenGIS functions
Julian Hyde created PHOENIX-4142: Summary: GEOMETRY data type and OpenGIS functions Key: PHOENIX-4142 URL: https://issues.apache.org/jira/browse/PHOENIX-4142 Project: Phoenix Issue Type: Bug Reporter: Julian Hyde Add a GEOMETRY data type and core OpenGIS functions to Phoenix. This would allow Phoenix to execute spatial queries like those that PostGIS can execute. It would build upon the work done in Calcite in CALCITE-1968. The functions would need to be re-implemented (since a Calcite UDF is not a Phoenix UDF) but could use the same underlying library. A further step would be a spatial index - a sorted index on a Hilbert coordinate derived from latitude and longitude - and query rewrites to use it. See CALCITE-1861. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4138) Create a hard limit on number of indexes per table
[ https://issues.apache.org/jira/browse/PHOENIX-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146910#comment-16146910 ] Andrew Purtell commented on PHOENIX-4138: - That's my point exactly - the client can supply their own value. By definition this is not a hard limit. A hard limit is one that cannot be changed by the user. A server side mechanism is required to assure the check is always made and the set value cannot be changed by a client configuration update or code patch. > Create a hard limit on number of indexes per table > -- > > Key: PHOENIX-4138 > URL: https://issues.apache.org/jira/browse/PHOENIX-4138 > Project: Phoenix > Issue Type: Bug >Reporter: Rahul Shrivastava >Assignee: Rahul Shrivastava > Original Estimate: 48h > Remaining Estimate: 48h > > There should be a config parameter to impose a hard limit on number of > indexes per table. There is a SQL Exception > https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionCode.java#L260 > , but it gets triggered on the server side > (https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L1589) > . > We need a client side limit that can be configured via Phoenix config > parameter. Something like if user create more than lets say 30 indexes per > table, it would not allow more index creation for the that specific table. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4130) Avoid server retries for mutable indexes
[ https://issues.apache.org/jira/browse/PHOENIX-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146906#comment-16146906 ] Andrew Purtell commented on PHOENIX-4130: - Separate JIRA for that sounds ok. > Avoid server retries for mutable indexes > > > Key: PHOENIX-4130 > URL: https://issues.apache.org/jira/browse/PHOENIX-4130 > Project: Phoenix > Issue Type: Improvement >Reporter: Lars Hofhansl > > Had some discussions with [~jamestaylor], [~samarthjain], and [~vincentpoon], > during which I suggested that we can possibly eliminate retry loops happening > at the server that cause the handler threads to be stuck potentially for > quite a while (at least multiple seconds to ride over common scenarios like > splits). > Instead we can do the retries at the Phoenix client that. > So: > # The index updates are not retried on the server. (retries = 0) > # A failed index update would set the failed index timestamp but leave the > index enabled. > # Now the handler thread is done, it throws an appropriate exception back to > the client. > # The Phoenix client can now retry. When those retries fail the index is > disabled (if the policy dictates that) and throw the exception back to its > caller. > So no more waiting is needed on the server, handler threads are freed > immediately. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4130) Avoid server retries for mutable indexes
[ https://issues.apache.org/jira/browse/PHOENIX-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146902#comment-16146902 ] Andrew Purtell commented on PHOENIX-4130: - The catalog table may be available but jammed with user driven traffic or there could be a temporary partial partition long enough. I agree that causes such as those will be impacting availability in their own right, but if a design where no successful writes are necessary upon index update failures is possible, this won't complicate or delay recovery e.g. by holding a handler waiting and spinning on a timestamp update. > Avoid server retries for mutable indexes > > > Key: PHOENIX-4130 > URL: https://issues.apache.org/jira/browse/PHOENIX-4130 > Project: Phoenix > Issue Type: Improvement >Reporter: Lars Hofhansl > > Had some discussions with [~jamestaylor], [~samarthjain], and [~vincentpoon], > during which I suggested that we can possibly eliminate retry loops happening > at the server that cause the handler threads to be stuck potentially for > quite a while (at least multiple seconds to ride over common scenarios like > splits). > Instead we can do the retries at the Phoenix client that. > So: > # The index updates are not retried on the server. (retries = 0) > # A failed index update would set the failed index timestamp but leave the > index enabled. > # Now the handler thread is done, it throws an appropriate exception back to > the client. > # The Phoenix client can now retry. When those retries fail the index is > disabled (if the policy dictates that) and throw the exception back to its > caller. > So no more waiting is needed on the server, handler threads are freed > immediately. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4140) Disable HiveTezIT and HiveMapReduceIT for 4.x-HBase-0.98
[ https://issues.apache.org/jira/browse/PHOENIX-4140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146842#comment-16146842 ] Hudson commented on PHOENIX-4140: - FAILURE: Integrated in Jenkins build Phoenix-master #1758 (See [https://builds.apache.org/job/Phoenix-master/1758/]) Revert "PHOENIX-4140 Disable HiveTezIT and HiveMapReduceIT since they (samarth: rev e6c1f01c5f7d5c9996017714efe90202a95355b2) * (edit) phoenix-hive/src/it/java/org/apache/phoenix/hive/HiveTezIT.java * (edit) phoenix-hive/src/it/java/org/apache/phoenix/hive/HiveMapReduceIT.java > Disable HiveTezIT and HiveMapReduceIT for 4.x-HBase-0.98 > > > Key: PHOENIX-4140 > URL: https://issues.apache.org/jira/browse/PHOENIX-4140 > Project: Phoenix > Issue Type: Bug >Reporter: Samarth Jain >Assignee: Samarth Jain > Fix For: 4.12.0 > > Attachments: PHOENIX-4140.patch > > > I have never seen the HiveTezIT and HiveMapReduceIT complete successfully. > Locally, on my laptop too, I was unable to get these tests to run > successfully. > See a sample run where they failed - > https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/1647/console > On my laptop, these tests failed with an OOM. I had to override the permgen > memory to 256m to get the tests to even start. > FYI, [~sergey.soldatov] -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146707#comment-16146707 ] Lars Hofhansl edited comment on PHOENIX-418 at 8/30/17 6:31 AM: Shouldn't the clearspring library be part of the server jar? I just did a clean build from scratch*, and indeed APPROX_COUNT_DISTINCT fails with {code} ... Caused by: java.lang.NoClassDefFoundError: com/clearspring/analytics/stream/cardinality/HyperLogLogPlus ... {code} thrown at the server. \* mvn -DskipTests package was (Author: lhofhansl): Shouldn't the clearspring library be part of the server jar? I just did a clean build from scratch, and indeed APPROX_COUNT_DISTINCT fails with {code} ... Caused by: java.lang.NoClassDefFoundError: com/clearspring/analytics/stream/cardinality/HyperLogLogPlus ... {code} thrown at the server. > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146707#comment-16146707 ] Lars Hofhansl commented on PHOENIX-418: --- Shouldn't the clearspring library be part of the server jar? I just did a clean build from scratch, and indeed APPROX_COUNT_DISTINCT fails with {code} ... Caused by: java.lang.NoClassDefFoundError: com/clearspring/analytics/stream/cardinality/HyperLogLogPlus ... {code} thrown at the server. > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). > Update: > Syntax of using approximate count distinct as: > select APPROX_COUNT_DISTINCT(name) from person > select APPROX_COUNT_DISTINCT(address||name) from person > It is equivalent of Select COUNT(DISTINCT ID) from person. Implemented using > hyperloglog, see discuss below. > Source code patch link below, co-authorred with [~swapna] > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4080) The error message for version mismatch is not accurate.
[ https://issues.apache.org/jira/browse/PHOENIX-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146674#comment-16146674 ] Hudson commented on PHOENIX-4080: - FAILURE: Integrated in Jenkins build Phoenix-master #1757 (See [https://builds.apache.org/job/Phoenix-master/1757/]) PHOENIX-4080 Amending error message when phoenix client version is not (gjacoby: rev 74762df7a5649592ad81ca4ad72425acc2b0a823) * (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java * (edit) phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java > The error message for version mismatch is not accurate. > --- > > Key: PHOENIX-4080 > URL: https://issues.apache.org/jira/browse/PHOENIX-4080 > Project: Phoenix > Issue Type: Wish >Affects Versions: 4.11.0 >Reporter: Ethan Wang >Assignee: Ethan Wang > Fix For: 4.12.0 > > Attachments: PHOENIX-4080.patch, PHOENIX-4080-v2.patch, > PHOENIX-4080-v3.patch > > > When accessing a 4.10 running cluster with 4.11 client, it referred as > The following servers require an updated phoenix.jar to be put in the > classpath of HBase: region=SYSTEM.CATALOG > It should be phoenix-[version]-server.jar rather than phoenix.jar -- This message was sent by Atlassian JIRA (v6.4.14#64029)