[jira] [Updated] (PHOENIX-4131) UngroupedAggregateRegionObserver.preClose() and doPostScannerOpen() can deadlock

2017-08-30 Thread Samarth Jain (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4131?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Samarth Jain updated PHOENIX-4131:
--
Attachment: PHOENIX-4131.patch

We need to make sure that we are closing all scanners otherwise it could 
prevent mini cluster from shutting down. Also, I have slightly relaxed the 
check we have for checking if number of scans that are involved in writing. 
With this patch, I saw that my local runs were no longer hanging. I also 
removed the jvm halt shutdown hook since it was sometimes causing otherwise 
successful builds to fail. Would be interesting to see test run results.

> UngroupedAggregateRegionObserver.preClose() and doPostScannerOpen() can 
> deadlock
> 
>
> Key: PHOENIX-4131
> URL: https://issues.apache.org/jira/browse/PHOENIX-4131
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Samarth Jain
>Assignee: Samarth Jain
> Attachments: PHOENIX-4131.patch
>
>
> On my local test run I saw that the tests were not completing because the 
> mini cluster couldn't shut down. So I took a jstack and discovered the 
> following deadlock:
> {code}
> "RS:0;samarthjai-wsm4:59006" #16265 prio=5 os_prio=31 tid=0x7fafa6327000 
> nid=0x37b3f runnable [0x7000115f5000]
>java.lang.Thread.State: RUNNABLE
>   at java.lang.Object.wait(Native Method)
>   at 
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.preClose(UngroupedAggregateRegionObserver.java:1201)
>   - locked <0x00072bc406b8> (a java.lang.Object)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$4.call(RegionCoprocessorHost.java:494)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1673)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1749)
>   at 
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.preClose(RegionCoprocessorHost.java:490)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegion(HRegionServer.java:2843)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeRegionIgnoreErrors(HRegionServer.java:2805)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.closeUserRegions(HRegionServer.java:2423)
>   at 
> org.apache.hadoop.hbase.regionserver.HRegionServer.run(HRegionServer.java:1052)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.runRegionServer(MiniHBaseCluster.java:157)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.access$000(MiniHBaseCluster.java:110)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer$1.run(MiniHBaseCluster.java:141)
>   at java.security.AccessController.doPrivileged(Native Method)
>   at javax.security.auth.Subject.doAs(Subject.java:360)
>   at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1637)
>   at 
> org.apache.hadoop.hbase.security.User$SecureHadoopUser.runAs(User.java:334)
>   at 
> org.apache.hadoop.hbase.MiniHBaseCluster$MiniHBaseClusterRegionServer.run(MiniHBaseCluster.java:139)
>   at java.lang.Thread.run(Thread.java:748)
> {code}
> {code}
> "RpcServer.FifoWFPBQ.default.handler=3,queue=0,port=59006" #16246 daemon 
> prio=5 os_prio=31 tid=0x7fafae856000 nid=0x1abdb waiting for monitor 
> entry [0x7000102bc000]
>java.lang.Thread.State: BLOCKED (on object monitor)
>   at 
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:734)
>   - waiting to lock <0x00072bc406b8> (a java.lang.Object)
>   at 
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.overrideDelegate(BaseScannerRegionObserver.java:236)
>   at 
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:281)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2629)
>   - locked <0x00072b625a90> (a 
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder)
>   at 
> org.apache.hadoop.hbase.regionserver.RSRpcServices.scan(RSRpcServices.java:2833)
>   at 
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:34950)
>   at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2339)
>   at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:123)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:188)
>   at 
> org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:168)
> {code}
> 

[jira] [Commented] (PHOENIX-4141) Fix flapping TableSnapshotReadsMapReduceIT

2017-08-30 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148449#comment-16148449
 ] 

Hudson commented on PHOENIX-4141:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1760 (See 
[https://builds.apache.org/job/Phoenix-master/1760/])
PHOENIX-4141 Addendum to fix test failure (samarth: rev 
7d8b8430212fae117ac09faf6b7c22bf673e9073)
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/TableSnapshotReadsMapReduceIT.java


> Fix flapping TableSnapshotReadsMapReduceIT
> --
>
> Key: PHOENIX-4141
> URL: https://issues.apache.org/jira/browse/PHOENIX-4141
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Samarth Jain
>Assignee: Samarth Jain
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4141.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread Lars Hofhansl (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148425#comment-16148425
 ] 

Lars Hofhansl commented on PHOENIX-418:
---

You guys kick a**. :)

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-3815) Only disable indexes on which write failures occurred

2017-08-30 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148368#comment-16148368
 ] 

Hudson commented on PHOENIX-3815:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1759 (See 
[https://builds.apache.org/job/Phoenix-master/1759/])
PHOENIX-3815 Only disable indexes on which write failures occurred (jtaylor: 
rev 7cf7b3abb10fc6a1ace0cb77615f054a8a6378f7)
* (edit) 
phoenix-core/src/test/java/org/apache/phoenix/hbase/index/write/TestParalleIndexWriter.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexFailurePolicy.java
* (delete) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/ParallelWriterIndexCommitter.java
* (edit) 
phoenix-core/src/test/java/org/apache/phoenix/hbase/index/write/TestIndexWriter.java
* (add) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/TrackingParallelWriterIndexCommitter.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/index/MutableIndexFailureIT.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/RecoveryIndexWriter.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/recovery/StoreFailuresInCachePolicy.java
* (edit) 
phoenix-core/src/test/java/org/apache/phoenix/hbase/index/write/TestParalleWriterIndexCommitter.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/IndexWriter.java
* (delete) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/write/recovery/TrackingParallelWriterIndexCommitter.java


> Only disable indexes on which write failures occurred
> -
>
> Key: PHOENIX-3815
> URL: https://issues.apache.org/jira/browse/PHOENIX-3815
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: Vincent Poon
> Fix For: 4.12.0
>
> Attachments: PHOENIX-3815.0.98.v2.patch, 
> PHOENIX-3815.master.v2.patch, PHOENIX-3815.v1.patch
>
>
> We currently disable all indexes if any of them fail to be written to. We 
> really only should disable the one in which the write failed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148366#comment-16148366
 ] 

Hudson commented on PHOENIX-4128:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1759 (See 
[https://builds.apache.org/job/Phoenix-master/1759/])
PHOENIX-4128 CNF org/apache/commons/math3/exception/OutOfRangeException 
(jtaylor: rev c83cec8903fd558777fb4dcd86a8bd67838f46e4)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/parse/ConcreteTableNode.java


> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4128-v1.patch
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148356#comment-16148356
 ] 

Ethan Wang commented on PHOENIX-4128:
-

Thanks for showing me manifestation related to the dependency changes on the 
server side. I'd like to include this test set that you are running as part of 
my future local CI flow. Thanks again! [~mujtabachohan].




> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4128-v1.patch
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148306#comment-16148306
 ] 

Hadoop QA commented on PHOENIX-4128:


{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  http://issues.apache.org/jira/secure/attachment/12884559/PHOENIX-4128-v1.patch
  against master branch at commit e6c1f01c5f7d5c9996017714efe90202a95355b2.
  ATTACHMENT ID: 12884559

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:red}-1 tests included{color}.  The patch doesn't appear to include 
any new or modified tests.
Please justify why no new tests are needed for this 
patch.
Also please list what manual steps were performed to 
verify this patch.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
62 warning messages.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TableSnapshotReadsMapReduceIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GroupByIT

Test results: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1326//testReport/
Javadoc warnings: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1326//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1326//console

This message is automatically generated.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4128-v1.patch
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Reopened] (PHOENIX-3815) Only disable indexes on which write failures occurred

2017-08-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor reopened PHOENIX-3815:
---

[~vincentpoon] - your patch causes the 0.98 and 1.1 branch to hang when running 
the unit tests. Would you mind taking a look? I'm using 1.7.0_80. I've reverted 
you patch for these two branches.

> Only disable indexes on which write failures occurred
> -
>
> Key: PHOENIX-3815
> URL: https://issues.apache.org/jira/browse/PHOENIX-3815
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: Vincent Poon
> Fix For: 4.12.0
>
> Attachments: PHOENIX-3815.0.98.v2.patch, 
> PHOENIX-3815.master.v2.patch, PHOENIX-3815.v1.patch
>
>
> We currently disable all indexes if any of them fail to be written to. We 
> really only should disable the one in which the write failed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Resolved] (PHOENIX-3815) Only disable indexes on which write failures occurred

2017-08-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-3815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor resolved PHOENIX-3815.
---
Resolution: Fixed

Pushed to 4.x and master branches. Thanks for the fix, [~vincentpoon].

> Only disable indexes on which write failures occurred
> -
>
> Key: PHOENIX-3815
> URL: https://issues.apache.org/jira/browse/PHOENIX-3815
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: Vincent Poon
> Fix For: 4.12.0
>
> Attachments: PHOENIX-3815.0.98.v2.patch, 
> PHOENIX-3815.master.v2.patch, PHOENIX-3815.v1.patch
>
>
> We currently disable all indexes if any of them fail to be written to. We 
> really only should disable the one in which the write failed.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148209#comment-16148209
 ] 

James Taylor commented on PHOENIX-4128:
---

+1 on the patch. Thanks, [~aertoria] and [~mujtabachohan]!

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4128-v1.patch
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Mujtaba Chohan (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148188#comment-16148188
 ] 

Mujtaba Chohan commented on PHOENIX-4128:
-

[~aertoria] with patch, things looks good.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4128-v1.patch
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread Hadoop QA (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148175#comment-16148175
 ] 

Hadoop QA commented on PHOENIX-418:
---

{color:red}-1 overall{color}.  Here are the results of testing the latest 
attachment 
  
http://issues.apache.org/jira/secure/attachment/12884517/PHOENIX-418-v6-pom.patch
  against master branch at commit e6c1f01c5f7d5c9996017714efe90202a95355b2.
  ATTACHMENT ID: 12884517

{color:green}+1 @author{color}.  The patch does not contain any @author 
tags.

{color:green}+0 tests included{color}.  The patch appears to be a 
documentation, build,
or dev patch that doesn't require tests.

{color:green}+1 javac{color}.  The applied patch does not increase the 
total number of javac compiler warnings.

{color:red}-1 javadoc{color}.  The javadoc tool appears to have generated 
62 warning messages.

{color:green}+1 release audit{color}.  The applied patch does not increase 
the total number of release audit warnings.

{color:green}+1 lineLengths{color}.  The patch does not introduce lines 
longer than 100

 {color:red}-1 core tests{color}.  The patch failed these unit tests:
 
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexFailureIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.GroupByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TableSnapshotReadsMapReduceIT

Test results: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1325//testReport/
Javadoc warnings: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1325//artifact/patchprocess/patchJavadocWarnings.txt
Console output: 
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1325//console

This message is automatically generated.

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Wang updated PHOENIX-4128:

Attachment: PHOENIX-4128-v1.patch

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4128-v1.patch
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130
 ] 

Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:28 PM:
---

[~jamestaylor]
I meant the later. Since this is the only one place using OutOfRangeException, 
we can use alternative exception so that we will not rely on Math3 dependency. 
I wasn't aware the complexity of adding a extra dependency into phoenix project.
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to:
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");


was (Author: aertoria):
[~jamestaylor]
I meant the later. Since this is the only one place using OutOfRangeException, 
we can use alternative exception so that we will not rely on Math3 dependency.
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to:
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130
 ] 

Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:27 PM:
---

[~jamestaylor]
I meant the later. Since this is the only one place using OutOfRangeException, 
we can use alternative exception so that we will not rely on Math3 dependency.
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to:
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");


was (Author: aertoria):
[~jamestaylor]
I meant the later. Since this is the only one place using OutOfRangeException, 
we can use alternative exception to get rid of this extra dependency.
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to:
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130
 ] 

Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:27 PM:
---

[~jamestaylor]
I meant the later. Since this is the only one place using OutOfRangeException, 
we can use alternative exception to get rid of this extra dependency.
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to:
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");


was (Author: aertoria):
[~jamestaylor]
I meant the later. Since there is only one place using OutofRangeException. We 
can use alternative exception to get rid of this extra dependency.
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to:
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130
 ] 

Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:20 PM:
---

[~jamestaylor]
I meant the later. Since there is only one place using OutofRangeException. We 
can use alternative exception to get rid of this extra dependency.
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to:
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");


was (Author: aertoria):
[~jamestaylor]
I meant the later. Since only there is only one place using 
OutofRangeException. We can change from:
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to:
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130
 ] 

Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 10:19 PM:
---

[~jamestaylor]
I meant the later. Since only there is only one place using 
OutofRangeException. We can change from:
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to:
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");


was (Author: aertoria):
[~jamestaylor]
I meant changing from
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148130#comment-16148130
 ] 

Ethan Wang commented on PHOENIX-4128:
-

[~jamestaylor]
I meant changing from
throw new 
org.apache.commons.math3.exception.OutOfRangeException(tableSamplingRate, 0d, 
100d);
to
throw new IllegalArgumentException("TableSamplingRate is out of bound of 0 and 
100");

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148116#comment-16148116
 ] 

James Taylor commented on PHOENIX-4128:
---

bq.  2. I will remove the use of OutOfRangeException, replace it with 
IllegalArguementException.

Do you mean changing the lib or are you throwing the exception? You'll want to 
just include all pom dependencies instead of changing the lib if the former.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (PHOENIX-4002) Document FETCH NEXT| n ROWS from Cursor

2017-08-30 Thread Biju Nair (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Biju Nair updated PHOENIX-4002:
---
Attachment: PHOENIX-4002-WIP.PATCH

> Document FETCH NEXT| n ROWS from Cursor
> ---
>
> Key: PHOENIX-4002
> URL: https://issues.apache.org/jira/browse/PHOENIX-4002
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Biju Nair
> Attachments: PHOENIX-4002-WIP.PATCH
>
>
> Now that PHOENIX-3572 is resolved and released, we need to add documentation 
> for this new functionality on our website. For directions on how to do that, 
> see http://phoenix.apache.org/building_website.html. I'd recommend adding a 
> new top level page linked off of our Features menu that explains from a users 
> perspective how to use it, and also updating our reference grammar here 
> (which is derived from content in phoenix.csv): 
> http://phoenix.apache.org/language/index.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (PHOENIX-4002) Document FETCH NEXT| n ROWS from Cursor

2017-08-30 Thread Biju Nair (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Biju Nair updated PHOENIX-4002:
---
Attachment: PATCH-4002-WIP.PATCH

Attaching the {{svn}} diff as a patch. Please let me know if there is another 
process that need to be followed for doc patches.

> Document FETCH NEXT| n ROWS from Cursor
> ---
>
> Key: PHOENIX-4002
> URL: https://issues.apache.org/jira/browse/PHOENIX-4002
> Project: Phoenix
>  Issue Type: Sub-task
>Reporter: James Taylor
>Assignee: Biju Nair
> Attachments: PATCH-4002-WIP.PATCH
>
>
> Now that PHOENIX-3572 is resolved and released, we need to add documentation 
> for this new functionality on our website. For directions on how to do that, 
> see http://phoenix.apache.org/building_website.html. I'd recommend adding a 
> new top level page linked off of our Features menu that explains from a users 
> perspective how to use it, and also updating our reference grammar here 
> (which is derived from content in phoenix.csv): 
> http://phoenix.apache.org/language/index.html



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148031#comment-16148031
 ] 

Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 8:51 PM:
--

Correct [~jamestaylor]. Both are being used.
Java.lang.math is being used in TableSamplerPredicate for generating hash
org.apache.common.math3.exception.OutOfRangeException is being used in 
ConcreteTableNode for checking if input tablesample rate is within 0 and 100. 
if not, throw out OutOfRangeException.

After discussed with [~mujtabachohan], two proposal options.
1, include org.apache.common.math3 in to the pom.
2, I will remove the use of OutOfRangeException, replace it with 
IllegalArguementException. (+1)

Meanwhile I'm confirming if hbase 0.98 does not include common math3.


was (Author: aertoria):
Correct [~jamestaylor]. Both are being used.
Java.lang.math is being used in TableSamplerPredicate for generating hash
org.apache.common.math3.exception.OutOfRangeException is being used in 
ConcreteTableNode for checking if input tablesample rate is within 0 and 100. 
if not, throw out OutOfRangeException.

After discussed with [~mujtabachohan], two proposes:
1, include org.apache.common.math3 in to the pom.
2, I will remove the use of OutOfRangeException, replace with 
IllegalArguementException. (+1)

Meanwhile I'm confirming if hbase 0.98 does not include common math3.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148031#comment-16148031
 ] 

Ethan Wang commented on PHOENIX-4128:
-

Correct [~jamestaylor]. Both are being used.
Java.lang.math is being used in TableSamplerPredicate for generating hash
org.apache.common.math3.exception.OutOfRangeException is being used in 
ConcreteTableNode for checking if input tablesample rate is within 0 and 100. 
if not, throw out OutOfRangeException.

After discussed with [~mujtabachohan], two proposes:
1, include org.apache.common.math3 in to the pom.
2, I will remove the use of OutOfRangeException, replace with 
IllegalArguementException. (+1)

Meanwhile I'm confirming if hbase 0.98 does not include common math3.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148022#comment-16148022
 ] 

James Taylor commented on PHOENIX-4128:
---

Might be another manifestation of the issue you fixed with your new patch on 
PHOENIX-418. Can you retest with the patch applied, [~mujtabachohan]? Please go 
ahead and commit that patch.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Assigned] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor reassigned PHOENIX-4128:
-

Assignee: Ethan Wang

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Assignee: Ethan Wang
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16148007#comment-16148007
 ] 

James Taylor commented on PHOENIX-4128:
---

FYI, looks like org.apache.commons.math3 is being used, not java.lang.Math.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (PHOENIX-4144) Query for latest version during index rebuilding

2017-08-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-4144:
--
Attachment: PHOENIX-4144_wip_v2.patch

Actually, I'm not able to repro it. My original repro had an error in the way 
it was manipulating its clock. The 
INDEX_FAILURE_HANDLING_REBUILD_OVERLAP_FORWARD_TIME_ATTRIB which is the overlap 
between when an index is marked INACTIVE versus when the rebuilder starts (by 
default 18ms) should be long enough to prevent this. Maybe if a client is 
stuck in an HBase retry loop for a long time (where the client has the index 
marked as disabled), this could occur. If that's the case, then I'm not sure 
how we can prevent it. Maybe Phoenix needs to manage the retry loops. If a 
client writes to the table while the index is disabled *after* the index 
builder is finished, then the index can get out of sync.

Maybe we can leave the index active and let the write be attempted every time.

> Query for latest version during index rebuilding
> 
>
> Key: PHOENIX-4144
> URL: https://issues.apache.org/jira/browse/PHOENIX-4144
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Attachments: PHOENIX-4144_wip.patch, PHOENIX-4144_wip_v2.patch
>
>
> We're currently setting an upper bound of the scan to find the previous data 
> table row when rebuilding the index. We need to remove this upper bound, 
> though, so that we know if there are more recent versions of a data table row 
> so that we delete the row we've added. We can then process one row past our 
> current timestamp so we don't miss any deletes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Mujtaba Chohan (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147996#comment-16147996
 ] 

Mujtaba Chohan commented on PHOENIX-4128:
-

Correct. This is with 0.98.23.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147992#comment-16147992
 ] 

James Taylor commented on PHOENIX-4128:
---

[~mujtabachohan] is likely testing on 0.98 branch.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147981#comment-16147981
 ] 

Ethan Wang edited comment on PHOENIX-4128 at 8/30/17 8:20 PM:
--

I was not able to reproduce the same error on hbase 1.3.1.

during table sampling hash generating process,  java.lang.Math is used.  I'll 
investigating the error stack to make sure if it need to be also included into 
the server side. Thanks for the info [~mujtabachohan]


was (Author: aertoria):
I was not able to reproduce the same error on hbase 1.3.1.

math3 should not be used during table sampling process. I'll investigating the 
error stack.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147981#comment-16147981
 ] 

Ethan Wang commented on PHOENIX-4128:
-

I was not able to reproduce the same error on hbase 1.3.1.

math3 should not be used during table sampling process. I'll investigating the 
error stack.

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4128) CNF org/apache/commons/math3/exception/OutOfRangeException

2017-08-30 Thread Mujtaba Chohan (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147958#comment-16147958
 ] 

Mujtaba Chohan commented on PHOENIX-4128:
-

[~jamestaylor] It's a drop column statement {{ALTER TABLE T DROP COLUMN C1, C2}}

This happens after this 
[commit|https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commit;h=0135c10754104b8a5515202edf3c78f72323ef18]
 for PHOENIX-153

FYI [~aertoria]

> CNF org/apache/commons/math3/exception/OutOfRangeException 
> ---
>
> Key: PHOENIX-4128
> URL: https://issues.apache.org/jira/browse/PHOENIX-4128
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.12.0
>Reporter: Mujtaba Chohan
>Priority: Minor
> Fix For: 4.12.0
>
>
> {noformat}
> Error: org.apache.hadoop.hbase.DoNotRetryIOException: T: 
> org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.util.ServerUtil.createIOException(ServerUtil.java:92)
> > at 
> > org.apache.phoenix.util.ServerUtil.throwIOException(ServerUtil.java:58)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.mutateColumn(MetaDataEndpointImpl.java:2163)
> > at 
> > org.apache.phoenix.coprocessor.MetaDataEndpointImpl.dropColumn(MetaDataEndpointImpl.java:3243)
> > at 
> > org.apache.phoenix.coprocessor.generated.MetaDataProtos$MetaDataService.callMethod(MetaDataProtos.java:16317)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegion.execService(HRegion.java:6082)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execServiceOnRegion(HRegionServer.java:3533)
> > at 
> > org.apache.hadoop.hbase.regionserver.HRegionServer.execService(HRegionServer.java:3515)
> > at 
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32496)
> > at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2195)
> ...
> > Caused by: java.lang.NoClassDefFoundError: 
> > org/apache/commons/math3/exception/OutOfRangeException
> > at 
> > org.apache.phoenix.parse.ParseNodeFactory.namedTable(ParseNodeFactory.java:404)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.upsert_node(PhoenixSQLParser.java:5239)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.oneStatement(PhoenixSQLParser.java:827)
> > at 
> > org.apache.phoenix.parse.PhoenixSQLParser.statement(PhoenixSQLParser.java:519)
> > at org.apache.phoenix.parse.SQLParser.parseStatement(SQLParser.java:108)
> > at 
> > org.apache.phoenix.jdbc.PhoenixStatement$PhoenixStatementParser.parseStatement(PhoenixStatement.java:1497)
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-4144) Query for latest version during index rebuilding

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147878#comment-16147878
 ] 

James Taylor edited comment on PHOENIX-4144 at 8/30/17 7:24 PM:


Attach WIP patch with test that repros this potential issue. Will work on fix 
soon. FYI, [~vincentpoon].


was (Author: jamestaylor):
Attach WIP patch with test that repros this potential issue. Will work on fix 
soon.

> Query for latest version during index rebuilding
> 
>
> Key: PHOENIX-4144
> URL: https://issues.apache.org/jira/browse/PHOENIX-4144
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Attachments: PHOENIX-4144_wip.patch
>
>
> We're currently setting an upper bound of the scan to find the previous data 
> table row when rebuilding the index. We need to remove this upper bound, 
> though, so that we know if there are more recent versions of a data table row 
> so that we delete the row we've added. We can then process one row past our 
> current timestamp so we don't miss any deletes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (PHOENIX-4144) Query for latest version during index rebuilding

2017-08-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-4144:
--
Attachment: PHOENIX-4144_wip.patch

Attach WIP patch with test that repros this potential issue. Will work on fix 
soon.

> Query for latest version during index rebuilding
> 
>
> Key: PHOENIX-4144
> URL: https://issues.apache.org/jira/browse/PHOENIX-4144
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Attachments: PHOENIX-4144_wip.patch
>
>
> We're currently setting an upper bound of the scan to find the previous data 
> table row when rebuilding the index. We need to remove this upper bound, 
> though, so that we know if there are more recent versions of a data table row 
> so that we delete the row we've added. We can then process one row past our 
> current timestamp so we don't miss any deletes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (PHOENIX-4144) Query for latest version during index rebuilding

2017-08-30 Thread James Taylor (JIRA)
James Taylor created PHOENIX-4144:
-

 Summary: Query for latest version during index rebuilding
 Key: PHOENIX-4144
 URL: https://issues.apache.org/jira/browse/PHOENIX-4144
 Project: Phoenix
  Issue Type: Bug
Reporter: James Taylor
Assignee: James Taylor


We're currently setting an upper bound of the scan to find the previous data 
table row when rebuilding the index. We need to remove this upper bound, 
though, so that we know if there are more recent versions of a data table row 
so that we delete the row we've added. We can then process one row past our 
current timestamp so we don't miss any deletes.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147867#comment-16147867
 ] 

James Taylor commented on PHOENIX-418:
--

+1. Thanks for the quick turnaround, [~aertoria] and for the help 
[~mujtabachohan].

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147826#comment-16147826
 ] 

Ethan Wang edited comment on PHOENIX-418 at 8/30/17 7:03 PM:
-

patch v6-pom, for including dependency 
com.clearspring.analytics.stream into server side. due to my oversight earlier.

Thank you [~mujtabachohan] for helping!


was (Author: aertoria):
patch v6-pom, for including dependency 
com.clearspring.analytics.stream into server side.

Thank you [~mujtabachohan] for helping!

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147826#comment-16147826
 ] 

Ethan Wang edited comment on PHOENIX-418 at 8/30/17 6:51 PM:
-

patch v6-pom, for including dependency 
com.clearspring.analytics.stream into server side.

Thank you [~mujtabachohan] for helping!


was (Author: aertoria):
Thank you [~mujtabachohan] for helping!

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread Ethan Wang (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ethan Wang updated PHOENIX-418:
---
Attachment: PHOENIX-418-v6-pom.patch

Thank you [~mujtabachohan] for helping!

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch, PHOENIX-418-v6-pom.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147704#comment-16147704
 ] 

Ethan Wang edited comment on PHOENIX-418 at 8/30/17 6:22 PM:
-

I see. I forgot to ask during our discussion, for me the local test to work, I 
have to copy both the compiled phoenix-server.jar as well as a stream.jar to 
hbase lib folder. It make sense. I will go prepare another patch


was (Author: aertoria):
I see. I forgot to mention that, for me the local test to work, I have to copy 
both the compiled phoenix-server.jar as well as a stream.jar to hbase lib 
folder. I will go prepare another patch

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147721#comment-16147721
 ] 

James Taylor commented on PHOENIX-418:
--

By updating the assembly of the phoenix server jar in phoenix-assembly, you can 
get the clearspring classes bundled in the jar so that copy isn't necessary.

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread Ethan Wang (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147704#comment-16147704
 ] 

Ethan Wang commented on PHOENIX-418:


I see. I forgot to mention that, for me the local test to work, I have to copy 
both the compiled phoenix-server.jar as well as a stream.jar to hbase lib 
folder. I will go prepare another patch

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4096) Disallow DML operations on connections with CURRENT_SCN set

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147700#comment-16147700
 ] 

James Taylor commented on PHOENIX-4096:
---

There are about 20 more tests derived from BaseClientManagedTimeIT classes that 
need to be converted before we can disallow DML and DDL on connections that set 
CURRENT_SCN. See my patch for ones I've already converted - pretty 
straightforward - just use unique table names and get rid of usage of 
CURRENT_SCN as these tests don't need it. If you can help, then please indicate 
which tests you're converting so we don't duplicate our efforts, 
[~vincentpoon], [~samarthjain], [~aertoria], [~tdsilva].

> Disallow DML operations on connections with CURRENT_SCN set
> ---
>
> Key: PHOENIX-4096
> URL: https://issues.apache.org/jira/browse/PHOENIX-4096
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Attachments: PHOENIX-4096_wip.patch
>
>
> We should make a connection read-only if CURRENT_SCN is set. It's really a 
> bad idea to go back in time and update data and it won't work with secondary 
> indexing, potentially leading to your index and table getting out of sync.
> For testing purposes, where we need to control the timestamp, we should rely 
> on the EnvironmentEdgeManager instead to control the current time.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (PHOENIX-4143) ConcurrentMutationsIT flaps

2017-08-30 Thread Samarth Jain (JIRA)
Samarth Jain created PHOENIX-4143:
-

 Summary: ConcurrentMutationsIT flaps
 Key: PHOENIX-4143
 URL: https://issues.apache.org/jira/browse/PHOENIX-4143
 Project: Phoenix
  Issue Type: Bug
Reporter: Samarth Jain


I have seen sometimes that the test flaps with a RegionOpeningException 
especially on master and 1.x branches. I believe this is likely because we are 
adding a co-processor by doing an admin.modify() call which is asynchronous. So 
it's possible that the region would likely be still opening even though the 
call returned. A potential fix is to poll for the updated table descriptor. Not 
sure if it will always prevent test from flapping (since the problem is similar 
to what Rajesh is solving in PHOENIX-3496) but it should reduce the chances.

FYI, [~jamestaylor]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread James Taylor (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16147690#comment-16147690
 ] 

James Taylor commented on PHOENIX-418:
--

[~aertoria] - please see comment above. Need to get the clearspring library 
into the server jar. This is controlled by phoenix-assembly. [~mujtabachohan], 
[~samarthjain], or [~tdsilva] can help if you run into any issues.

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-418:
-
Priority: Blocker

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>Priority: Blocker
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-418:
-
Fix Version/s: 4.12.0

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Reopened] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor reopened PHOENIX-418:
--

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: Task
>Reporter: James Taylor
>Assignee: Ethan Wang
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread James Taylor (JIRA)

 [ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor updated PHOENIX-418:
-
Issue Type: New Feature  (was: Task)

> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: James Taylor
>Assignee: Ethan Wang
>  Labels: gsoc2016
> Fix For: 4.12.0
>
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Created] (PHOENIX-4142) GEOMETRY data type and OpenGIS functions

2017-08-30 Thread Julian Hyde (JIRA)
Julian Hyde created PHOENIX-4142:


 Summary: GEOMETRY data type and OpenGIS functions
 Key: PHOENIX-4142
 URL: https://issues.apache.org/jira/browse/PHOENIX-4142
 Project: Phoenix
  Issue Type: Bug
Reporter: Julian Hyde


Add a GEOMETRY data type and core OpenGIS functions to Phoenix. This would 
allow Phoenix to execute spatial queries like those that PostGIS can execute.

It would build upon the work done in Calcite in CALCITE-1968. The functions 
would need to be re-implemented (since a Calcite UDF is not a Phoenix UDF) but 
could use the same underlying library.

A further step would be a spatial index - a sorted index on a Hilbert 
coordinate derived from latitude and longitude - and query rewrites to use it. 
See CALCITE-1861.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4138) Create a hard limit on number of indexes per table

2017-08-30 Thread Andrew Purtell (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4138?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146910#comment-16146910
 ] 

Andrew Purtell commented on PHOENIX-4138:
-

That's my point exactly - the client can supply their own value. By definition 
this is not a hard limit. A hard limit is one that cannot be changed by the 
user. A server side mechanism is required to assure the check is always made 
and the set value cannot be changed by a client configuration update or code 
patch. 

> Create a hard limit on number of indexes per table
> --
>
> Key: PHOENIX-4138
> URL: https://issues.apache.org/jira/browse/PHOENIX-4138
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Rahul Shrivastava
>Assignee: Rahul Shrivastava
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> There should be a config parameter to impose a hard limit on number of 
> indexes per table. There is a SQL Exception 
> https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionCode.java#L260
>  , but it gets triggered on the server side  
> (https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java#L1589)
>  . 
> We need a client side limit that can be configured via Phoenix config 
> parameter. Something like if user create more than lets say 30 indexes per 
> table, it would not allow more index creation for the that specific table. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4130) Avoid server retries for mutable indexes

2017-08-30 Thread Andrew Purtell (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146906#comment-16146906
 ] 

Andrew Purtell commented on PHOENIX-4130:
-

Separate JIRA for that sounds ok. 

> Avoid server retries for mutable indexes
> 
>
> Key: PHOENIX-4130
> URL: https://issues.apache.org/jira/browse/PHOENIX-4130
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lars Hofhansl
>
> Had some discussions with [~jamestaylor], [~samarthjain], and [~vincentpoon], 
> during which I suggested that we can possibly eliminate retry loops happening 
> at the server that cause the handler threads to be stuck potentially for 
> quite a while (at least multiple seconds to ride over common scenarios like 
> splits).
> Instead we can do the retries at the Phoenix client that.
> So:
> # The index updates are not retried on the server. (retries = 0)
> # A failed index update would set the failed index timestamp but leave the 
> index enabled.
> # Now the handler thread is done, it throws an appropriate exception back to 
> the client.
> # The Phoenix client can now retry. When those retries fail the index is 
> disabled (if the policy dictates that) and throw the exception back to its 
> caller.
> So no more waiting is needed on the server, handler threads are freed 
> immediately.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4130) Avoid server retries for mutable indexes

2017-08-30 Thread Andrew Purtell (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4130?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146902#comment-16146902
 ] 

Andrew Purtell commented on PHOENIX-4130:
-

The catalog table may be available but jammed with user driven traffic or there 
could be a temporary partial partition long enough. I agree that causes such as 
those will be impacting availability in their own right, but if a design where 
no successful writes are necessary upon index update failures is possible, this 
won't complicate or delay recovery e.g. by holding a handler waiting and 
spinning on a timestamp update. 

> Avoid server retries for mutable indexes
> 
>
> Key: PHOENIX-4130
> URL: https://issues.apache.org/jira/browse/PHOENIX-4130
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lars Hofhansl
>
> Had some discussions with [~jamestaylor], [~samarthjain], and [~vincentpoon], 
> during which I suggested that we can possibly eliminate retry loops happening 
> at the server that cause the handler threads to be stuck potentially for 
> quite a while (at least multiple seconds to ride over common scenarios like 
> splits).
> Instead we can do the retries at the Phoenix client that.
> So:
> # The index updates are not retried on the server. (retries = 0)
> # A failed index update would set the failed index timestamp but leave the 
> index enabled.
> # Now the handler thread is done, it throws an appropriate exception back to 
> the client.
> # The Phoenix client can now retry. When those retries fail the index is 
> disabled (if the policy dictates that) and throw the exception back to its 
> caller.
> So no more waiting is needed on the server, handler threads are freed 
> immediately.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4140) Disable HiveTezIT and HiveMapReduceIT for 4.x-HBase-0.98

2017-08-30 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4140?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146842#comment-16146842
 ] 

Hudson commented on PHOENIX-4140:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1758 (See 
[https://builds.apache.org/job/Phoenix-master/1758/])
Revert "PHOENIX-4140 Disable HiveTezIT and HiveMapReduceIT since they (samarth: 
rev e6c1f01c5f7d5c9996017714efe90202a95355b2)
* (edit) phoenix-hive/src/it/java/org/apache/phoenix/hive/HiveTezIT.java
* (edit) phoenix-hive/src/it/java/org/apache/phoenix/hive/HiveMapReduceIT.java


> Disable HiveTezIT and HiveMapReduceIT for 4.x-HBase-0.98
> 
>
> Key: PHOENIX-4140
> URL: https://issues.apache.org/jira/browse/PHOENIX-4140
> Project: Phoenix
>  Issue Type: Bug
>Reporter: Samarth Jain
>Assignee: Samarth Jain
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4140.patch
>
>
> I have never seen the HiveTezIT and HiveMapReduceIT complete successfully. 
> Locally, on my laptop too, I was unable to get these tests to run 
> successfully. 
> See a sample run where they failed - 
> https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/1647/console
> On my laptop, these tests failed with an OOM. I had to override the permgen 
> memory to 256m to get the tests to even start. 
> FYI, [~sergey.soldatov]



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread Lars Hofhansl (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146707#comment-16146707
 ] 

Lars Hofhansl edited comment on PHOENIX-418 at 8/30/17 6:31 AM:


Shouldn't the clearspring library be part of the server jar? I just did a clean 
build from scratch*, and indeed APPROX_COUNT_DISTINCT fails with
{code}
...
Caused by: java.lang.NoClassDefFoundError: 
com/clearspring/analytics/stream/cardinality/HyperLogLogPlus
...
{code}
thrown at the server.

\* mvn -DskipTests package


was (Author: lhofhansl):
Shouldn't the clearspring library be part of the server jar? I just did a clean 
build from scratch, and indeed APPROX_COUNT_DISTINCT fails with
{code}
...
Caused by: java.lang.NoClassDefFoundError: 
com/clearspring/analytics/stream/cardinality/HyperLogLogPlus
...
{code}
thrown at the server.


> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: Task
>Reporter: James Taylor
>Assignee: Ethan Wang
>  Labels: gsoc2016
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT

2017-08-30 Thread Lars Hofhansl (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146707#comment-16146707
 ] 

Lars Hofhansl commented on PHOENIX-418:
---

Shouldn't the clearspring library be part of the server jar? I just did a clean 
build from scratch, and indeed APPROX_COUNT_DISTINCT fails with
{code}
...
Caused by: java.lang.NoClassDefFoundError: 
com/clearspring/analytics/stream/cardinality/HyperLogLogPlus
...
{code}
thrown at the server.


> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
>  Issue Type: Task
>Reporter: James Taylor
>Assignee: Ethan Wang
>  Labels: gsoc2016
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, 
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, 
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to 
> all distinct values (since this will not scale well when the number of 
> distinct values is huge). The Apache Drill folks have had some interesting 
> discussions on this 
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
>  They recommend using  [Welford's 
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
>  I'm open to having a config option that uses exact versus approximate. I 
> don't have experience implementing an approximate implementation, so I'm not 
> sure how much state is required to keep on the server and return to the 
> client (other than realizing it'd be much less that returning all distinct 
> values and their counts).
> Update:
> Syntax of using approximate count distinct as:
> select APPROX_COUNT_DISTINCT(name) from person
> select APPROX_COUNT_DISTINCT(address||name) from person
> It is equivalent of  Select COUNT(DISTINCT ID) from person. Implemented using 
> hyperloglog, see discuss below.
> Source code patch link below, co-authorred with [~swapna]
> https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=d6381afc3af976ccdbb874d4458ea17b1e8a1d32



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)


[jira] [Commented] (PHOENIX-4080) The error message for version mismatch is not accurate.

2017-08-30 Thread Hudson (JIRA)

[ 
https://issues.apache.org/jira/browse/PHOENIX-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146674#comment-16146674
 ] 

Hudson commented on PHOENIX-4080:
-

FAILURE: Integrated in Jenkins build Phoenix-master #1757 (See 
[https://builds.apache.org/job/Phoenix-master/1757/])
PHOENIX-4080 Amending error message when phoenix client version is not 
(gjacoby: rev 74762df7a5649592ad81ca4ad72425acc2b0a823)
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java


> The error message for version mismatch is not accurate.
> ---
>
> Key: PHOENIX-4080
> URL: https://issues.apache.org/jira/browse/PHOENIX-4080
> Project: Phoenix
>  Issue Type: Wish
>Affects Versions: 4.11.0
>Reporter: Ethan Wang
>Assignee: Ethan Wang
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4080.patch, PHOENIX-4080-v2.patch, 
> PHOENIX-4080-v3.patch
>
>
> When accessing a 4.10 running cluster with 4.11 client, it referred as 
> The following servers require an updated phoenix.jar to be put in the 
> classpath of HBase: region=SYSTEM.CATALOG
> It should be phoenix-[version]-server.jar rather than phoenix.jar



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)