[jira] [Commented] (PHOENIX-153) Implement TABLESAMPLE clause
[ https://issues.apache.org/jira/browse/PHOENIX-153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141232#comment-16141232 ] Lars Hofhansl commented on PHOENIX-153: --- Nice job [~aertoria]! > Implement TABLESAMPLE clause > > > Key: PHOENIX-153 > URL: https://issues.apache.org/jira/browse/PHOENIX-153 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: enhancement > Fix For: 4.12.0 > > Attachments: Sampling_Accuracy_Performance.jpg > > > Support the standard SQL TABLESAMPLE clause by implementing a filter that > uses a skip next hint based on the region boundaries of the table to only > return n rows per region. > [Update] > Patch: > https://git-wip-us.apache.org/repos/asf?p=phoenix.git;a=commitdiff;h=5e33dc12bc088bd0008d89f0a5cd7d5c368efa25 -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141219#comment-16141219 ] James Taylor commented on PHOENIX-418: -- Thanks for the info, [~elserj]. So we should add that text to our NOTICE file, right? > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Wang updated PHOENIX-418: --- Attachment: PHOENIX-418-v6.patch > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Wang updated PHOENIX-418: --- Attachment: (was: PHOENIX-418-v6.patch) > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141126#comment-16141126 ] Ethan Wang commented on PHOENIX-418: Thanks [~elserj]. 2.9.5 should be equally just fine. > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT
[
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141092#comment-16141092
]
Ethan Wang edited comment on PHOENIX-418 at 8/25/17 3:10 AM:
-
Local run was OK (a few IT fails, re run OK, like usual). I'll monitor the
pre-commit run for v6 patch
(locally running on java version "1.8.0_102" with
{code:java}
mvn -fae install -DnumForkedIT=3
{code})
was (Author: aertoria):
Local run was OK (a few IT fails, re run OK, like usual). I'll monitor the
pre-commit run for v6 patch
Running on java version "1.8.0_102" with
{code:java}
mvn -fae install -DnumForkedIT=3
{code}
> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
> Issue Type: Task
>Reporter: James Taylor
>Assignee: Ethan Wang
> Labels: gsoc2016
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch,
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch,
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to
> all distinct values (since this will not scale well when the number of
> distinct values is huge). The Apache Drill folks have had some interesting
> discussions on this
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
> They recommend using [Welford's
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
> I'm open to having a config option that uses exact versus approximate. I
> don't have experience implementing an approximate implementation, so I'm not
> sure how much state is required to keep on the server and return to the
> client (other than realizing it'd be much less that returning all distinct
> values and their counts).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141100#comment-16141100
]
Josh Elser commented on PHOENIX-418:
[~jamestaylor], thanks for thinking about the licensing! Turns out, there is
something that we should propagate in the binary tarball
https://github.com/addthis/stream-lib/blob/master/NOTICE.txt
Specifically, we should include text, something like:
{noformat}
stream-lib
Copyright 2016 AddThis
This product includes software developed by AddThis.
{noformat}
Aside, it appears that there is a 2.9.5 version of
{{com.clearspring.analytics:stream}} in maven-central. Is there a reason for us
to use 2.7.0, specifically?
Neat little change, Ethan. I look forward to playing around with the feature :)
> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
> Issue Type: Task
>Reporter: James Taylor
>Assignee: Ethan Wang
> Labels: gsoc2016
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch,
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch,
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to
> all distinct values (since this will not scale well when the number of
> distinct values is huge). The Apache Drill folks have had some interesting
> discussions on this
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
> They recommend using [Welford's
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
> I'm open to having a config option that uses exact versus approximate. I
> don't have experience implementing an approximate implementation, so I'm not
> sure how much state is required to keep on the server and return to the
> client (other than realizing it'd be much less that returning all distinct
> values and their counts).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT
[
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141092#comment-16141092
]
Ethan Wang edited comment on PHOENIX-418 at 8/25/17 3:03 AM:
-
Local run was OK (a few IT fails, re run OK, like usual). I'll monitor the
pre-commit run for v6 patch
Running on java version "1.8.0_102" with
{code:java}
mvn -fae install -DnumForkedIT=3
{code}
was (Author: aertoria):
Local run was OK (a few IT fails, re run OK, like usual). I'll monitor the
pre-commit run for v6 patch
> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
> Issue Type: Task
>Reporter: James Taylor
>Assignee: Ethan Wang
> Labels: gsoc2016
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch,
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch,
> PHOENIX-418-v6.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to
> all distinct values (since this will not scale well when the number of
> distinct values is huge). The Apache Drill folks have had some interesting
> discussions on this
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
> They recommend using [Welford's
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
> I'm open to having a config option that uses exact versus approximate. I
> don't have experience implementing an approximate implementation, so I'm not
> sure how much state is required to keep on the server and return to the
> client (other than realizing it'd be much less that returning all distinct
> values and their counts).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Wang updated PHOENIX-418: --- Attachment: PHOENIX-418-v6.patch > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Wang updated PHOENIX-418: --- Attachment: (was: PHOENIX-418-v6.patch) > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Wang updated PHOENIX-418: --- Attachment: PHOENIX-418-v6.patch Local run was OK (a few IT fails, re run OK, like usual). I'll monitor the pre-commit run for v6 patch > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch, > PHOENIX-418-v6.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-3953) Clear INDEX_DISABLED_TIMESTAMP and disable index on compaction
[
https://issues.apache.org/jira/browse/PHOENIX-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141040#comment-16141040
]
Hadoop QA commented on PHOENIX-3953:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12883623/PHOENIX-3953.patch
against master branch at commit dd008f4fe4e67836a6fb362843c9618e4e518182.
ATTACHMENT ID: 12883623
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
56 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 2 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+conn.createStatement().execute("CREATE TABLE " + fullTableName
+ "(k INTEGER PRIMARY KEY, v1 INTEGER, v2 INTEGER) COLUMN_ENCODED_BYTES = 0,
STORE_NULLS=true");
+conn.createStatement().execute("CREATE INDEX " + indexName + " ON
" + fullTableName + " (v1) INCLUDE (v2)");
+HTableInterface metaTable =
conn.unwrap(PhoenixConnection.class).getQueryServices().getTable(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES);
+boolean guidepostOnStatement = guidePostPerRegionBytes != null ||
guidePostWidthBytes != null;
+TableName htableName =
SchemaUtil.getPhysicalTableName(PhoenixDatabaseMetaData.SYSTEM_CATALOG_NAME_BYTES,
env.getConfiguration());
+get.addColumn(PhoenixDatabaseMetaData.TABLE_FAMILY_BYTES,
PhoenixDatabaseMetaData.INDEX_DISABLE_TIMESTAMP_BYTES);
+Cell gpwCell =
result.getColumnLatestCell(PhoenixDatabaseMetaData.TABLE_FAMILY_BYTES,
PhoenixDatabaseMetaData.GUIDE_POSTS_WIDTH_BYTES);
+guidepostWidth =
PLong.INSTANCE.getCodec().decodeLong(gpwCell.getValueArray(),
gpwCell.getValueOffset(), SortOrder.getDefault());
+Cell idtsCell =
result.getColumnLatestCell(PhoenixDatabaseMetaData.TABLE_FAMILY_BYTES,
PhoenixDatabaseMetaData.INDEX_DISABLE_TIMESTAMP_BYTES);
+long indexDisableTimestamp =
PLong.INSTANCE.getCodec().decodeLong(idtsCell.getValueArray(),
idtsCell.getValueOffset(), SortOrder.getDefault());
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.FlappingTransactionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CreateTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexReplicationIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.RollbackIT
{color:red}-1 core zombie tests{color}. There are 1 zombie test(s):
at
org.apache.activemq.artemis.tests.integration.largemessage.LargeMessageTestBase.testChunks(LargeMessageTestBase.java:342)
at
org.apache.activemq.artemis.tests.integration.largemessage.LargeMessageTestBase.testChunks(LargeMessageTestBase.java:109)
at
org.apache.activemq.artemis.tests.integration.client.LargeMessageTest.testFilePersistenceDelayedXAConsumer(LargeMessageTest.java:1205)
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1296//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1296//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1296//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1296//console
This message is automatically generated.
> Clear INDEX_DISABLED_TIMESTAMP and disable index on compaction
> --
>
> Key: PHOENIX-3953
> URL: https://issues.apache.org/jira/browse/PHOENIX-3953
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Labels: globalMutableSecondaryIndex
> Fix For: 4.12.0
>
> Attachments: PHOENIX-3953.patch
>
>
> To guard against a compaction occurring (which would potentially clear delete
> markers and puts that the partial index rebuild process counts on to properly
> catch up an index with the data table), we should clear the
>
[jira] [Closed] (PHOENIX-4120) Can't get the records by select when the operator is "!=" and the specified column value is null
[
https://issues.apache.org/jira/browse/PHOENIX-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
alexBai closed PHOENIX-4120.
> Can't get the records by select when the operator is "!=" and the specified
> column value is null
> --
>
> Key: PHOENIX-4120
> URL: https://issues.apache.org/jira/browse/PHOENIX-4120
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
> Environment: phoenix-4.8.1-HBase-1.0-SNAPSHOT-server.jar
>Reporter: alexBai
> Labels: null-values
>
> {code:java}
> 0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
> BIGINT, constraint pk primary key("id"));
> 2 rows affected (1.257 seconds)
> 0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
> 1 row affected (0.071 seconds)
> 0: jdbc:phoenix:> upsert into alex values(2, null, 28);
> 1 row affected (0.012 seconds)
> 0: jdbc:phoenix:> select * from alex;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 1 | alex | 28|
> | 2 | | 28|
> +-+---+--+
> 2 rows selected (0.063 seconds)
> 0: jdbc:phoenix:> select * from alex where "name"!='alex';
> +-+---+--+
> | id | name | age |
> +-+---+--+
> +-+---+--+
> No rows selected (0.053 seconds)
> 0: jdbc:phoenix:> select * from alex where "name" is null;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 2 | | 28 |
> +-+---+--+
> {code}
> Does phoenix just design like that?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141019#comment-16141019 ] James Taylor commented on PHOENIX-418: -- One more thing: are any changes required to phoenix-assembly for the new jar, [~elserj] or [~mujtabachohan]? Might want to double check the license dependencies too, Josh. They looked ok to me. > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141013#comment-16141013 ] James Taylor commented on PHOENIX-418: -- +1. I'll remove CountDistinctApproximateHyperLogLogStatsEnabledIT as I don't think we need that. Your local test runs look ok, [~aertoria]? > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] phoenix issue #269: PHOENIX-2460 Implement scrutiny command to validate whet...
Github user vincentpoon commented on the issue: https://github.com/apache/phoenix/pull/269 @JamesRTaylor thanks for the comments, I now also write the timestamp from the cell to the output table when there's a mismatch. Also fixed the nits and use md5. Ran fine on a cluster with 800m rows. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (PHOENIX-2460) Implement scrutiny command to validate whether or not an index is in sync with the data table
[ https://issues.apache.org/jira/browse/PHOENIX-2460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16141008#comment-16141008 ] ASF GitHub Bot commented on PHOENIX-2460: - Github user vincentpoon commented on the issue: https://github.com/apache/phoenix/pull/269 @JamesRTaylor thanks for the comments, I now also write the timestamp from the cell to the output table when there's a mismatch. Also fixed the nits and use md5. Ran fine on a cluster with 800m rows. > Implement scrutiny command to validate whether or not an index is in sync > with the data table > - > > Key: PHOENIX-2460 > URL: https://issues.apache.org/jira/browse/PHOENIX-2460 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: Vincent Poon > Attachments: PHOENIX-2460.patch > > > We should have a process that runs to verify that an index is valid against a > data table and potentially fixes it if discrepancies are found. This could > either be a MR job or a low priority background task. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4124) Two new tests without Apache licenses
[ https://issues.apache.org/jira/browse/PHOENIX-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140969#comment-16140969 ] Geoffrey Jacoby commented on PHOENIX-4124: -- +1, thanks! > Two new tests without Apache licenses > - > > Key: PHOENIX-4124 > URL: https://issues.apache.org/jira/browse/PHOENIX-4124 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Geoffrey Jacoby >Assignee: James Taylor >Priority: Blocker > Fix For: 4.12.0 > > Attachments: PHOENIX-4124.patch, PHOENIX-4124_v2.patch > > > It appears that new test files, IndexScrutinyIT and RunUntilFailure, were > recently checked in without Apache licenses. This is causing the release > audit jenkins job to fail on unrelated JIRAs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4124) Two new tests without Apache licenses
[ https://issues.apache.org/jira/browse/PHOENIX-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4124: -- Attachment: PHOENIX-4124_v2.patch > Two new tests without Apache licenses > - > > Key: PHOENIX-4124 > URL: https://issues.apache.org/jira/browse/PHOENIX-4124 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Geoffrey Jacoby >Assignee: James Taylor >Priority: Blocker > Fix For: 4.12.0 > > Attachments: PHOENIX-4124.patch, PHOENIX-4124_v2.patch > > > It appears that new test files, IndexScrutinyIT and RunUntilFailure, were > recently checked in without Apache licenses. This is causing the release > audit jenkins job to fail on unrelated JIRAs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4124) Two new tests without Apache licenses
[ https://issues.apache.org/jira/browse/PHOENIX-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140957#comment-16140957 ] James Taylor commented on PHOENIX-4124: --- Oops. Let me fix that. > Two new tests without Apache licenses > - > > Key: PHOENIX-4124 > URL: https://issues.apache.org/jira/browse/PHOENIX-4124 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Geoffrey Jacoby >Assignee: James Taylor >Priority: Blocker > Fix For: 4.12.0 > > Attachments: PHOENIX-4124.patch > > > It appears that new test files, IndexScrutinyIT and RunUntilFailure, were > recently checked in without Apache licenses. This is causing the release > audit jenkins job to fail on unrelated JIRAs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4124) Two new tests without Apache licenses
[ https://issues.apache.org/jira/browse/PHOENIX-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140947#comment-16140947 ] Geoffrey Jacoby commented on PHOENIX-4124: -- [~jamestaylor] - the license is now between the imports and the class declaration, and I think in most files it goes before the imports. I'm sure that's legally sufficient either way, but will that confuse the Jenkins job? > Two new tests without Apache licenses > - > > Key: PHOENIX-4124 > URL: https://issues.apache.org/jira/browse/PHOENIX-4124 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Geoffrey Jacoby >Assignee: James Taylor >Priority: Blocker > Fix For: 4.12.0 > > Attachments: PHOENIX-4124.patch > > > It appears that new test files, IndexScrutinyIT and RunUntilFailure, were > recently checked in without Apache licenses. This is causing the release > audit jenkins job to fail on unrelated JIRAs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-4124) Two new tests without Apache licenses
[ https://issues.apache.org/jira/browse/PHOENIX-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-4124: -- Attachment: PHOENIX-4124.patch > Two new tests without Apache licenses > - > > Key: PHOENIX-4124 > URL: https://issues.apache.org/jira/browse/PHOENIX-4124 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Geoffrey Jacoby >Assignee: James Taylor >Priority: Blocker > Fix For: 4.12.0 > > Attachments: PHOENIX-4124.patch > > > It appears that new test files, IndexScrutinyIT and RunUntilFailure, were > recently checked in without Apache licenses. This is causing the release > audit jenkins job to fail on unrelated JIRAs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4122) Prevent IllegalMonitorStateException from occurring when releasing row locks
[
https://issues.apache.org/jira/browse/PHOENIX-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140911#comment-16140911
]
Hadoop QA commented on PHOENIX-4122:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12883606/PHOENIX-4122.patch
against master branch at commit dd008f4fe4e67836a6fb362843c9618e4e518182.
ATTACHMENT ID: 12883606
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
56 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 2 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+ public List rowLocks =
Lists.newArrayListWithExpectedSize(QueryServicesOptions.DEFAULT_MUTATE_BATCH_SIZE);
+ private void
setBatchMutateContext(ObserverContext c,
BatchMutateContext context) {
+ private BatchMutateContext
getBatchMutateContext(ObserverContext c) {
+ private void doPost(ObserverContext c,
BatchMutateContext context) throws IOException {
+ private void
doPostWithExceptions(ObserverContext c,
BatchMutateContext context)
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MutableQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AggregateQueryIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1295//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1295//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1295//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1295//console
This message is automatically generated.
> Prevent IllegalMonitorStateException from occurring when releasing row locks
>
>
> Key: PHOENIX-4122
> URL: https://issues.apache.org/jira/browse/PHOENIX-4122
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4122.patch
>
>
> In this iteration, after RS process was killed, Indexes were disabled and
> soon its status was changed to INACTIVE state.
> After all regions were re-assigned, we observed that two RS were aborted with
> below exception in logs
> {code}
> org.apache.phoenix.hbase.index.Indexer threw
> java.lang.IllegalMonitorStateException
> java.lang.IllegalMonitorStateException
> at
> java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:151)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1261)
> at
> java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
> at
> org.apache.phoenix.hbase.index.LockManager$RowLockImpl.release(LockManager.java:223)
> at
> org.apache.phoenix.hbase.index.LockManager$RowLockContext.releaseRowLock(LockManager.java:166)
> at
> org.apache.phoenix.hbase.index.LockManager.unlockRow(LockManager.java:131)
> at
> org.apache.phoenix.hbase.index.Indexer.postBatchMutateIndispensably(Indexer.java:590)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$37.call(RegionCoprocessorHost.java:1023)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1621)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1697)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1653)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postBatchMutateIndispensably(RegionCoprocessorHost.java:1019)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2837)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2410)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2365)
>
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140890#comment-16140890
]
Hadoop QA commented on PHOENIX-418:
---
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12883574/PHOENIX-418-v5.patch
against master branch at commit dd008f4fe4e67836a6fb362843c9618e4e518182.
ATTACHMENT ID: 12883574
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
56 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 2 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+String query = "SELECT
APPROX_COUNT_DISTINCT(ORGANIZATION_ID||A_STRING||B_STRING) FROM " + tableName;
+String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+String query = "SELECT
APPROX_COUNT_DISTINCT(a.ORGANIZATION_ID||a.A_STRING||b.ENTITY_ID) FROM "
+ + tableName + " a, "+ tableName + " b where
a.ORGANIZATION_ID=b.ORGANIZATION_ID and a.ENTITY_ID = b.ENTITY_ID";
+String tableName = initATableValues(null, tenantId,
getDefaultSplits(tenantId), (Date)null, null, getUrl(), null);
+String query = "explain SELECT
APPROX_COUNT_DISTINCT(ORGANIZATION_ID||ENTITY_ID) FROM " + tableName;
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.salted.SaltedTableVarLengthRowKeyIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DerivedTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SequenceIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CustomEntityDataIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TransactionalViewIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.MutableIndexReplicationIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CreateTableIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.OrderByIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.FirstValuesFunctionIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StatementHintsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RowValueConstructorIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.hadoop.hbase.regionserver.wal.WALRecoveryRegionPostOpenIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PowerFunctionEnd2EndIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.InListIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.IsNullIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.txn.RollbackIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.StatsCollectorIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DynamicUpsertIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryExecWithoutSCNIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.SubqueryUsingSortMergeJoinIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.RenewLeaseIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.ChildViewsUseParentViewIndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.TopNIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.EvaluationOfORIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.CsvBulkLoadToolIT
[jira] [Commented] (PHOENIX-4124) Two new tests without Apache licenses
[ https://issues.apache.org/jira/browse/PHOENIX-4124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140881#comment-16140881 ] Geoffrey Jacoby commented on PHOENIX-4124: -- [~jamestaylor], [~vincentpoon], fyi. > Two new tests without Apache licenses > - > > Key: PHOENIX-4124 > URL: https://issues.apache.org/jira/browse/PHOENIX-4124 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.12.0 >Reporter: Geoffrey Jacoby >Assignee: James Taylor >Priority: Blocker > Fix For: 4.12.0 > > > It appears that new test files, IndexScrutinyIT and RunUntilFailure, were > recently checked in without Apache licenses. This is causing the release > audit jenkins job to fail on unrelated JIRAs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (PHOENIX-4124) Two new tests without Apache licenses
Geoffrey Jacoby created PHOENIX-4124: Summary: Two new tests without Apache licenses Key: PHOENIX-4124 URL: https://issues.apache.org/jira/browse/PHOENIX-4124 Project: Phoenix Issue Type: Bug Affects Versions: 4.12.0 Reporter: Geoffrey Jacoby Assignee: James Taylor Priority: Blocker Fix For: 4.12.0 It appears that new test files, IndexScrutinyIT and RunUntilFailure, were recently checked in without Apache licenses. This is causing the release audit jenkins job to fail on unrelated JIRAs. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
Re: RC for 4.12.0
Hi Jasper, Sure, I'd be happy to commit that patch. Would you mind attaching a rebased patch as the current one doesn't apply to master. Thanks, James On Thu, Aug 24, 2017 at 5:08 AM, jasper.van...@baesystems.com < jasper.van...@baesystems.com> wrote: > Hi James, > > Could you take a second look at https://issues.apache.org/ > jira/browse/PHOENIX-2314 > It have been pushed forward since 4.9.0 I believe. > The fix is simple, so we've been patching ourselves, but it would be great > if this could officially make 4.12.0. > (the last results was that it failed 1 core test) > > Kind regards, > > Jasper. > > -Original Message- > From: James Taylor [mailto:jamestay...@apache.org] > Sent: 24 August 2017 00:51 > To: dev@phoenix.apache.org > Subject: RC for 4.12.0 > > ATTENTION. This message originates from outside BAE Systems. > > I'd like to target a 4.12 RC by the end of next week. Any objections? > There are many fixes for global mutable secondary indexing and some for > local indexing as well. As far as outstanding JIRAs, I know of PHOENIX-4010 > and probably PHOENIX-418. I'd like to get in PHOENIX-3953, PHOENIX-3815, > and maybe PHOENIX-4096. There's also work underway to stabilize our test > runs which would be good to get in. > > Anything else? > > Thanks, > James > Please consider the environment before printing this email. This message > should be regarded as confidential. If you have received this email in > error please notify the sender and destroy it immediately. Statements of > intent shall only become binding when confirmed in hard copy by an > authorised signatory. The contents of this email may relate to dealings > with other companies under the control of BAE Systems PLC, details of which > can be found at http://www.baesystems.com/Businesses/index.htm. >
[jira] [Updated] (PHOENIX-3953) Clear INDEX_DISABLED_TIMESTAMP and disable index on compaction
[ https://issues.apache.org/jira/browse/PHOENIX-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor updated PHOENIX-3953: -- Attachment: PHOENIX-3953.patch Please review, [~samarthjain]. > Clear INDEX_DISABLED_TIMESTAMP and disable index on compaction > -- > > Key: PHOENIX-3953 > URL: https://issues.apache.org/jira/browse/PHOENIX-3953 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Labels: globalMutableSecondaryIndex > Fix For: 4.12.0 > > Attachments: PHOENIX-3953.patch > > > To guard against a compaction occurring (which would potentially clear delete > markers and puts that the partial index rebuild process counts on to properly > catch up an index with the data table), we should clear the > INDEX_DISABLED_TIMESTAMP and mark the index as disabled. This could be done > in the post compaction coprocessor hook. At this point, a manual rebuild of > the index would be required. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Assigned] (PHOENIX-3953) Clear INDEX_DISABLED_TIMESTAMP and disable index on compaction
[ https://issues.apache.org/jira/browse/PHOENIX-3953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor reassigned PHOENIX-3953: - Assignee: James Taylor > Clear INDEX_DISABLED_TIMESTAMP and disable index on compaction > -- > > Key: PHOENIX-3953 > URL: https://issues.apache.org/jira/browse/PHOENIX-3953 > Project: Phoenix > Issue Type: Bug >Reporter: James Taylor >Assignee: James Taylor > Labels: globalMutableSecondaryIndex > Fix For: 4.12.0 > > > To guard against a compaction occurring (which would potentially clear delete > markers and puts that the partial index rebuild process counts on to properly > catch up an index with the data table), we should clear the > INDEX_DISABLED_TIMESTAMP and mark the index as disabled. This could be done > in the post compaction coprocessor hook. At this point, a manual rebuild of > the index would be required. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (PHOENIX-4080) The error message for version mismatch is not accurate.
[
https://issues.apache.org/jira/browse/PHOENIX-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140852#comment-16140852
]
Hadoop QA commented on PHOENIX-4080:
{color:red}-1 overall{color}. Here are the results of testing the latest
attachment
http://issues.apache.org/jira/secure/attachment/12883587/PHOENIX-4080-v2.patch
against master branch at commit dd008f4fe4e67836a6fb362843c9618e4e518182.
ATTACHMENT ID: 12883587
{color:green}+1 @author{color}. The patch does not contain any @author
tags.
{color:red}-1 tests included{color}. The patch doesn't appear to include
any new or modified tests.
Please justify why no new tests are needed for this
patch.
Also please list what manual steps were performed to
verify this patch.
{color:green}+1 javac{color}. The applied patch does not increase the
total number of javac compiler warnings.
{color:red}-1 javadoc{color}. The javadoc tool appears to have generated
56 warning messages.
{color:red}-1 release audit{color}. The applied patch generated 2 release
audit warnings (more than the master's current 0 warnings).
{color:red}-1 lineLengths{color}. The patch introduces the following lines
longer than 100:
+StringBuilder buf = new StringBuilder("Newer Phoenix clients can't
communicate with older Phoenix servers. The following servers require an
updated " + QueryConstants.DEFAULT_COPROCESS_JAR_NAME + " to be put in the
classpath of HBase: ");
+.setMessage("Ensure that " +
QueryConstants.DEFAULT_COPROCESS_JAR_NAME + " is put on the classpath of HBase
in every region server: " + t.getMessage())
{color:red}-1 core tests{color}. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.IndexIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.tx.TxCheckpointIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.MutableQueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.QueryIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.DropSchemaIT
Test results:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1294//testReport/
Release audit warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1294//artifact/patchprocess/patchReleaseAuditWarnings.txt
Javadoc warnings:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1294//artifact/patchprocess/patchJavadocWarnings.txt
Console output:
https://builds.apache.org/job/PreCommit-PHOENIX-Build/1294//console
This message is automatically generated.
> The error message for version mismatch is not accurate.
> ---
>
> Key: PHOENIX-4080
> URL: https://issues.apache.org/jira/browse/PHOENIX-4080
> Project: Phoenix
> Issue Type: Wish
>Affects Versions: 4.11.0
>Reporter: Ethan Wang
>Assignee: Ethan Wang
> Attachments: PHOENIX-4080.patch, PHOENIX-4080-v2.patch
>
>
> When accessing a 4.10 running cluster with 4.11 client, it referred as
> The following servers require an updated phoenix.jar to be put in the
> classpath of HBase: region=SYSTEM.CATALOG
> It should be phoenix-[version]-server.jar rather than phoenix.jar
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Resolved] (PHOENIX-4123) LockManager references org.apache.htrace in Phoenix 0.98 branch
[
https://issues.apache.org/jira/browse/PHOENIX-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor resolved PHOENIX-4123.
---
Resolution: Fixed
Strange that it compiled at all, but fixed now.
> LockManager references org.apache.htrace in Phoenix 0.98 branch
> ---
>
> Key: PHOENIX-4123
> URL: https://issues.apache.org/jira/browse/PHOENIX-4123
> Project: Phoenix
> Issue Type: Bug
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4123.patch
>
>
> Should be replaced with org.cloudera.htrace as this causes CDNF in 0.98
> branch:
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace
> at
> org.apache.phoenix.hbase.index.LockManager.lockRow(LockManager.java:68)
> at
> org.apache.phoenix.hbase.index.Indexer.preBatchMutateWithExceptions(Indexer.java:415)
> at
> org.apache.phoenix.hbase.index.Indexer.preBatchMutate(Indexer.java:352)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4123) LockManager references org.apache.htrace in Phoenix 0.98 branch
[
https://issues.apache.org/jira/browse/PHOENIX-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4123:
--
Attachment: PHOENIX-4123.patch
> LockManager references org.apache.htrace in Phoenix 0.98 branch
> ---
>
> Key: PHOENIX-4123
> URL: https://issues.apache.org/jira/browse/PHOENIX-4123
> Project: Phoenix
> Issue Type: Bug
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4123.patch
>
>
> Should be replaced with org.cloudera.htrace as this causes CDNF in 0.98
> branch:
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace
> at
> org.apache.phoenix.hbase.index.LockManager.lockRow(LockManager.java:68)
> at
> org.apache.phoenix.hbase.index.Indexer.preBatchMutateWithExceptions(Indexer.java:415)
> at
> org.apache.phoenix.hbase.index.Indexer.preBatchMutate(Indexer.java:352)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4123) LockManager references org.apache.htrace in Phoenix 0.98 branch
[
https://issues.apache.org/jira/browse/PHOENIX-4123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4123:
--
Priority: Minor (was: Major)
> LockManager references org.apache.htrace in Phoenix 0.98 branch
> ---
>
> Key: PHOENIX-4123
> URL: https://issues.apache.org/jira/browse/PHOENIX-4123
> Project: Phoenix
> Issue Type: Bug
>Reporter: Mujtaba Chohan
>Assignee: James Taylor
>Priority: Minor
> Fix For: 4.12.0
>
>
> Should be replaced with org.cloudera.htrace as this causes CDNF in 0.98
> branch:
> {noformat}
> Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace
> at
> org.apache.phoenix.hbase.index.LockManager.lockRow(LockManager.java:68)
> at
> org.apache.phoenix.hbase.index.Indexer.preBatchMutateWithExceptions(Indexer.java:415)
> at
> org.apache.phoenix.hbase.index.Indexer.preBatchMutate(Indexer.java:352)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (PHOENIX-4123) Minor: LogManager references org.apache.htrace in Phoenix 0.98 branch
Mujtaba Chohan created PHOENIX-4123:
---
Summary: Minor: LogManager references org.apache.htrace in Phoenix
0.98 branch
Key: PHOENIX-4123
URL: https://issues.apache.org/jira/browse/PHOENIX-4123
Project: Phoenix
Issue Type: Bug
Reporter: Mujtaba Chohan
Assignee: James Taylor
Should be replaced with org.cloudera.htrace as this causes CDNF in 0.98 branch:
{noformat}
Caused by: java.lang.NoClassDefFoundError: org/apache/htrace/Trace
at
org.apache.phoenix.hbase.index.LockManager.lockRow(LockManager.java:68)
at
org.apache.phoenix.hbase.index.Indexer.preBatchMutateWithExceptions(Indexer.java:415)
at
org.apache.phoenix.hbase.index.Indexer.preBatchMutate(Indexer.java:352)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4122) Prevent IllegalMonitorStateException from occurring when releasing row locks
[
https://issues.apache.org/jira/browse/PHOENIX-4122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor updated PHOENIX-4122:
--
Attachment: PHOENIX-4122.patch
Please review, [~tdsilva] or [~vincentpoon].
> Prevent IllegalMonitorStateException from occurring when releasing row locks
>
>
> Key: PHOENIX-4122
> URL: https://issues.apache.org/jira/browse/PHOENIX-4122
> Project: Phoenix
> Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
> Fix For: 4.12.0
>
> Attachments: PHOENIX-4122.patch
>
>
> In this iteration, after RS process was killed, Indexes were disabled and
> soon its status was changed to INACTIVE state.
> After all regions were re-assigned, we observed that two RS were aborted with
> below exception in logs
> {code}
> org.apache.phoenix.hbase.index.Indexer threw
> java.lang.IllegalMonitorStateException
> java.lang.IllegalMonitorStateException
> at
> java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:151)
> at
> java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1261)
> at
> java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
> at
> org.apache.phoenix.hbase.index.LockManager$RowLockImpl.release(LockManager.java:223)
> at
> org.apache.phoenix.hbase.index.LockManager$RowLockContext.releaseRowLock(LockManager.java:166)
> at
> org.apache.phoenix.hbase.index.LockManager.unlockRow(LockManager.java:131)
> at
> org.apache.phoenix.hbase.index.Indexer.postBatchMutateIndispensably(Indexer.java:590)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$37.call(RegionCoprocessorHost.java:1023)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1621)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1697)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1653)
> at
> org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postBatchMutateIndispensably(RegionCoprocessorHost.java:1019)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2837)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2410)
> at
> org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2365)
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:225)
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commit(UngroupedAggregateRegionObserver.java:791)
> at
> org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:705)
> at
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.overrideDelegate(BaseScannerRegionObserver.java:255)
> at
> org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:306)
> at
> org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3361)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32492)
> at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2210)
> at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
> at
> org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
> at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
> at java.lang.Thread.run(Thread.java:745)
> 2017-08-24 07:47:19,916 FATAL [4,queue=4,port=60020]
> regionserver.HRegionServer - ABORTING region server
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (PHOENIX-4122) Prevent IllegalMonitorStateException from occurring when releasing row locks
James Taylor created PHOENIX-4122:
-
Summary: Prevent IllegalMonitorStateException from occurring when
releasing row locks
Key: PHOENIX-4122
URL: https://issues.apache.org/jira/browse/PHOENIX-4122
Project: Phoenix
Issue Type: Bug
Reporter: James Taylor
Assignee: James Taylor
Fix For: 4.12.0
In this iteration, after RS process was killed, Indexes were disabled and soon
its status was changed to INACTIVE state.
After all regions were re-assigned, we observed that two RS were aborted with
below exception in logs
{code}
org.apache.phoenix.hbase.index.Indexer threw
java.lang.IllegalMonitorStateException
java.lang.IllegalMonitorStateException
at
java.util.concurrent.locks.ReentrantLock$Sync.tryRelease(ReentrantLock.java:151)
at
java.util.concurrent.locks.AbstractQueuedSynchronizer.release(AbstractQueuedSynchronizer.java:1261)
at
java.util.concurrent.locks.ReentrantLock.unlock(ReentrantLock.java:457)
at
org.apache.phoenix.hbase.index.LockManager$RowLockImpl.release(LockManager.java:223)
at
org.apache.phoenix.hbase.index.LockManager$RowLockContext.releaseRowLock(LockManager.java:166)
at
org.apache.phoenix.hbase.index.LockManager.unlockRow(LockManager.java:131)
at
org.apache.phoenix.hbase.index.Indexer.postBatchMutateIndispensably(Indexer.java:590)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$37.call(RegionCoprocessorHost.java:1023)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost$RegionOperation.call(RegionCoprocessorHost.java:1621)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1697)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.execOperation(RegionCoprocessorHost.java:1653)
at
org.apache.hadoop.hbase.regionserver.RegionCoprocessorHost.postBatchMutateIndispensably(RegionCoprocessorHost.java:1019)
at
org.apache.hadoop.hbase.regionserver.HRegion.doMiniBatchMutation(HRegion.java:2837)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2410)
at
org.apache.hadoop.hbase.regionserver.HRegion.batchMutate(HRegion.java:2365)
at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commitBatch(UngroupedAggregateRegionObserver.java:225)
at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.commit(UngroupedAggregateRegionObserver.java:791)
at
org.apache.phoenix.coprocessor.UngroupedAggregateRegionObserver.doPostScannerOpen(UngroupedAggregateRegionObserver.java:705)
at
org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.overrideDelegate(BaseScannerRegionObserver.java:255)
at
org.apache.phoenix.coprocessor.BaseScannerRegionObserver$RegionScannerHolder.nextRaw(BaseScannerRegionObserver.java:306)
at
org.apache.hadoop.hbase.regionserver.HRegionServer.scan(HRegionServer.java:3361)
at
org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$2.callBlockingMethod(ClientProtos.java:32492)
at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:2210)
at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:104)
at
org.apache.hadoop.hbase.ipc.RpcExecutor.consumerLoop(RpcExecutor.java:133)
at org.apache.hadoop.hbase.ipc.RpcExecutor$1.run(RpcExecutor.java:108)
at java.lang.Thread.run(Thread.java:745)
2017-08-24 07:47:19,916 FATAL [4,queue=4,port=60020] regionserver.HRegionServer
- ABORTING region server
{code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140664#comment-16140664 ] James Taylor commented on PHOENIX-418: -- bq. I think it also makes sense to have another test derived from ParallelStatsEnabledIT Never mind about this - I was thinking of TABLESAMPLE, but you've already done this. > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT
[
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140468#comment-16140468
]
Ethan Wang edited comment on PHOENIX-418 at 8/24/17 8:34 PM:
-
Thanks for suggestions [~jamestaylor].
{quote}Make sure to call TestUtil.analyzeTable(connection, fullTableName) prior
to running your TABLESAMPLE queries. You'll get more rows back, since you'll
have guideposts in addition to region boundaries.{quote}
Correct me if I'm mistaken. Within the test for approximate count distinct,
table sampling technique was not used since HyperLogLog doesn't rely on
sampling (hence no run time speed gain during counting). So updating guidepost
may have less impact on approximate count distinct.
was (Author: aertoria):
Thanks for suggestions [~jamestaylor].
{quote}Make sure to call TestUtil.analyzeTable(connection, fullTableName) prior
to running your TABLESAMPLE queries. You'll get more rows back, since you'll
have guideposts in addition to region boundaries.{quote}
Correct me if I'm mistaken. Within the test for approximate count distinct,
table sampling technique was not used since HyperLogLog doesn't rely on
sampling (hence no run time speed gain during counting). So updating guidepost
may have less impact on approximate count distinct.
> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
> Issue Type: Task
>Reporter: James Taylor
>Assignee: Ethan Wang
> Labels: gsoc2016
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch,
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to
> all distinct values (since this will not scale well when the number of
> distinct values is huge). The Apache Drill folks have had some interesting
> discussions on this
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
> They recommend using [Welford's
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
> I'm open to having a config option that uses exact versus approximate. I
> don't have experience implementing an approximate implementation, so I'm not
> sure how much state is required to keep on the server and return to the
> client (other than realizing it'd be much less that returning all distinct
> values and their counts).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Comment Edited] (PHOENIX-418) Support approximate COUNT DISTINCT
[
https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16140468#comment-16140468
]
Ethan Wang edited comment on PHOENIX-418 at 8/24/17 8:34 PM:
-
Thanks for suggestions [~jamestaylor].
{quote}Make sure to call TestUtil.analyzeTable(connection, fullTableName) prior
to running your TABLESAMPLE queries. You'll get more rows back, since you'll
have guideposts in addition to region boundaries.{quote}
Correct me if I'm mistaken. Within the test for approximate count distinct,
table sampling technique was not used since HyperLogLog doesn't rely on
sampling (hence no run time speed gain during counting). So updating guidepost
may have less impact on approximate count distinct.
was (Author: aertoria):
Thanks for suggestions [~jamestaylor].
bq. Make sure to call TestUtil.analyzeTable(connection, fullTableName) prior to
running your TABLESAMPLE queries. You'll get more rows back, since you'll have
guideposts in addition to region boundaries.
If I understand this part right, within the test for approximate count
distinct, table sampling technique was not used since HyperLogLog doesn't rely
on sampling (hence no run time speed gain during counting).
> Support approximate COUNT DISTINCT
> --
>
> Key: PHOENIX-418
> URL: https://issues.apache.org/jira/browse/PHOENIX-418
> Project: Phoenix
> Issue Type: Task
>Reporter: James Taylor
>Assignee: Ethan Wang
> Labels: gsoc2016
> Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch,
> PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch
>
>
> Support an "approximation" of count distinct to prevent having to hold on to
> all distinct values (since this will not scale well when the number of
> distinct values is huge). The Apache Drill folks have had some interesting
> discussions on this
> [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E).
> They recommend using [Welford's
> method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm).
> I'm open to having a config option that uses exact versus approximate. I
> don't have experience implementing an approximate implementation, so I'm not
> sure how much state is required to keep on the server and return to the
> client (other than realizing it'd be much less that returning all distinct
> values and their counts).
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4080) The error message for version mismatch is not accurate.
[ https://issues.apache.org/jira/browse/PHOENIX-4080?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Wang updated PHOENIX-4080: Attachment: PHOENIX-4080-v2.patch > The error message for version mismatch is not accurate. > --- > > Key: PHOENIX-4080 > URL: https://issues.apache.org/jira/browse/PHOENIX-4080 > Project: Phoenix > Issue Type: Wish >Affects Versions: 4.11.0 >Reporter: Ethan Wang >Assignee: Ethan Wang > Attachments: PHOENIX-4080.patch, PHOENIX-4080-v2.patch > > > When accessing a 4.10 running cluster with 4.11 client, it referred as > The following servers require an updated phoenix.jar to be put in the > classpath of HBase: region=SYSTEM.CATALOG > It should be phoenix-[version]-server.jar rather than phoenix.jar -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Updated] (PHOENIX-418) Support approximate COUNT DISTINCT
[ https://issues.apache.org/jira/browse/PHOENIX-418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ethan Wang updated PHOENIX-418: --- Attachment: PHOENIX-418-v5.patch Thanks for suggestions [~jamestaylor]. bq. Make sure to call TestUtil.analyzeTable(connection, fullTableName) prior to running your TABLESAMPLE queries. You'll get more rows back, since you'll have guideposts in addition to region boundaries. If I understand this part right, within the test for approximate count distinct, table sampling technique was not used since HyperLogLog doesn't rely on sampling (hence no run time speed gain during counting). > Support approximate COUNT DISTINCT > -- > > Key: PHOENIX-418 > URL: https://issues.apache.org/jira/browse/PHOENIX-418 > Project: Phoenix > Issue Type: Task >Reporter: James Taylor >Assignee: Ethan Wang > Labels: gsoc2016 > Attachments: PHOENIX-418-v1.patch, PHOENIX-418-v2.patch, > PHOENIX-418-v3.patch, PHOENIX-418-v4.patch, PHOENIX-418-v5.patch > > > Support an "approximation" of count distinct to prevent having to hold on to > all distinct values (since this will not scale well when the number of > distinct values is huge). The Apache Drill folks have had some interesting > discussions on this > [here](http://mail-archives.apache.org/mod_mbox/incubator-drill-dev/201306.mbox/%3CJIRA.12650169.1369931282407.88049.1370645900553%40arcas%3E). > They recommend using [Welford's > method](http://en.wikipedia.org/wiki/Algorithms_for_calculating_variance_Online_algorithm). > I'm open to having a config option that uses exact versus approximate. I > don't have experience implementing an approximate implementation, so I'm not > sure how much state is required to keep on the server and return to the > client (other than realizing it'd be much less that returning all distinct > values and their counts). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (PHOENIX-4119) Upsert uses timestamp of server holding table metadata
[ https://issues.apache.org/jira/browse/PHOENIX-4119?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] James Taylor resolved PHOENIX-4119. --- Resolution: Duplicate With PHOENIX-4089 (which will be part of upcoming 4.12 release), Phoenix uses the latest timestamp when writing data. I'd recommend you synchronize your clocks in your cluster, though. > Upsert uses timestamp of server holding table metadata > -- > > Key: PHOENIX-4119 > URL: https://issues.apache.org/jira/browse/PHOENIX-4119 > Project: Phoenix > Issue Type: Bug >Affects Versions: 4.8.0 >Reporter: Mark Christiaens > > When doing an upsert, I noticed that the resulting put-command to HBase uses > a timestamp obtained from the server managing the corresponding table's > metadata (cfr. > https://issues.apache.org/jira/browse/PHOENIX-1674?focusedCommentId=14349982=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14349982). > > Now, if the row that is updated resides on a second server, that second > server could have some clock skew. After the upsert, if you fetch that > updated row without specifying an explicit query timestamp, then you might > not see the effects of the upsert until the second server's clock has caught > up with the first server's clock. > I think that the desired behavior is that the puts performed occur with a > timestamp derived from the region server where their rows are stored. (Of > course, that still leaves the problem of region migration from one server to > the next but it's a step in the right direction). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (PHOENIX-4121) SecureUserConnectionsTest has dependencies on testing environment kerberos setting
Ethan Wang created PHOENIX-4121: --- Summary: SecureUserConnectionsTest has dependencies on testing environment kerberos setting Key: PHOENIX-4121 URL: https://issues.apache.org/jira/browse/PHOENIX-4121 Project: Phoenix Issue Type: Bug Environment: Phoenix 4.12 Mac OS 10.12 Reporter: Ethan Wang Priority: Trivial When there is Kerberos that is currently active on the local os, SecureUserConnectionsTest will fails with "couldn't get the current user" error. Note that this error is observed on mac os 10.12, where kerberos is not implemented in MIT Krb5. Error msg detail java.lang.RuntimeException: Couldn't get the current user!! -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Resolved] (PHOENIX-4120) Can't get the records by select when the operator is "!=" and the specified column value is null
[
https://issues.apache.org/jira/browse/PHOENIX-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
James Taylor resolved PHOENIX-4120.
---
Resolution: Not A Problem
That's the way the trilogic is supposed to work in SQL. See
https://en.m.wikipedia.org/wiki/Three-valued_logic
> Can't get the records by select when the operator is "!=" and the specified
> column value is null
> --
>
> Key: PHOENIX-4120
> URL: https://issues.apache.org/jira/browse/PHOENIX-4120
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
> Environment: phoenix-4.8.1-HBase-1.0-SNAPSHOT-server.jar
>Reporter: alexBai
> Labels: null-values
>
> {code:java}
> 0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
> BIGINT, constraint pk primary key("id"));
> 2 rows affected (1.257 seconds)
> 0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
> 1 row affected (0.071 seconds)
> 0: jdbc:phoenix:> upsert into alex values(2, null, 28);
> 1 row affected (0.012 seconds)
> 0: jdbc:phoenix:> select * from alex;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 1 | alex | 28|
> | 2 | | 28|
> +-+---+--+
> 2 rows selected (0.063 seconds)
> 0: jdbc:phoenix:> select * from alex where "name"!='alex';
> +-+---+--+
> | id | name | age |
> +-+---+--+
> +-+---+--+
> No rows selected (0.053 seconds)
> 0: jdbc:phoenix:> select * from alex where "name" is null;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 2 | | 28 |
> +-+---+--+
> {code}
> Does phoenix just design like that?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
RE: RC for 4.12.0
Hi James, Could you take a second look at https://issues.apache.org/jira/browse/PHOENIX-2314 It have been pushed forward since 4.9.0 I believe. The fix is simple, so we've been patching ourselves, but it would be great if this could officially make 4.12.0. (the last results was that it failed 1 core test) Kind regards, Jasper. -Original Message- From: James Taylor [mailto:jamestay...@apache.org] Sent: 24 August 2017 00:51 To: dev@phoenix.apache.org Subject: RC for 4.12.0 ATTENTION. This message originates from outside BAE Systems. I'd like to target a 4.12 RC by the end of next week. Any objections? There are many fixes for global mutable secondary indexing and some for local indexing as well. As far as outstanding JIRAs, I know of PHOENIX-4010 and probably PHOENIX-418. I'd like to get in PHOENIX-3953, PHOENIX-3815, and maybe PHOENIX-4096. There's also work underway to stabilize our test runs which would be good to get in. Anything else? Thanks, James Please consider the environment before printing this email. This message should be regarded as confidential. If you have received this email in error please notify the sender and destroy it immediately. Statements of intent shall only become binding when confirmed in hard copy by an authorised signatory. The contents of this email may relate to dealings with other companies under the control of BAE Systems PLC, details of which can be found at http://www.baesystems.com/Businesses/index.htm.
[jira] [Updated] (PHOENIX-4116) UPSERT from SELECT of NULL fields does not work as expected
[
https://issues.apache.org/jira/browse/PHOENIX-4116?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Roberto Coluccio updated PHOENIX-4116:
--
Environment:
HBase 1.2
CDH 5.8.4
4.7.0-1.clabs_phoenix1.3.0.p0.000
was:
HBase 1.2
CDH 5.8.4
> UPSERT from SELECT of NULL fields does not work as expected
> ---
>
> Key: PHOENIX-4116
> URL: https://issues.apache.org/jira/browse/PHOENIX-4116
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.7.0
> Environment: HBase 1.2
> CDH 5.8.4
> 4.7.0-1.clabs_phoenix1.3.0.p0.000
>Reporter: Roberto Coluccio
>
> When performing an UPSERT from SELECT statement, if a column value returned
> from the SELECT one is NULL then the update is not performed on the target
> column:
> {code:sql}
> sqlline version 1.1.8
> 0: jdbc:phoenix:myhost> create table if not exists my_temp_table (my_key
> char(4) not null, value_1 varchar, value_2 varchar, constraint
> pk_my_temp_table primary key (my_key));
> No rows affected (1,602 seconds)
> 0: jdbc:phoenix:myhost> select * from my_temp_table;
> +-+--+--+
> | MY_KEY | VALUE_1 | VALUE_2 |
> +-+--+--+
> +-+--+--+
> No rows selected (0,657 seconds)
> {code}
> Now let's insert a record with a null and a not null values:
> {code:sql}
> 0: jdbc:phoenix:myhost> upsert into my_temp_table (my_key,value_1, value_2)
> values ('','abc', null);
> 1 row affected (0,138 seconds)
> 0: jdbc:phoenix:myhost> select * from my_temp_table where my_key = '';
> +-+--+--+
> | MY_KEY | VALUE_1 | VALUE_2 |
> +-+--+--+
> | | abc | |
> +-+--+--+
> 1 row selected (1,109 seconds)
> {code}
> Now I want to put VALUE_2 value (which is currently NULL) into VALUE_1:
> {code:sql}
> 0: jdbc:phoenix:myhost> upsert into my_temp_table (my_key,value_1) select
> my_key,value_2 from my_temp_table where my_key = '';
> 1 row affected (0,047 seconds)
> 0: jdbc:phoenix:myhost> select * from my_temp_table where my_key = '';
> +-+--+--+
> | MY_KEY | VALUE_1 | VALUE_2 |
> +-+--+--+
> | | abc | |
> +-+--+--+
> 1 row selected (0,04 seconds)
> {code}
> I'd have expected VALUE_1 to be NULL, instead it still has the value it had
> before the UPSERT statement.
> Obviously, updating VALUE_2 with the VALUE_1 content (not null) works as
> expected:
> {code:sql}
> 0: jdbc:phoenix:myhost> upsert into my_temp_table (my_key,value_2) select
> my_key,value_1 from my_temp_table where my_key = '';
> 1 row affected (0,048 seconds)
> 0: jdbc:phoenix:myhost> select * from my_temp_table where my_key = '';
> +-+--+--+
> | MY_KEY | VALUE_1 | VALUE_2 |
> +-+--+--+
> | | abc | abc |
> +-+--+--+
> 1 row selected (0,06 seconds)
> {code}
> *NOTE*: The problem occurs both with varchar and numeric (e.g. float) column
> types.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (PHOENIX-4120) Can't get the records by select when the operator is "!=" and the specified column value is null
[
https://issues.apache.org/jira/browse/PHOENIX-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16139754#comment-16139754
]
Csaba Skrabak commented on PHOENIX-4120:
I think this is the expected behavior in SQL in general. According to
three-valued logic, "name"!='alex' means NOT("name" = 'alex'), which evaluates
to NOT(NULL = 'alex') in the second row of your example. NULL = 'alex'
evaluates to Unknown, its negative is also Unknown, and Unknown in a WHERE
clause should filter out the row. Or am I wrong? See
[https://en.wikipedia.org/wiki/Null_(SQL)#Effect_of_Unknown_in_WHERE_clauses]
> Can't get the records by select when the operator is "!=" and the specified
> column value is null
> --
>
> Key: PHOENIX-4120
> URL: https://issues.apache.org/jira/browse/PHOENIX-4120
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
> Environment: phoenix-4.8.1-HBase-1.0-SNAPSHOT-server.jar
>Reporter: alexBai
> Labels: null-values
>
> {code:java}
> 0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
> BIGINT, constraint pk primary key("id"));
> 2 rows affected (1.257 seconds)
> 0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
> 1 row affected (0.071 seconds)
> 0: jdbc:phoenix:> upsert into alex values(2, null, 28);
> 1 row affected (0.012 seconds)
> 0: jdbc:phoenix:> select * from alex;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 1 | alex | 28|
> | 2 | | 28|
> +-+---+--+
> 2 rows selected (0.063 seconds)
> 0: jdbc:phoenix:> select * from alex where "name"!='alex';
> +-+---+--+
> | id | name | age |
> +-+---+--+
> +-+---+--+
> No rows selected (0.053 seconds)
> 0: jdbc:phoenix:> select * from alex where "name" is null;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 2 | | 28 |
> +-+---+--+
> {code}
> Does phoenix just design like that?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4120) Can't get the records by select when the operator is "!=" and the specified column value is null
[
https://issues.apache.org/jira/browse/PHOENIX-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
alexBai updated PHOENIX-4120:
-
Summary: Can't get the records by select when the operator is "!=" and
the specified column value is null (was: Can not get the records by select
when the operator is "!=" and the specified column value is null)
> Can't get the records by select when the operator is "!=" and the specified
> column value is null
> --
>
> Key: PHOENIX-4120
> URL: https://issues.apache.org/jira/browse/PHOENIX-4120
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
> Environment: phoenix-4.8.1-HBase-1.0-SNAPSHOT-server.jar
>Reporter: alexBai
> Labels: null-values
>
> {code:java}
> 0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
> BIGINT, constraint pk primary key("id"));
> 2 rows affected (1.257 seconds)
> 0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
> 1 row affected (0.071 seconds)
> 0: jdbc:phoenix:> upsert into alex values(2, null, 28);
> 1 row affected (0.012 seconds)
> 0: jdbc:phoenix:> select * from alex;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 1 | alex | 28|
> | 2 | | 28|
> +-+---+--+
> 2 rows selected (0.063 seconds)
> 0: jdbc:phoenix:> select * from alex where "name"!='alex';
> +-+---+--+
> | id | name | age |
> +-+---+--+
> +-+---+--+
> No rows selected (0.053 seconds)
> 0: jdbc:phoenix:> select * from alex where "name" is null;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 2 | | 28 |
> +-+---+--+
> {code}
> Does phoenix just design like that?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4120) Can not get the records by select when the operator is "!=" and the specified column value is null
[
https://issues.apache.org/jira/browse/PHOENIX-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
alexBai updated PHOENIX-4120:
-
Description:
{code:java}
0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
BIGINT, constraint pk primary key("id"));
2 rows affected (1.257 seconds)
0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
1 row affected (0.071 seconds)
0: jdbc:phoenix:> upsert into alex values(2, null, 28);
1 row affected (0.012 seconds)
0: jdbc:phoenix:> select * from alex;
+-+---+--+
| id | name | age |
+-+---+--+
| 1 | alex | 28|
| 2 | | 28|
+-+---+--+
2 rows selected (0.063 seconds)
0: jdbc:phoenix:> select * from alex where "name"!='alex';
+-+---+--+
| id | name | age |
+-+---+--+
+-+---+--+
No rows selected (0.053 seconds)
0: jdbc:phoenix:> select * from alex where "name" is null;
+-+---+--+
| id | name | age |
+-+---+--+
| 2 | | 28 |
+-+---+--+
{code}
Does phoenix just design like that?
was:
{code:java}
0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
BIGINT, constraint pk primary key("id"));
2 rows affected (1.257 seconds)
0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
1 row affected (0.071 seconds)
0: jdbc:phoenix:> upsert into alex values(2, null, 28);
1 row affected (0.012 seconds)
0: jdbc:phoenix:> select * from alex;
+-+---+--+
| id | name | age |
+-+---+--+
| 1 | alex | 28|
| 2 | | 28|
+-+---+--+
2 rows selected (0.063 seconds)
0: jdbc:phoenix:> select * from alex where "name"!='alex';
+-+---+--+
| id | name | age |
+-+---+--+
+-+---+--+
No rows selected (0.053 seconds)
0: jdbc:phoenix:> select * from alex where "name" is null;
+-+---+--+
| id | name | age |
+-+---+--+
| 2 | | 28 |
+-+---+--+
{code}
> Can not get the records by select when the operator is "!=" and the
> specified column value is null
>
>
> Key: PHOENIX-4120
> URL: https://issues.apache.org/jira/browse/PHOENIX-4120
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
> Environment: phoenix-4.8.1-HBase-1.0-SNAPSHOT-server.jar
>Reporter: alexBai
> Labels: null-values
>
> {code:java}
> 0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
> BIGINT, constraint pk primary key("id"));
> 2 rows affected (1.257 seconds)
> 0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
> 1 row affected (0.071 seconds)
> 0: jdbc:phoenix:> upsert into alex values(2, null, 28);
> 1 row affected (0.012 seconds)
> 0: jdbc:phoenix:> select * from alex;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 1 | alex | 28|
> | 2 | | 28|
> +-+---+--+
> 2 rows selected (0.063 seconds)
> 0: jdbc:phoenix:> select * from alex where "name"!='alex';
> +-+---+--+
> | id | name | age |
> +-+---+--+
> +-+---+--+
> No rows selected (0.053 seconds)
> 0: jdbc:phoenix:> select * from alex where "name" is null;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 2 | | 28 |
> +-+---+--+
> {code}
> Does phoenix just design like that?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Updated] (PHOENIX-4120) Can not get the records by select when the operator is "!=" and the specified column value is null
[
https://issues.apache.org/jira/browse/PHOENIX-4120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
alexBai updated PHOENIX-4120:
-
Description:
{code:java}
0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
BIGINT, constraint pk primary key("id"));
2 rows affected (1.257 seconds)
0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
1 row affected (0.071 seconds)
0: jdbc:phoenix:> upsert into alex values(2, null, 28);
1 row affected (0.012 seconds)
0: jdbc:phoenix:> select * from alex;
+-+---+--+
| id | name | age |
+-+---+--+
| 1 | alex | 28|
| 2 | | 28|
+-+---+--+
2 rows selected (0.063 seconds)
0: jdbc:phoenix:> select * from alex where "name"!='alex';
+-+---+--+
| id | name | age |
+-+---+--+
+-+---+--+
No rows selected (0.053 seconds)
0: jdbc:phoenix:> select * from alex where "name" is null;
+-+---+--+
| id | name | age |
+-+---+--+
| 2 | | 28 |
+-+---+--+
{code}
was:
0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
BIGINT, constraint pk primary key("id"));
2 rows affected (1.257 seconds)
0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
1 row affected (0.071 seconds)
0: jdbc:phoenix:> upsert into alex values(2, null, 28);
1 row affected (0.012 seconds)
0: jdbc:phoenix:> select * from alex;
+-+---+--+
| id | name | age |
+-+---+--+
| 1 | alex | 28 |
| 2 | | 28 |
+-+---+--+
2 rows selected (0.063 seconds)
0: jdbc:phoenix:> select * from alex where "name"!='alex';
+-+---+--+
| id | name | age |
+-+---+--+
+-+---+--+
No rows selected (0.053 seconds)
0: jdbc:phoenix:> select * from alex where "name" is null;
+-+---+--+
| id | name | age |
+-+---+--+
| 2 | | 28 |
+-+---+--+
> Can not get the records by select when the operator is "!=" and the
> specified column value is null
>
>
> Key: PHOENIX-4120
> URL: https://issues.apache.org/jira/browse/PHOENIX-4120
> Project: Phoenix
> Issue Type: Bug
>Affects Versions: 4.8.0
> Environment: phoenix-4.8.1-HBase-1.0-SNAPSHOT-server.jar
>Reporter: alexBai
> Labels: null-values
>
> {code:java}
> 0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
> BIGINT, constraint pk primary key("id"));
> 2 rows affected (1.257 seconds)
> 0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
> 1 row affected (0.071 seconds)
> 0: jdbc:phoenix:> upsert into alex values(2, null, 28);
> 1 row affected (0.012 seconds)
> 0: jdbc:phoenix:> select * from alex;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 1 | alex | 28|
> | 2 | | 28|
> +-+---+--+
> 2 rows selected (0.063 seconds)
> 0: jdbc:phoenix:> select * from alex where "name"!='alex';
> +-+---+--+
> | id | name | age |
> +-+---+--+
> +-+---+--+
> No rows selected (0.053 seconds)
> 0: jdbc:phoenix:> select * from alex where "name" is null;
> +-+---+--+
> | id | name | age |
> +-+---+--+
> | 2 | | 28 |
> +-+---+--+
> {code}
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (PHOENIX-4120) Can not get the records by select when the operator is "!=" and the specified column value is null
alexBai created PHOENIX-4120:
Summary: Can not get the records by select when the operator is
"!=" and the specified column value is null
Key: PHOENIX-4120
URL: https://issues.apache.org/jira/browse/PHOENIX-4120
Project: Phoenix
Issue Type: Bug
Affects Versions: 4.8.0
Environment: phoenix-4.8.1-HBase-1.0-SNAPSHOT-server.jar
Reporter: alexBai
0: jdbc:phoenix:> create table alex ("id" BIGINT, "name" varchar(10), "age"
BIGINT, constraint pk primary key("id"));
2 rows affected (1.257 seconds)
0: jdbc:phoenix:> upsert into alex values(1, 'alex', 28);
1 row affected (0.071 seconds)
0: jdbc:phoenix:> upsert into alex values(2, null, 28);
1 row affected (0.012 seconds)
0: jdbc:phoenix:> select * from alex;
+-+---+--+
| id | name | age |
+-+---+--+
| 1 | alex | 28 |
| 2 | | 28 |
+-+---+--+
2 rows selected (0.063 seconds)
0: jdbc:phoenix:> select * from alex where "name"!='alex';
+-+---+--+
| id | name | age |
+-+---+--+
+-+---+--+
No rows selected (0.053 seconds)
0: jdbc:phoenix:> select * from alex where "name" is null;
+-+---+--+
| id | name | age |
+-+---+--+
| 2 | | 28 |
+-+---+--+
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Created] (PHOENIX-4119) Upsert uses timestamp of server holding table metadata
Mark Christiaens created PHOENIX-4119: - Summary: Upsert uses timestamp of server holding table metadata Key: PHOENIX-4119 URL: https://issues.apache.org/jira/browse/PHOENIX-4119 Project: Phoenix Issue Type: Bug Affects Versions: 4.8.0 Reporter: Mark Christiaens When doing an upsert, I noticed that the resulting put-command to HBase uses a timestamp obtained from the server managing the corresponding table's metadata (cfr. https://issues.apache.org/jira/browse/PHOENIX-1674?focusedCommentId=14349982=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14349982). Now, if the row that is updated resides on a second server, that second server could have some clock skew. After the upsert, if you fetch that updated row without specifying an explicit query timestamp, then you might not see the effects of the upsert until the second server's clock has caught up with the first server's clock. I think that the desired behavior is that the puts performed occur with a timestamp derived from the region server where their rows are stored. (Of course, that still leaves the problem of region migration from one server to the next but it's a step in the right direction). -- This message was sent by Atlassian JIRA (v6.4.14#64029)
