[jira] [Resolved] (PHOENIX-4496) Fix RowValueConstructorIT and IndexMetadataIT

[Ankit Singhal (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Ankit Singhal resolved PHOENIX-4496.

Resolution: Fixed

> Fix RowValueConstructorIT and IndexMetadataIT
> -
>
> Key: PHOENIX-4496
> URL: https://issues.apache.org/jira/browse/PHOENIX-4496
> Project: Phoenix
>  Issue Type: Sub-task
>Affects Versions: 4.14.0
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4496.patch
>
>
> {noformat}
> [ERROR] Tests run: 46, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 117.444 s <<< FAILURE! - in org.apache.phoenix.end2end.RowValueConstructorIT
> [ERROR] 
> testRVCLastPkIsTable1stPkIndex(org.apache.phoenix.end2end.RowValueConstructorIT)
>   Time elapsed: 4.516 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.RowValueConstructorIT.testRVCLastPkIsTable1stPkIndex(RowValueConstructorIT.java:1584)
> {noformat}
> {noformat}
> ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 79.381 s <<< FAILURE! - in org.apache.phoenix.end2end.index.IndexMetadataIT
> [ERROR] 
> testMutableTableOnlyHasPrimaryKeyIndex(org.apache.phoenix.end2end.index.IndexMetadataIT)
>   Time elapsed: 4.504 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.helpTestTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:662)
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.testMutableTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:623)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4496) Fix RowValueConstructorIT and IndexMetadataIT

[Ankit Singhal (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438226#comment-16438226
 ] 

Ankit Singhal commented on PHOENIX-4496:


bq. This doesn't compile in master against HBase 1.4, so I've reverted for now.
Sorry about that, I ran mvn verify for these test on all the branches (where it 
is applicable) but don't know how master got missed. Fixed it now. Thanks

> Fix RowValueConstructorIT and IndexMetadataIT
> -
>
> Key: PHOENIX-4496
> URL: https://issues.apache.org/jira/browse/PHOENIX-4496
> Project: Phoenix
>  Issue Type: Sub-task
>Affects Versions: 4.14.0
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4496.patch
>
>
> {noformat}
> [ERROR] Tests run: 46, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 117.444 s <<< FAILURE! - in org.apache.phoenix.end2end.RowValueConstructorIT
> [ERROR] 
> testRVCLastPkIsTable1stPkIndex(org.apache.phoenix.end2end.RowValueConstructorIT)
>   Time elapsed: 4.516 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.RowValueConstructorIT.testRVCLastPkIsTable1stPkIndex(RowValueConstructorIT.java:1584)
> {noformat}
> {noformat}
> ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 79.381 s <<< FAILURE! - in org.apache.phoenix.end2end.index.IndexMetadataIT
> [ERROR] 
> testMutableTableOnlyHasPrimaryKeyIndex(org.apache.phoenix.end2end.index.IndexMetadataIT)
>   Time elapsed: 4.504 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.helpTestTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:662)
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.testMutableTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:623)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4575) Phoenix metadata KEEP_DELETED_CELLS and VERSIONS should be property driven

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438187#comment-16438187
 ] 

Hudson commented on PHOENIX-4575:
-

FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-0.98 #1858 (See 
[https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/1858/])
PHOENIX-4575 Phoenix metadata KEEP_DELETED_CELLS and VERSIONS should be 
(jtaylor: rev fce9af53423d1804f5dc8daf8f28f9f7b38bbff5)
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/UpgradeUtil.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/QueryServicesOptions.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataProtocol.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java


> Phoenix metadata KEEP_DELETED_CELLS and VERSIONS should be property driven
> --
>
> Key: PHOENIX-4575
> URL: https://issues.apache.org/jira/browse/PHOENIX-4575
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4575.patch, PHOENIX-4575.patch, 
> PHOENIX-4575_v2.patch
>
>
> This is to cater for circumstances where we need to alter state of 
> KEEP_DELETED_CELLS/VERSION on Phoenix meta tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4668) Remove unnecessary table descriptor modification for SPLIT_POLICY column

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438186#comment-16438186
 ] 

Hudson commented on PHOENIX-4668:
-

FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-0.98 #1858 (See 
[https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/1858/])
PHOENIX-4668 Remove unnecessary table descriptor modification for (jtaylor: rev 
0a28d6aa8ac3197417353e32b8395738ac664ce7)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java


> Remove unnecessary table descriptor modification for SPLIT_POLICY column
> 
>
> Key: PHOENIX-4668
> URL: https://issues.apache.org/jira/browse/PHOENIX-4668
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4668.patch
>
>
> Inside _ConnectionQueryServicesImpl.ensureTableCreated()_, we modify the 
> table descriptor with
> newDesc.setValue(HTableDescriptor.SPLIT_POLICY, 
> MetaDataSplitPolicy.class.getName()), however we already have this mentioned 
> in the create statement DDL for system tables, so we can remove this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4668) Remove unnecessary table descriptor modification for SPLIT_POLICY column

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438113#comment-16438113
 ] 

Hudson commented on PHOENIX-4668:
-

SUCCESS: Integrated in Jenkins build Phoenix-4.x-HBase-1.3 #96 (See 
[https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/96/])
PHOENIX-4668 Remove unnecessary table descriptor modification for (jtaylor: rev 
8e61260bb2df936670be836ad02018c35b6842b7)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java


> Remove unnecessary table descriptor modification for SPLIT_POLICY column
> 
>
> Key: PHOENIX-4668
> URL: https://issues.apache.org/jira/browse/PHOENIX-4668
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4668.patch
>
>
> Inside _ConnectionQueryServicesImpl.ensureTableCreated()_, we modify the 
> table descriptor with
> newDesc.setValue(HTableDescriptor.SPLIT_POLICY, 
> MetaDataSplitPolicy.class.getName()), however we already have this mentioned 
> in the create statement DDL for system tables, so we can remove this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4579) Add a config to conditionally create Phoenix meta tables on first client connection

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438074#comment-16438074
 ] 

Hudson commented on PHOENIX-4579:
-

FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-0.98 #1857 (See 
[https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/1857/])
PHOENIX-4579 Remove SYSTEM.LOG from list of Phoenix system tables since 
(jtaylor: rev 3e8049c91d1730f5dd81c8bdd8430d20d1885ec4)
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/SystemCatalogCreationOnConnectionIT.java


> Add a config to conditionally create Phoenix meta tables on first client 
> connection
> ---
>
> Key: PHOENIX-4579
> URL: https://issues.apache.org/jira/browse/PHOENIX-4579
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4579.patch
>
>
> Currently we create/modify Phoenix meta tables on first client connection. 
> Adding a property to make it configurable (with default true as it is 
> currently implemented).
> With this property set to false, it will avoid lockstep upgrade requirement 
> for all clients when changing meta properties using PHOENIX-4575 as this 
> property can be flipped back on once all the clients are upgraded.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4231) Support restriction of remote UDF load sources

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438068#comment-16438068
 ] 

ASF GitHub Bot commented on PHOENIX-4231:
-

Github user ChinmaySKulkarni closed the pull request at:

https://github.com/apache/phoenix/pull/292


> Support restriction of remote UDF load sources 
> ---
>
> Key: PHOENIX-4231
> URL: https://issues.apache.org/jira/browse/PHOENIX-4231
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Andrew Purtell
>Assignee: Chinmay Kulkarni
>Priority: Major
> Fix For: 4.14.0
>
> Attachments: PHOENIX-4231-v2.patch, PHOENIX-4231.patch
>
>
> When allowUserDefinedFunctions is true, users can load UDFs remotely via a 
> jar file from any HDFS filesystem reachable on the network. The setting 
> hbase.dynamic.jars.dir can be used to restrict locations for jar loading but 
> is only applied to jars loaded from the local filesystem.  We should 
> implement support for similar restriction via configuration for jars loaded 
> via hdfs:// URIs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] phoenix pull request #292: PHOENIX-4231: Support restriction of remote UDF l...

[ChinmaySKulkarni]

Github user ChinmaySKulkarni closed the pull request at:

https://github.com/apache/phoenix/pull/292


---

[jira] [Commented] (PHOENIX-4666) Add a subquery cache that persists beyond the life of a query

[Maryann Xue (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438067#comment-16438067
 ] 

Maryann Xue commented on PHOENIX-4666:
--

Sounds good! Please remember to add more tests.

> Add a subquery cache that persists beyond the life of a query
> -
>
> Key: PHOENIX-4666
> URL: https://issues.apache.org/jira/browse/PHOENIX-4666
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Marcell Ortutay
>Assignee: Marcell Ortutay
>Priority: Major
>
> The user list thread for additional context is here: 
> [https://lists.apache.org/thread.html/e62a6f5d79bdf7cd238ea79aed8886816d21224d12b0f1fe9b6bb075@%3Cuser.phoenix.apache.org%3E]
> 
> A Phoenix query may contain expensive subqueries, and moreover those 
> expensive subqueries may be used across multiple different queries. While 
> whole result caching is possible at the application level, it is not possible 
> to cache subresults in the application. This can cause bad performance for 
> queries in which the subquery is the most expensive part of the query, and 
> the application is powerless to do anything at the query level. It would be 
> good if Phoenix provided a way to cache subquery results, as it would provide 
> a significant performance gain.
> An illustrative example:
>     SELECT * FROM table1 JOIN (SELECT id_1 FROM large_table WHERE x = 10) 
> expensive_result ON table1.id_1 = expensive_result.id_2 AND table1.id_1 = 
> \{id}
> In this case, the subquery "expensive_result" is expensive to compute, but it 
> doesn't change between queries. The rest of the query does because of the 
> \{id} parameter. This means the application can't cache it, but it would be 
> good if there was a way to cache expensive_result.
> Note that there is currently a coprocessor based "server cache", but the data 
> in this "cache" is not persisted across queries. It is deleted after a TTL 
> expires (30sec by default), or when the query completes.
> This is issue is fairly high priority for us at 23andMe and we'd be happy to 
> provide a patch with some guidance from Phoenix maintainers. We are currently 
> putting together a design document for a solution, and we'll post it to this 
> Jira ticket for review in a few days.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[DISCUSS] time for 4.14 release?

[James Taylor]

We've got a ton of great fixes in the 4.x branches and it's been a while
since we did a release. There are a couple of stragglers I'd like to get in:
- PHOENIX-4600 Add retry logic for partial index rebuilder writes
- PHOENIX-4601 Perform server-side retries if client version < 4.14
- PHOENIX-2715 Query Log (just missing from a couple of branches)

I also noticed that PHOENIX-4685 is marked as a blocker.

Any others that need to go in?

How about we shoot for having an RC by the end of next week?

James

[jira] [Commented] (PHOENIX-4666) Add a subquery cache that persists beyond the life of a query

[Marcell Ortutay (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16438013#comment-16438013
 ] 

Marcell Ortutay commented on PHOENIX-4666:
--

FYI added hint-to-enable ("USE_PERSISTENT_CACHE") to my fork. This is the last 
"feature" that I think is needed for a v1. I'm going to clean things up a bit 
and refactor for the noChildParentJoinOptimization approach suggested above and 
then submit for a full review.

> Add a subquery cache that persists beyond the life of a query
> -
>
> Key: PHOENIX-4666
> URL: https://issues.apache.org/jira/browse/PHOENIX-4666
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Marcell Ortutay
>Assignee: Marcell Ortutay
>Priority: Major
>
> The user list thread for additional context is here: 
> [https://lists.apache.org/thread.html/e62a6f5d79bdf7cd238ea79aed8886816d21224d12b0f1fe9b6bb075@%3Cuser.phoenix.apache.org%3E]
> 
> A Phoenix query may contain expensive subqueries, and moreover those 
> expensive subqueries may be used across multiple different queries. While 
> whole result caching is possible at the application level, it is not possible 
> to cache subresults in the application. This can cause bad performance for 
> queries in which the subquery is the most expensive part of the query, and 
> the application is powerless to do anything at the query level. It would be 
> good if Phoenix provided a way to cache subquery results, as it would provide 
> a significant performance gain.
> An illustrative example:
>     SELECT * FROM table1 JOIN (SELECT id_1 FROM large_table WHERE x = 10) 
> expensive_result ON table1.id_1 = expensive_result.id_2 AND table1.id_1 = 
> \{id}
> In this case, the subquery "expensive_result" is expensive to compute, but it 
> doesn't change between queries. The rest of the query does because of the 
> \{id} parameter. This means the application can't cache it, but it would be 
> good if there was a way to cache expensive_result.
> Note that there is currently a coprocessor based "server cache", but the data 
> in this "cache" is not persisted across queries. It is deleted after a TTL 
> expires (30sec by default), or when the query completes.
> This is issue is fairly high priority for us at 23andMe and we'd be happy to 
> provide a patch with some guidance from Phoenix maintainers. We are currently 
> putting together a design document for a solution, and we'll post it to this 
> Jira ticket for review in a few days.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Reopened] (PHOENIX-2715) Query Log

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor reopened PHOENIX-2715:
---

> Query Log
> -
>
> Key: PHOENIX-2715
> URL: https://issues.apache.org/jira/browse/PHOENIX-2715
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Nick Dimiduk
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-2715.patch, PHOENIX-2715_master.patch, 
> PHOENIX-2715_master_V1.patch, PHOENIX-2715_master_V2.patch
>
>
> One useful feature of other database systems is the query log. It allows the 
> DBA to review the queries run, who's run them, time taken,  This serves 
> both as an audit and also as a source of "ground truth" for performance 
> optimization. For instance, which columns should be indexed. It may also 
> serve as the foundation for automated performance recommendations/actions.
> What queries are being run is the first piece. Have this data tied into 
> tracing results and perhaps client-side metrics (PHOENIX-1819) becomes very 
> useful.
> This might take the form of clients writing data to a new system table, but 
> other implementation suggestions are welcome.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-2715) Query Log

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437996#comment-16437996
 ] 

James Taylor commented on PHOENIX-2715:
---

Actually, it was backported to 1.1 (thanks!), but just not 0.98.

> Query Log
> -
>
> Key: PHOENIX-2715
> URL: https://issues.apache.org/jira/browse/PHOENIX-2715
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Nick Dimiduk
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-2715.patch, PHOENIX-2715_master.patch, 
> PHOENIX-2715_master_V1.patch, PHOENIX-2715_master_V2.patch
>
>
> One useful feature of other database systems is the query log. It allows the 
> DBA to review the queries run, who's run them, time taken,  This serves 
> both as an audit and also as a source of "ground truth" for performance 
> optimization. For instance, which columns should be indexed. It may also 
> serve as the foundation for automated performance recommendations/actions.
> What queries are being run is the first piece. Have this data tied into 
> tracing results and perhaps client-side metrics (PHOENIX-1819) becomes very 
> useful.
> This might take the form of clients writing data to a new system table, but 
> other implementation suggestions are welcome.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4579) Add a config to conditionally create Phoenix meta tables on first client connection

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437960#comment-16437960
 ] 

Hudson commented on PHOENIX-4579:
-

FAILURE: Integrated in Jenkins build PreCommit-PHOENIX-Build #1830 (See 
[https://builds.apache.org/job/PreCommit-PHOENIX-Build/1830/])
PHOENIX-4579 Add a config to conditionally create Phoenix meta tables on 
(jtaylor: rev 87fdda8b180438e57d0e2f6082e5a9e988220245)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServices.java
* (edit) 
phoenix-core/src/test/java/org/apache/phoenix/query/ConnectionQueryServicesImplTest.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataProtocol.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
* (add) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/SystemCatalogCreationOnConnectionIT.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/exception/UpgradeRequiredException.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/AppendOnlySchemaIT.java
* (edit) phoenix-protocol/src/main/MetaDataService.proto
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/generated/MetaDataProtos.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/DelegateConnectionQueryServices.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/MigrateSystemTablesToSystemNamespaceIT.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java


> Add a config to conditionally create Phoenix meta tables on first client 
> connection
> ---
>
> Key: PHOENIX-4579
> URL: https://issues.apache.org/jira/browse/PHOENIX-4579
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4579.patch
>
>
> Currently we create/modify Phoenix meta tables on first client connection. 
> Adding a property to make it configurable (with default true as it is 
> currently implemented).
> With this property set to false, it will avoid lockstep upgrade requirement 
> for all clients when changing meta properties using PHOENIX-4575 as this 
> property can be flipped back on once all the clients are upgraded.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4496) Fix RowValueConstructorIT and IndexMetadataIT

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437959#comment-16437959
 ] 

Hudson commented on PHOENIX-4496:
-

FAILURE: Integrated in Jenkins build PreCommit-PHOENIX-Build #1830 (See 
[https://builds.apache.org/job/PreCommit-PHOENIX-Build/1830/])
PHOENIX-4496 Fix RowValueConstructorIT and IndexMetadataIT (ankitsinghal59: rev 
70c9c6bb2d2073bb40640a0a794032b166c4b63b)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/scanner/ScannerBuilder.java
Revert "PHOENIX-4496 Fix RowValueConstructorIT and IndexMetadataIT" (jtaylor: 
rev 7b2c7e135916d5a6087ec3e6c4b6c261779795ac)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/scanner/ScannerBuilder.java


> Fix RowValueConstructorIT and IndexMetadataIT
> -
>
> Key: PHOENIX-4496
> URL: https://issues.apache.org/jira/browse/PHOENIX-4496
> Project: Phoenix
>  Issue Type: Sub-task
>Affects Versions: 4.14.0
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4496.patch
>
>
> {noformat}
> [ERROR] Tests run: 46, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 117.444 s <<< FAILURE! - in org.apache.phoenix.end2end.RowValueConstructorIT
> [ERROR] 
> testRVCLastPkIsTable1stPkIndex(org.apache.phoenix.end2end.RowValueConstructorIT)
>   Time elapsed: 4.516 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.RowValueConstructorIT.testRVCLastPkIsTable1stPkIndex(RowValueConstructorIT.java:1584)
> {noformat}
> {noformat}
> ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 79.381 s <<< FAILURE! - in org.apache.phoenix.end2end.index.IndexMetadataIT
> [ERROR] 
> testMutableTableOnlyHasPrimaryKeyIndex(org.apache.phoenix.end2end.index.IndexMetadataIT)
>   Time elapsed: 4.504 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.helpTestTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:662)
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.testMutableTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:623)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4605) Support running multiple transaction providers

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437958#comment-16437958
 ] 

Hudson commented on PHOENIX-4605:
-

FAILURE: Integrated in Jenkins build PreCommit-PHOENIX-Build #1830 (See 
[https://builds.apache.org/job/PreCommit-PHOENIX-Build/1830/])
PHOENIX-4605 Support running multiple transaction providers (jtaylor: rev 
de83b8d5d042098faa294e742525ab84175bb271)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/DelegateConnectionQueryServices.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/compile/UpsertCompiler.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixDatabaseMetaData.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/PhoenixTransactionContext.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryServices.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/ScanUtil.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/BaseScannerRegionObserver.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/util/PhoenixRuntime.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/tx/FlappingTransactionIT.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/TephraTransactionContext.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexCodec.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/index/PhoenixIndexMetaDataBuilder.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataProtocol.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/compile/DeleteCompiler.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataRegionObserver.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/compile/UnionCompiler.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/mapreduce/index/PhoenixIndexPartialBuildMapper.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/OmidTransactionProvider.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/AlterTableWithViewsIT.java
* (edit) phoenix-core/src/it/java/org/apache/phoenix/tx/TxCheckpointIT.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/FromCompiler.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/query/QueryConstants.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/TephraTransactionProvider.java
* (edit) 
phoenix-core/src/test/java/org/apache/phoenix/execute/CorrelatePlanTest.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PTable.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/ServerCachingEndpointImpl.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/OmidTransactionContext.java
* (delete) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixTransactionalProcessor.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/PTableImpl.java
* (edit) phoenix-core/src/test/java/org/apache/phoenix/query/BaseTest.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/compile/TupleProjectionCompiler.java
* (add) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/PhoenixTransactionClient.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/expression/ExpressionType.java
* (add) 
phoenix-core/src/main/java/org/apache/phoenix/expression/function/TransactionProviderNameFunction.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/iterate/NonAggregateRegionScannerFactory.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/tx/ParameterizedTransactionIT.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/ConnectionQueryServicesTestImpl.java
* (add) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/PhoenixTransactionService.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/jdbc/PhoenixStatement.java
* (delete) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/TransactionProvider.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/compile/JoinCompiler.java
* (edit) phoenix-core/src/main/java/org/apache/phoenix/schema/TableProperty.java
* (delete) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/TephraTransactionTable.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/exception/SQLExceptionCode.java
* (add) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/PhoenixTransactionProvider.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/index/IndexMetaDataCacheFactory.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/transaction/TransactionFactory.java
* (edit) 

Re: getting rid of deprecated APIs to reduce effort in 5.x porting?

[James Taylor]

The original plan was to get rid of the deprecated APIs as the first change
for the 5.x branch (before the branches diverged). Then it would have
applied to all the 4.x-1.* branches. Unfortunately that ship has sailed.
Probably easiest now to remove the deprecated API usage in 1.3 or master
branch and apply it to other branches (except 0.98). The 4.x-1.* branches
are all very similar.

We tried nuking the other branches, but we got community push back and
volunteers stepped up to become release managers. On the plus side, this
has worked out really well for 4.x-1.2 as now we have CDH support.

I think we should do a 4.14 release across all of our 4.x branches and then
start the conversation again about deprecating the 1.1 branch. I'd love to
do the same for 0.98, but it seems to be hanging around a bit longer than
expected.

On Fri, Apr 13, 2018 at 12:09 PM, Josh Elser  wrote:

> It's definitely a goal, but I don't know of anyone actively working on
> that.
>
> What ever came about dropping old versions of HBase support? I thought we
> nuked <=1.2 (leaving just 1.3 and 1.4), but it seems like they have come
> back... The reason I ask: backporting the deprecation changes will be
> tough, and likely be tough for each branch they're targeted on. Trying to
> reduce where they need to go would be a big help.
>
>
> On 4/13/18 12:29 AM, James Taylor wrote:
>
>> Is it still the plan to get rid of deprecated HBase APIs in the 4.x
>> branches? This would ease the pain of porting changes between 4.x and 5.x
>> branches. I think it's likely that both branches will need to be
>> maintained
>> for some time. We'd likely need to get a Tephra release with this done
>> too,
>> but I think the work has already been done, no?
>>
>> Thanks,
>> James
>>
>>

[jira] [Comment Edited] (PHOENIX-4666) Add a subquery cache that persists beyond the life of a query

[Marcell Ortutay (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437921#comment-16437921
 ] 

Marcell Ortutay edited comment on PHOENIX-4666 at 4/13/18 9:10 PM:
---

Ah! got it, thanks


was (Author: ortutay):
Ah! got it, thank

> Add a subquery cache that persists beyond the life of a query
> -
>
> Key: PHOENIX-4666
> URL: https://issues.apache.org/jira/browse/PHOENIX-4666
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Marcell Ortutay
>Assignee: Marcell Ortutay
>Priority: Major
>
> The user list thread for additional context is here: 
> [https://lists.apache.org/thread.html/e62a6f5d79bdf7cd238ea79aed8886816d21224d12b0f1fe9b6bb075@%3Cuser.phoenix.apache.org%3E]
> 
> A Phoenix query may contain expensive subqueries, and moreover those 
> expensive subqueries may be used across multiple different queries. While 
> whole result caching is possible at the application level, it is not possible 
> to cache subresults in the application. This can cause bad performance for 
> queries in which the subquery is the most expensive part of the query, and 
> the application is powerless to do anything at the query level. It would be 
> good if Phoenix provided a way to cache subquery results, as it would provide 
> a significant performance gain.
> An illustrative example:
>     SELECT * FROM table1 JOIN (SELECT id_1 FROM large_table WHERE x = 10) 
> expensive_result ON table1.id_1 = expensive_result.id_2 AND table1.id_1 = 
> \{id}
> In this case, the subquery "expensive_result" is expensive to compute, but it 
> doesn't change between queries. The rest of the query does because of the 
> \{id} parameter. This means the application can't cache it, but it would be 
> good if there was a way to cache expensive_result.
> Note that there is currently a coprocessor based "server cache", but the data 
> in this "cache" is not persisted across queries. It is deleted after a TTL 
> expires (30sec by default), or when the query completes.
> This is issue is fairly high priority for us at 23andMe and we'd be happy to 
> provide a patch with some guidance from Phoenix maintainers. We are currently 
> putting together a design document for a solution, and we'll post it to this 
> Jira ticket for review in a few days.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4666) Add a subquery cache that persists beyond the life of a query

[Marcell Ortutay (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437901#comment-16437901
 ] 

Marcell Ortutay commented on PHOENIX-4666:
--

Got it–for option (2) that would basically require calling setScanRange() with 
argument to scan "all"?

> Add a subquery cache that persists beyond the life of a query
> -
>
> Key: PHOENIX-4666
> URL: https://issues.apache.org/jira/browse/PHOENIX-4666
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Marcell Ortutay
>Assignee: Marcell Ortutay
>Priority: Major
>
> The user list thread for additional context is here: 
> [https://lists.apache.org/thread.html/e62a6f5d79bdf7cd238ea79aed8886816d21224d12b0f1fe9b6bb075@%3Cuser.phoenix.apache.org%3E]
> 
> A Phoenix query may contain expensive subqueries, and moreover those 
> expensive subqueries may be used across multiple different queries. While 
> whole result caching is possible at the application level, it is not possible 
> to cache subresults in the application. This can cause bad performance for 
> queries in which the subquery is the most expensive part of the query, and 
> the application is powerless to do anything at the query level. It would be 
> good if Phoenix provided a way to cache subquery results, as it would provide 
> a significant performance gain.
> An illustrative example:
>     SELECT * FROM table1 JOIN (SELECT id_1 FROM large_table WHERE x = 10) 
> expensive_result ON table1.id_1 = expensive_result.id_2 AND table1.id_1 = 
> \{id}
> In this case, the subquery "expensive_result" is expensive to compute, but it 
> doesn't change between queries. The rest of the query does because of the 
> \{id} parameter. This means the application can't cache it, but it would be 
> good if there was a way to cache expensive_result.
> Note that there is currently a coprocessor based "server cache", but the data 
> in this "cache" is not persisted across queries. It is deleted after a TTL 
> expires (30sec by default), or when the query completes.
> This is issue is fairly high priority for us at 23andMe and we'd be happy to 
> provide a patch with some guidance from Phoenix maintainers. We are currently 
> putting together a design document for a solution, and we'll post it to this 
> Jira ticket for review in a few days.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-2715) Query Log

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-2715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437898#comment-16437898
 ] 

James Taylor commented on PHOENIX-2715:
---

[~an...@apache.org] - would it be possible to get this into 1.1 and 0.98 
branches too? Merges are becoming increasingly harder without this there.

> Query Log
> -
>
> Key: PHOENIX-2715
> URL: https://issues.apache.org/jira/browse/PHOENIX-2715
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Nick Dimiduk
>Assignee: Ankit Singhal
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-2715.patch, PHOENIX-2715_master.patch, 
> PHOENIX-2715_master_V1.patch, PHOENIX-2715_master_V2.patch
>
>
> One useful feature of other database systems is the query log. It allows the 
> DBA to review the queries run, who's run them, time taken,  This serves 
> both as an audit and also as a source of "ground truth" for performance 
> optimization. For instance, which columns should be indexed. It may also 
> serve as the foundation for automated performance recommendations/actions.
> What queries are being run is the first piece. Have this data tied into 
> tracing results and perhaps client-side metrics (PHOENIX-1819) becomes very 
> useful.
> This might take the form of clients writing data to a new system table, but 
> other implementation suggestions are welcome.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4666) Add a subquery cache that persists beyond the life of a query

[Maryann Xue (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437881#comment-16437881
 ] 

Maryann Xue commented on PHOENIX-4666:
--

Thank you for explaining 2! And now it's clear to me. It is actually an 
optimization implemented in PHOENIX-852. It doesn't not apply to all join 
queries, so I guess in the test case you have there this optimization is 
triggered. So I'm thinking two options here:
 1) Call {{HashCacheClient#evaluateKeyExpression()}} to get the key ranges if 
cache is already available on the server side, in which case 
{{CachedSubqueryResultIterator}} would still be needed but we do not add cache 
one more time. We can have a client-side cache for such key-range values as 
well. And if this is the first client building the cache for the first time, we 
get these values from calling {{addHashCache()}} and then cache them on the 
client side.
 2) A more radical but easier approach is to disable this "child-parent (FK-PK) 
join optimization" when using persistent cache. This makes some practical 
sense: if we can make a big performance gain from avoiding rebuilding the hash 
cache, it could indicate that the cache itself might be of some considerable 
side, and thus the key ranges generated from a relatively large amount of 
values might not be that useful to narrow down the scan anyway.
 For now, I actually prefer the second approach, since we can focus on the main 
part of this issue and move forward faster.

For 5: Just for simplicity. Feel like we can have less getters and "get" calls 
here.

> Add a subquery cache that persists beyond the life of a query
> -
>
> Key: PHOENIX-4666
> URL: https://issues.apache.org/jira/browse/PHOENIX-4666
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Marcell Ortutay
>Assignee: Marcell Ortutay
>Priority: Major
>
> The user list thread for additional context is here: 
> [https://lists.apache.org/thread.html/e62a6f5d79bdf7cd238ea79aed8886816d21224d12b0f1fe9b6bb075@%3Cuser.phoenix.apache.org%3E]
> 
> A Phoenix query may contain expensive subqueries, and moreover those 
> expensive subqueries may be used across multiple different queries. While 
> whole result caching is possible at the application level, it is not possible 
> to cache subresults in the application. This can cause bad performance for 
> queries in which the subquery is the most expensive part of the query, and 
> the application is powerless to do anything at the query level. It would be 
> good if Phoenix provided a way to cache subquery results, as it would provide 
> a significant performance gain.
> An illustrative example:
>     SELECT * FROM table1 JOIN (SELECT id_1 FROM large_table WHERE x = 10) 
> expensive_result ON table1.id_1 = expensive_result.id_2 AND table1.id_1 = 
> \{id}
> In this case, the subquery "expensive_result" is expensive to compute, but it 
> doesn't change between queries. The rest of the query does because of the 
> \{id} parameter. This means the application can't cache it, but it would be 
> good if there was a way to cache expensive_result.
> Note that there is currently a coprocessor based "server cache", but the data 
> in this "cache" is not persisted across queries. It is deleted after a TTL 
> expires (30sec by default), or when the query completes.
> This is issue is fairly high priority for us at 23andMe and we'd be happy to 
> provide a patch with some guidance from Phoenix maintainers. We are currently 
> putting together a design document for a solution, and we'll post it to this 
> Jira ticket for review in a few days.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4579) Add a config to conditionally create Phoenix meta tables on first client connection

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437854#comment-16437854
 ] 

Hudson commented on PHOENIX-4579:
-

FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-0.98 #1856 (See 
[https://builds.apache.org/job/Phoenix-4.x-HBase-0.98/1856/])
PHOENIX-4579 Add a config to conditionally create Phoenix meta tables on 
(jtaylor: rev c0409f815816ba6b81e4fa08647586e8ad0e3a54)
* (add) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/SystemCatalogCreationOnConnectionIT.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServicesImpl.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataProtocol.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/AppendOnlySchemaIT.java
* (edit) 
phoenix-core/src/it/java/org/apache/phoenix/end2end/MigrateSystemTablesToSystemNamespaceIT.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/exception/UpgradeRequiredException.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionQueryServices.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/coprocessor/generated/MetaDataProtos.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/DelegateConnectionQueryServices.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/query/ConnectionlessQueryServicesImpl.java
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java
* (edit) phoenix-protocol/src/main/MetaDataService.proto
* (edit) 
phoenix-core/src/test/java/org/apache/phoenix/query/ConnectionQueryServicesImplTest.java


> Add a config to conditionally create Phoenix meta tables on first client 
> connection
> ---
>
> Key: PHOENIX-4579
> URL: https://issues.apache.org/jira/browse/PHOENIX-4579
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4579.patch
>
>
> Currently we create/modify Phoenix meta tables on first client connection. 
> Adding a property to make it configurable (with default true as it is 
> currently implemented).
> With this property set to false, it will avoid lockstep upgrade requirement 
> for all clients when changing meta properties using PHOENIX-4575 as this 
> property can be flipped back on once all the clients are upgraded.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4575) Phoenix metadata KEEP_DELETED_CELLS and VERSIONS should be property driven

[Chinmay Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinmay Kulkarni updated PHOENIX-4575:
--
Attachment: PHOENIX-4575.patch

> Phoenix metadata KEEP_DELETED_CELLS and VERSIONS should be property driven
> --
>
> Key: PHOENIX-4575
> URL: https://issues.apache.org/jira/browse/PHOENIX-4575
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Attachments: PHOENIX-4575.patch, PHOENIX-4575.patch, 
> PHOENIX-4575_v2.patch
>
>
> This is to cater for circumstances where we need to alter state of 
> KEEP_DELETED_CELLS/VERSION on Phoenix meta tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4668) Remove unnecessary table descriptor modification for SPLIT_POLICY column

[Chinmay Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinmay Kulkarni updated PHOENIX-4668:
--
Attachment: PHOENIX-4668.patch

> Remove unnecessary table descriptor modification for SPLIT_POLICY column
> 
>
> Key: PHOENIX-4668
> URL: https://issues.apache.org/jira/browse/PHOENIX-4668
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Major
> Attachments: PHOENIX-4668.patch
>
>
> Inside _ConnectionQueryServicesImpl.ensureTableCreated()_, we modify the 
> table descriptor with
> newDesc.setValue(HTableDescriptor.SPLIT_POLICY, 
> MetaDataSplitPolicy.class.getName()), however we already have this mentioned 
> in the create statement DDL for system tables, so we can remove this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (PHOENIX-4668) Remove unnecessary table descriptor modification for SPLIT_POLICY column

[Chinmay Kulkarni (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4668?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chinmay Kulkarni updated PHOENIX-4668:
--
Attachment: (was: PHOENIX-4668.patch)

> Remove unnecessary table descriptor modification for SPLIT_POLICY column
> 
>
> Key: PHOENIX-4668
> URL: https://issues.apache.org/jira/browse/PHOENIX-4668
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Chinmay Kulkarni
>Assignee: Chinmay Kulkarni
>Priority: Major
>
> Inside _ConnectionQueryServicesImpl.ensureTableCreated()_, we modify the 
> table descriptor with
> newDesc.setValue(HTableDescriptor.SPLIT_POLICY, 
> MetaDataSplitPolicy.class.getName()), however we already have this mentioned 
> in the create statement DDL for system tables, so we can remove this.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4579) Add a config to conditionally create Phoenix meta tables on first client connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437760#comment-16437760
 ] 

ASF GitHub Bot commented on PHOENIX-4579:
-

Github user ChinmaySKulkarni commented on the issue:

https://github.com/apache/phoenix/pull/295
  
@JamesRTaylor the patch to 5.x-HBase-2.0 branch lgtm! Thanks!


> Add a config to conditionally create Phoenix meta tables on first client 
> connection
> ---
>
> Key: PHOENIX-4579
> URL: https://issues.apache.org/jira/browse/PHOENIX-4579
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4579.patch
>
>
> Currently we create/modify Phoenix meta tables on first client connection. 
> Adding a property to make it configurable (with default true as it is 
> currently implemented).
> With this property set to false, it will avoid lockstep upgrade requirement 
> for all clients when changing meta properties using PHOENIX-4575 as this 
> property can be flipped back on once all the clients are upgraded.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] phoenix issue #295: PHOENIX-4579: Add a config to conditionally create Phoen...

[ChinmaySKulkarni]

Github user ChinmaySKulkarni commented on the issue:

https://github.com/apache/phoenix/pull/295
  
@JamesRTaylor the patch to 5.x-HBase-2.0 branch lgtm! Thanks!


---

[jira] [Commented] (PHOENIX-4666) Add a subquery cache that persists beyond the life of a query

[Marcell Ortutay (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437759#comment-16437759
 ] 

Marcell Ortutay commented on PHOENIX-4666:
--

Thanks for taking a look [~maryannxue]. responses below:

> 1. First of all, I think it's important that we have an option to enable and 
>disable the persistent cache, making sure that users can still run join 
>queries in the default temp-cache way.

Yes definitely. In fact I am adding a hint, and for now I think it makes sense 
to only enable it if that hint is there, so we don't break any existing 
behavior.

> 2. Regarding to your change [2], can you explain what exactly is the problem 
>of key-range generation? Looks like checkCache() and addCache() are doing 
>redundant work, and CachedSubqueryResultIterator should be unnecessary. We do 
>not wish to read the cache on the client side and then re-add the cache again.

Yes in my first attempt at this I did not have the redundant work, but I ran 
into a bug where I was getting empty results when using the cached code path. 
If you look at ServerCache.addHashCache() you'll notice that it calls 
serialize(): 
[https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/join/HashCacheClient.java#L85]

serialize() produces that serialized RHS join cache and iterates over all the 
results in the ResultIterator for the RHS query. In this line 
[https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/join/HashCacheClient.java#L131]
 it adds entries to keyRangeRhsValues. This is a list of the key values on the 
RHS of the join. It is used here 
[https://github.com/apache/phoenix/blob/master/phoenix-core/src/main/java/org/apache/phoenix/execute/HashJoinPlan.java#L226]
 in HashJoinPlan as part of the query, and at some point it becomes an argument 
to set the scan range (I can dig up where if you'd like).

For this reason the cached code path somehow needs to generate correct values 
for keyRangeRhsValues, or correct values for the scan range, and these values 
need to be available on the client side.

The approach I did just re-runs the same codepath for both no-cache and cached 
queries. The advantage is it was fairly simple to implement and it guarantees 
identical execution. The downside is the redundant work. It would also be 
possible to have special case code to set the scan range for cached queries. 
This is a bit harder to implement but is more efficient.

Happy to hear what people think about this. Maybe there is something much 
simpler that I am missing!

> 3. We need to be aware that the string representation of the sub-query 
>statement is not reliable, which means the same join-tables or sub-queries do 
>not necessarily map to the same string representation, and thus will have 
>different generated cache-id. It'd be optimal if we can have some 
>normalization here. We can consider leaving this as a future improvement, yet 
>at this point we'd better have some test cases (counter cases as well) to 
>cover this point.

Yes, definitely. I'd prefer to leave this as a future improvement to keep the 
initial PR focused. IIRC I saw for some complex queries there is a "$1" or "$2" 
placeholder, which changes even across identical queries. There are probably 
more things like this, eg. "x=10 AND y=20" is the same as "y=20 AND x=10".

> 4. Is there a way for us to update the cache content if tables have been 
>updated? This might be related to what approach we take to add and re-validate 
>cache in (2).

Currently, no. I was thinking though that the user application can control 
invalidation using a hint, like this:

    /*+ CACHE_PERSISTENT('2018-01-01 12:00') */

The '2018-01-01 12:00' would be a suffix to whatever cacheId we generate, like 
this:

    cacheId = hash(cacheId + '2018-01-01 12:00')

which lets the application explicitly invalidate the cache when needed.

> 5. A rather minor point as it just occurred to me: Can we have CacheEntry 
>implement Closable?

Yes. Just so I know, what is the benefit of this?

And yes, apologies for the messy code. I'm fixing it up today and it should be 
ready for a more thorough review today or tomorrow.

> Add a subquery cache that persists beyond the life of a query
> -
>
> Key: PHOENIX-4666
> URL: https://issues.apache.org/jira/browse/PHOENIX-4666
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Marcell Ortutay
>Assignee: Marcell Ortutay
>Priority: Major
>
> The user list thread for additional context is here: 
> [https://lists.apache.org/thread.html/e62a6f5d79bdf7cd238ea79aed8886816d21224d12b0f1fe9b6bb075@%3Cuser.phoenix.apache.org%3E]
> 
> A Phoenix query may contain expensive subqueries, and moreover those 
> expensive subqueries may be used 

[jira] [Resolved] (PHOENIX-4579) Add a config to conditionally create Phoenix meta tables on first client connection

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor resolved PHOENIX-4579.
---
   Resolution: Fixed
Fix Version/s: 5.0.0
   4.14.0

Thanks for the contribution, [~ckulkarni]. I've committed to 4.x, 5.x, and 
master. You might want to double check my merges there as some of them were non 
trivial.

> Add a config to conditionally create Phoenix meta tables on first client 
> connection
> ---
>
> Key: PHOENIX-4579
> URL: https://issues.apache.org/jira/browse/PHOENIX-4579
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4579.patch
>
>
> Currently we create/modify Phoenix meta tables on first client connection. 
> Adding a property to make it configurable (with default true as it is 
> currently implemented).
> With this property set to false, it will avoid lockstep upgrade requirement 
> for all clients when changing meta properties using PHOENIX-4575 as this 
> property can be flipped back on once all the clients are upgraded.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Reopened] (PHOENIX-4496) Fix RowValueConstructorIT and IndexMetadataIT

[James Taylor (JIRA)]

 [ 
https://issues.apache.org/jira/browse/PHOENIX-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

James Taylor reopened PHOENIX-4496:
---

> Fix RowValueConstructorIT and IndexMetadataIT
> -
>
> Key: PHOENIX-4496
> URL: https://issues.apache.org/jira/browse/PHOENIX-4496
> Project: Phoenix
>  Issue Type: Sub-task
>Affects Versions: 4.14.0
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4496.patch
>
>
> {noformat}
> [ERROR] Tests run: 46, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 117.444 s <<< FAILURE! - in org.apache.phoenix.end2end.RowValueConstructorIT
> [ERROR] 
> testRVCLastPkIsTable1stPkIndex(org.apache.phoenix.end2end.RowValueConstructorIT)
>   Time elapsed: 4.516 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.RowValueConstructorIT.testRVCLastPkIsTable1stPkIndex(RowValueConstructorIT.java:1584)
> {noformat}
> {noformat}
> ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 79.381 s <<< FAILURE! - in org.apache.phoenix.end2end.index.IndexMetadataIT
> [ERROR] 
> testMutableTableOnlyHasPrimaryKeyIndex(org.apache.phoenix.end2end.index.IndexMetadataIT)
>   Time elapsed: 4.504 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.helpTestTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:662)
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.testMutableTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:623)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4496) Fix RowValueConstructorIT and IndexMetadataIT

[James Taylor (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437597#comment-16437597
 ] 

James Taylor commented on PHOENIX-4496:
---

This doesn't compile in master against HBase 1.4, so I've reverted for now.

> Fix RowValueConstructorIT and IndexMetadataIT
> -
>
> Key: PHOENIX-4496
> URL: https://issues.apache.org/jira/browse/PHOENIX-4496
> Project: Phoenix
>  Issue Type: Sub-task
>Affects Versions: 4.14.0
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4496.patch
>
>
> {noformat}
> [ERROR] Tests run: 46, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 117.444 s <<< FAILURE! - in org.apache.phoenix.end2end.RowValueConstructorIT
> [ERROR] 
> testRVCLastPkIsTable1stPkIndex(org.apache.phoenix.end2end.RowValueConstructorIT)
>   Time elapsed: 4.516 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.RowValueConstructorIT.testRVCLastPkIsTable1stPkIndex(RowValueConstructorIT.java:1584)
> {noformat}
> {noformat}
> ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 79.381 s <<< FAILURE! - in org.apache.phoenix.end2end.index.IndexMetadataIT
> [ERROR] 
> testMutableTableOnlyHasPrimaryKeyIndex(org.apache.phoenix.end2end.index.IndexMetadataIT)
>   Time elapsed: 4.504 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.helpTestTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:662)
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.testMutableTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:623)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4575) Phoenix metadata KEEP_DELETED_CELLS and VERSIONS should be property driven

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437579#comment-16437579
 ] 

ASF GitHub Bot commented on PHOENIX-4575:
-

Github user JamesRTaylor commented on the issue:

https://github.com/apache/phoenix/pull/297
  
+1. Thanks for taking this to the finish line, @ChinmaySKulkarni. Is this 
on top of your patch for PHOENIX-4579? What order should I commit them?


> Phoenix metadata KEEP_DELETED_CELLS and VERSIONS should be property driven
> --
>
> Key: PHOENIX-4575
> URL: https://issues.apache.org/jira/browse/PHOENIX-4575
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Attachments: PHOENIX-4575.patch, PHOENIX-4575_v2.patch
>
>
> This is to cater for circumstances where we need to alter state of 
> KEEP_DELETED_CELLS/VERSION on Phoenix meta tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] phoenix issue #297: PHOENIX-4575: Phoenix metadata KEEP_DELETED_CELLS and VE...

[JamesRTaylor]

Github user JamesRTaylor commented on the issue:

https://github.com/apache/phoenix/pull/297
  
+1. Thanks for taking this to the finish line, @ChinmaySKulkarni. Is this 
on top of your patch for PHOENIX-4579? What order should I commit them?


---

[jira] [Commented] (PHOENIX-4579) Add a config to conditionally create Phoenix meta tables on first client connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437568#comment-16437568
 ] 

ASF GitHub Bot commented on PHOENIX-4579:
-

Github user JamesRTaylor commented on the issue:

https://github.com/apache/phoenix/pull/295
  
+1. Nice work, @ChinmaySKulkarni! I'll get this committed. Might need some 
help with a patch that applied to 5.x-HBase-2.0 branch.


> Add a config to conditionally create Phoenix meta tables on first client 
> connection
> ---
>
> Key: PHOENIX-4579
> URL: https://issues.apache.org/jira/browse/PHOENIX-4579
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Attachments: PHOENIX-4579.patch
>
>
> Currently we create/modify Phoenix meta tables on first client connection. 
> Adding a property to make it configurable (with default true as it is 
> currently implemented).
> With this property set to false, it will avoid lockstep upgrade requirement 
> for all clients when changing meta properties using PHOENIX-4575 as this 
> property can be flipped back on once all the clients are upgraded.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] phoenix issue #295: PHOENIX-4579: Add a config to conditionally create Phoen...

[JamesRTaylor]

Github user JamesRTaylor commented on the issue:

https://github.com/apache/phoenix/pull/295
  
+1. Nice work, @ChinmaySKulkarni! I'll get this committed. Might need some 
help with a patch that applied to 5.x-HBase-2.0 branch.


---

[jira] [Commented] (PHOENIX-4496) Fix RowValueConstructorIT and IndexMetadataIT

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437284#comment-16437284
 ] 

Hudson commented on PHOENIX-4496:
-

SUCCESS: Integrated in Jenkins build Phoenix-4.x-HBase-1.3 #94 (See 
[https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/94/])
PHOENIX-4496 Fix RowValueConstructorIT and IndexMetadataIT (ankitsinghal59: rev 
672f195ae71e20f2358e16bec5c8bb7a3d608858)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/scanner/ScannerBuilder.java
PHOENIX-4496 Fix RowValueConstructorIT and IndexMetadataIT (ankitsinghal59: rev 
f543d99a358a3bd3164754bfb81f9f1582c320f9)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/hbase/index/scanner/ScannerBuilder.java


> Fix RowValueConstructorIT and IndexMetadataIT
> -
>
> Key: PHOENIX-4496
> URL: https://issues.apache.org/jira/browse/PHOENIX-4496
> Project: Phoenix
>  Issue Type: Sub-task
>Affects Versions: 4.14.0
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4496.patch
>
>
> {noformat}
> [ERROR] Tests run: 46, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 117.444 s <<< FAILURE! - in org.apache.phoenix.end2end.RowValueConstructorIT
> [ERROR] 
> testRVCLastPkIsTable1stPkIndex(org.apache.phoenix.end2end.RowValueConstructorIT)
>   Time elapsed: 4.516 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.RowValueConstructorIT.testRVCLastPkIsTable1stPkIndex(RowValueConstructorIT.java:1584)
> {noformat}
> {noformat}
> ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 79.381 s <<< FAILURE! - in org.apache.phoenix.end2end.index.IndexMetadataIT
> [ERROR] 
> testMutableTableOnlyHasPrimaryKeyIndex(org.apache.phoenix.end2end.index.IndexMetadataIT)
>   Time elapsed: 4.504 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.helpTestTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:662)
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.testMutableTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:623)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4688) Add kerberos authentication to python-phoenixdb

[Lev Bronshtein (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437183#comment-16437183
 ] 

Lev Bronshtein commented on PHOENIX-4688:
-

Another potential issue with this approach is that some user may at some point 
override our version of requests-kerberos with a newer one.  This can be 
avoided if a local phoenix install decides to have its own python environment

> Add kerberos authentication to python-phoenixdb
> ---
>
> Key: PHOENIX-4688
> URL: https://issues.apache.org/jira/browse/PHOENIX-4688
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Lev Bronshtein
>Priority: Minor
>
> In its current state python-phoenixdv does not support support kerberos 
> authentication.  Using a modern python http library such as requests or 
> urllib it would be simple (if not trivial) to add this support.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4496) Fix RowValueConstructorIT and IndexMetadataIT

[Ankit Singhal (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4496?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16437095#comment-16437095
 ] 

Ankit Singhal commented on PHOENIX-4496:


Thanks [~sergey.soldatov] for the review, pushed to all branches now.

> Fix RowValueConstructorIT and IndexMetadataIT
> -
>
> Key: PHOENIX-4496
> URL: https://issues.apache.org/jira/browse/PHOENIX-4496
> Project: Phoenix
>  Issue Type: Sub-task
>Affects Versions: 4.14.0
>Reporter: Rajeshbabu Chintaguntla
>Assignee: Ankit Singhal
>Priority: Major
>  Labels: HBase-2.0
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4496.patch
>
>
> {noformat}
> [ERROR] Tests run: 46, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 117.444 s <<< FAILURE! - in org.apache.phoenix.end2end.RowValueConstructorIT
> [ERROR] 
> testRVCLastPkIsTable1stPkIndex(org.apache.phoenix.end2end.RowValueConstructorIT)
>   Time elapsed: 4.516 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.RowValueConstructorIT.testRVCLastPkIsTable1stPkIndex(RowValueConstructorIT.java:1584)
> {noformat}
> {noformat}
> ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 79.381 s <<< FAILURE! - in org.apache.phoenix.end2end.index.IndexMetadataIT
> [ERROR] 
> testMutableTableOnlyHasPrimaryKeyIndex(org.apache.phoenix.end2end.index.IndexMetadataIT)
>   Time elapsed: 4.504 s  <<< FAILURE!
> java.lang.AssertionError
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.helpTestTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:662)
> at 
> org.apache.phoenix.end2end.index.IndexMetadataIT.testMutableTableOnlyHasPrimaryKeyIndex(IndexMetadataIT.java:623)
> {noformat}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4689) cannot bind enum descriptor to a non-enum class

[Pedro Boado (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436985#comment-16436985
 ] 

Pedro Boado commented on PHOENIX-4689:
--

Hi, that version has neither released not supported by Apache Phoenix. Please 
raise the issue with Cloudera. 

We have started building packages for Phoenix and CDH starting with 4.13.2 and 
only for CDH 5.11.2 . Next versions will hopefully support a wider range of CDH 
versions starting in CDH 5.11.x 

> cannot bind enum descriptor to a non-enum class
> ---
>
> Key: PHOENIX-4689
> URL: https://issues.apache.org/jira/browse/PHOENIX-4689
> Project: Phoenix
>  Issue Type: Bug
>Affects Versions: 4.7.0
> Environment: phoenix-4.7.0-cdh5.5.1
> pig-0.12.0-cdh5.5.0
> hadoop-2.6.0-cdh5.5.0
> hbase.1.0-cdh5.5.0
>Reporter: ZhongyuWang
>Priority: Major
>
> [https://github.com/chiastic-security/phoenix-for-cloudera]
> I 'use phoenix-4.7.0-cdh5.5.1'phoenix version from above link in github, but 
> where i use pig to load data to HDFS from hbase with mapreduce , i got 
> "cannot bind enum descriptor to a non-enum class" error log. i can run it in 
> local mapreduce mode successfully.
> pig -x mapreduce example1.pig
> example1.pig
> REGISTER 
> /e3base/phoenix/phoenix-4.7.0-cdh5.5.1/phoenix-4.7.0-cdh5.5.1-client.jar;
> rows = load 'hbase://query/SELECT ID,ACCOUNT,PASSWD FROM USER' USING 
> org.apache.phoenix.pig.PhoenixHBaseLoader('KFAPP74:11001');
> STORE rows INTO 'USER.csv' USING PigStorage(',');
> Mapreduce error log
> [main] INFO 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher
>  - Failed!
> [main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2997: Unable to 
> recreate exception from backed error: 
> AttemptID:attempt_1515656040682_0049_m_00_3 Info:Error: 
> java.io.IOException: Deserialization error: cannot bind enum descriptor to a 
> non-enum class
>  at 
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:62)
>  at 
> org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.setup(PigGenericMapBase.java:171)
>  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
>  at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
>  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
>  at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
>  at java.security.AccessController.doPrivileged(Native Method)
>  at javax.security.auth.Subject.doAs(Subject.java:415)
>  at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
>  at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
> Caused by: java.io.InvalidClassException: cannot bind enum descriptor to a 
> non-enum class
>  at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:608)
>  at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620)
>  at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515)
>  at java.io.ObjectInputStream.readEnum(ObjectInputStream.java:1723)
>  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345)
>  at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
>  at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
>  at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
>  at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
>  at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
>  at 
> org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:60)
>  ... 9 more
>  
>  
>  
>  
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (PHOENIX-4689) cannot bind enum descriptor to a non-enum class

[ZhongyuWang (JIRA)]

ZhongyuWang created PHOENIX-4689:


 Summary: cannot bind enum descriptor to a non-enum class
 Key: PHOENIX-4689
 URL: https://issues.apache.org/jira/browse/PHOENIX-4689
 Project: Phoenix
  Issue Type: Bug
Affects Versions: 4.7.0
 Environment: phoenix-4.7.0-cdh5.5.1

pig-0.12.0-cdh5.5.0

hadoop-2.6.0-cdh5.5.0

hbase.1.0-cdh5.5.0
Reporter: ZhongyuWang


[https://github.com/chiastic-security/phoenix-for-cloudera]

I 'use phoenix-4.7.0-cdh5.5.1'phoenix version from above link in github, but 
where i use pig to load data to HDFS from hbase with mapreduce , i got "cannot 
bind enum descriptor to a non-enum class" error log. i can run it in local 
mapreduce mode successfully.

pig -x mapreduce example1.pig

example1.pig

REGISTER 
/e3base/phoenix/phoenix-4.7.0-cdh5.5.1/phoenix-4.7.0-cdh5.5.1-client.jar;
rows = load 'hbase://query/SELECT ID,ACCOUNT,PASSWD FROM USER' USING 
org.apache.phoenix.pig.PhoenixHBaseLoader('KFAPP74:11001');
STORE rows INTO 'USER.csv' USING PigStorage(',');

Mapreduce error log

[main] INFO 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher 
- Failed!
[main] ERROR org.apache.pig.tools.grunt.GruntParser - ERROR 2997: Unable to 
recreate exception from backed error: 
AttemptID:attempt_1515656040682_0049_m_00_3 Info:Error: 
java.io.IOException: Deserialization error: cannot bind enum descriptor to a 
non-enum class
 at 
org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:62)
 at 
org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigGenericMapBase.setup(PigGenericMapBase.java:171)
 at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
 at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:787)
 at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
 at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
 at java.security.AccessController.doPrivileged(Native Method)
 at javax.security.auth.Subject.doAs(Subject.java:415)
 at 
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
 at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.io.InvalidClassException: cannot bind enum descriptor to a 
non-enum class
 at java.io.ObjectStreamClass.initNonProxy(ObjectStreamClass.java:608)
 at java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1620)
 at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1515)
 at java.io.ObjectInputStream.readEnum(ObjectInputStream.java:1723)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1345)
 at java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:1989)
 at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:1913)
 at java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:1796)
 at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1348)
 at java.io.ObjectInputStream.readObject(ObjectInputStream.java:370)
 at 
org.apache.pig.impl.util.ObjectSerializer.deserialize(ObjectSerializer.java:60)
 ... 9 more

 

 

 

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4666) Add a subquery cache that persists beyond the life of a query

[Maryann Xue (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436978#comment-16436978
 ] 

Maryann Xue commented on PHOENIX-4666:
--

Thank you very much for your work, [~ortutay]! Here's a few things on the 
high-level:
1. First of all, I think it's important that we have an option to enable and 
disable the persistent cache, making sure that users can still run join queries 
in the default temp-cache way.
2. Regarding to your change [2], can you explain what exactly is the problem of 
key-range generation? Looks like checkCache() and addCache() are doing 
redundant work, and CachedSubqueryResultIterator should be unnecessary. We do 
not wish to read the cache on the client side and then re-add the cache again.
3. We need to be aware that the string representation of the sub-query 
statement is not reliable, which means the same join-tables or sub-queries do 
not necessarily map to the same string representation, and thus will have 
different generated cache-id. It'd be optimal if we can have some normalization 
here. We can consider leaving this as a future improvement, yet at this point 
we'd better have some test cases (counter cases as well) to cover this point.
4. Is there a way for us to update the cache content if tables have been 
updated? This might be related to what approach we take to add and re-validate 
cache in (2).
5. A rather minor point as it just occurred to me: Can we have CacheEntry 
implement Closable?
Lastly, I understand that it's work in progress, but as we move on, could you 
please do a little clean-up so it would be easier for discussions and code 
reviews? For example, correct the indentation (make sure there's no tabs); 
instead of commenting out a line of code, can you just remove it; and get rid 
of all "system.out.println" or replace them with logging if necessary?

> Add a subquery cache that persists beyond the life of a query
> -
>
> Key: PHOENIX-4666
> URL: https://issues.apache.org/jira/browse/PHOENIX-4666
> Project: Phoenix
>  Issue Type: Improvement
>Reporter: Marcell Ortutay
>Assignee: Marcell Ortutay
>Priority: Major
>
> The user list thread for additional context is here: 
> [https://lists.apache.org/thread.html/e62a6f5d79bdf7cd238ea79aed8886816d21224d12b0f1fe9b6bb075@%3Cuser.phoenix.apache.org%3E]
> 
> A Phoenix query may contain expensive subqueries, and moreover those 
> expensive subqueries may be used across multiple different queries. While 
> whole result caching is possible at the application level, it is not possible 
> to cache subresults in the application. This can cause bad performance for 
> queries in which the subquery is the most expensive part of the query, and 
> the application is powerless to do anything at the query level. It would be 
> good if Phoenix provided a way to cache subquery results, as it would provide 
> a significant performance gain.
> An illustrative example:
>     SELECT * FROM table1 JOIN (SELECT id_1 FROM large_table WHERE x = 10) 
> expensive_result ON table1.id_1 = expensive_result.id_2 AND table1.id_1 = 
> \{id}
> In this case, the subquery "expensive_result" is expensive to compute, but it 
> doesn't change between queries. The rest of the query does because of the 
> \{id} parameter. This means the application can't cache it, but it would be 
> good if there was a way to cache expensive_result.
> Note that there is currently a coprocessor based "server cache", but the data 
> in this "cache" is not persisted across queries. It is deleted after a TTL 
> expires (30sec by default), or when the query completes.
> This is issue is fairly high priority for us at 23andMe and we'd be happy to 
> provide a patch with some guidance from Phoenix maintainers. We are currently 
> putting together a design document for a solution, and we'll post it to this 
> Jira ticket for review in a few days.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4575) Phoenix metadata KEEP_DELETED_CELLS and VERSIONS should be property driven

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436945#comment-16436945
 ] 

ASF GitHub Bot commented on PHOENIX-4575:
-

Github user ChinmaySKulkarni commented on the issue:

https://github.com/apache/phoenix/pull/297
  
@JamesRTaylor rebased. @ankitsinghal phoenix build seems to be failing 
after commit 70c9c6bb2d2073bb40640a0a794032b166c4b63b for PHOENIX-4496. Please 
take a look.


> Phoenix metadata KEEP_DELETED_CELLS and VERSIONS should be property driven
> --
>
> Key: PHOENIX-4575
> URL: https://issues.apache.org/jira/browse/PHOENIX-4575
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Attachments: PHOENIX-4575.patch, PHOENIX-4575_v2.patch
>
>
> This is to cater for circumstances where we need to alter state of 
> KEEP_DELETED_CELLS/VERSION on Phoenix meta tables.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] phoenix issue #297: PHOENIX-4575: Phoenix metadata KEEP_DELETED_CELLS and VE...

[ChinmaySKulkarni]

Github user ChinmaySKulkarni commented on the issue:

https://github.com/apache/phoenix/pull/297
  
@JamesRTaylor rebased. @ankitsinghal phoenix build seems to be failing 
after commit 70c9c6bb2d2073bb40640a0a794032b166c4b63b for PHOENIX-4496. Please 
take a look.


---

[jira] [Commented] (PHOENIX-4605) Support running multiple transaction providers

[Hudson (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4605?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436940#comment-16436940
 ] 

Hudson commented on PHOENIX-4605:
-

ABORTED: Integrated in Jenkins build Phoenix-4.x-HBase-1.3 #93 (See 
[https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/93/])
PHOENIX-4605 Support running multiple transaction providers (ammendment) 
(jtaylor: rev e79228c4f35fafedd7a5c7ded0c053fc561846e7)
* (edit) 
phoenix-core/src/main/java/org/apache/phoenix/schema/MetaDataClient.java


> Support running multiple transaction providers
> --
>
> Key: PHOENIX-4605
> URL: https://issues.apache.org/jira/browse/PHOENIX-4605
> Project: Phoenix
>  Issue Type: Bug
>Reporter: James Taylor
>Assignee: James Taylor
>Priority: Major
> Fix For: 4.14.0, 5.0.0
>
> Attachments: PHOENIX-4605_v1.patch, PHOENIX-4605_v2.patch, 
> PHOENIX-4605_v3.patch, PHOENIX-4605_wip1.patch, PHOENIX-4605_wip2.patch, 
> PHOENIX_4605_wip3.patch
>
>
> We should deprecate QueryServices.DEFAULT_TABLE_ISTRANSACTIONAL_ATTRIB and 
> instead have a QueryServices.DEFAULT_TRANSACTION_PROVIDER now that we'll have 
> two transaction providers: Tephra and Omid. Along the same lines, we should 
> add a TRANSACTION_PROVIDER column to SYSTEM.CATALOG  and stop using the 
> IS_TRANSACTIONAL table property. For backwards compatibility, we can assume 
> the provider is Tephra if the existing properties are set to true.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (PHOENIX-4579) Add a config to conditionally create Phoenix meta tables on first client connection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/PHOENIX-4579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16436933#comment-16436933
 ] 

ASF GitHub Bot commented on PHOENIX-4579:
-

Github user ChinmaySKulkarni commented on the issue:

https://github.com/apache/phoenix/pull/295
  
Squashed all commits and rebased. @JamesRTaylor 


> Add a config to conditionally create Phoenix meta tables on first client 
> connection
> ---
>
> Key: PHOENIX-4579
> URL: https://issues.apache.org/jira/browse/PHOENIX-4579
> Project: Phoenix
>  Issue Type: New Feature
>Reporter: Mujtaba Chohan
>Assignee: Chinmay Kulkarni
>Priority: Major
> Attachments: PHOENIX-4579.patch
>
>
> Currently we create/modify Phoenix meta tables on first client connection. 
> Adding a property to make it configurable (with default true as it is 
> currently implemented).
> With this property set to false, it will avoid lockstep upgrade requirement 
> for all clients when changing meta properties using PHOENIX-4575 as this 
> property can be flipped back on once all the clients are upgraded.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[GitHub] phoenix issue #295: PHOENIX-4579: Add a config to conditionally create Phoen...

[ChinmaySKulkarni]

Github user ChinmaySKulkarni commented on the issue:

https://github.com/apache/phoenix/pull/295
  
Squashed all commits and rebased. @JamesRTaylor 


---