Re: [DISCUSS] Working towards a connectors release
+1. At $dayjob we have a legacy feature that uses phoenix-pig, but I believe that usage is scheduled for deprecation soon and we can maintain it in our internal fork until then. Pig hasn't had a release in 6 years and last I heard doesn't support Hadoop 3; no reason to keep supporting it. Geoffrey On Tue, Apr 18, 2023 at 1:37 AM Istvan Toth wrote: > Hi! > > We've never had a connectors release, because of multiple unsolved > problems. > Some, like java 11/17 support are relatively straightforward and don't > really need discussion, but some are more impactful. > > I propose the following plan, which should give us a chance to have a > release in the foreseeable future: > Disclosure: at $dayjob, we only support the Spark and HBase connectors, and > those are the ones we can dedicate resources to. > > *- Drop the connectors for Phoenix 4.x* > 4.x is EOL, and it complicates the project structure, build time, etc. > We've never had a release for 4.x either. > > *- Drop the Kafka connector* > It has CVEs, and only works with an ancient Kafka version. > I have also seen zero developer or user interest in it. > If someone volunteers to update and maintain it, we can always add it back > later > > > *- Drop the Pig connector*This doesn't have critical problems, but I have > seen zero interest in it. > The shaded artifact doesn't use maven-shade-plugin, and I suspect that it > would have classpath conflict issues. > Fixing up the shading to be on par with the rest of the connectors would be > a non-trivial amount of work. > If someone volunteers to update and maintain it, we can always add it back > later. > > *- Re-shade the hive 3 connector for hbase-shaded* > Hbase in Hive 3 is very broken, we already need to replace the shipped > HBase jars anyway. > To avoid conflict with the included hbase jars, we want to avoid > duplicating them. > The solution is to omit the Hbase and Hadoop JARs from the shaded > connector, and change the relocations > to handle the binary incompatibilities between the shaded and non-shaded > HBase API. > We already do this for Spark, and this is also how the Hive 4 connector > will have to work. > (This already works well at $dayjob ) > > This would leave us with only three connectors, but those would at least be > released, and easier to support: > Spark 2 > Spark 3 > Hive 3 > > Please share your thoughts! > > Istvan >
[jira] [Resolved] (PHOENIX-6898) Index tests fail with HBase 2.5
[ https://issues.apache.org/jira/browse/PHOENIX-6898?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Khurana resolved PHOENIX-6898. Fix Version/s: 5.2.0 5.1.4 Resolution: Fixed > Index tests fail with HBase 2.5 > --- > > Key: PHOENIX-6898 > URL: https://issues.apache.org/jira/browse/PHOENIX-6898 > Project: Phoenix > Issue Type: Bug > Components: core >Affects Versions: 5.2.0, 5.1.4 >Reporter: Istvan Toth >Assignee: Tanuj Khurana >Priority: Blocker > Fix For: 5.2.0, 5.1.4 > > > A lot of indexing tests fail with HBase 2.5. > We haven't had a successful 2.5 run on master with HBase 2.5 since last > August. > The last successful run on 5.1 was on Jan 26. > [https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/5.1/] > [https://ci-hadoop.apache.org/job/Phoenix/job/Phoenix-mulitbranch/job/master/] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (PHOENIX-6929) Build failing on master branch.
[
https://issues.apache.org/jira/browse/PHOENIX-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani resolved PHOENIX-6929.
---
Resolution: Fixed
> Build failing on master branch.
> ---
>
> Key: PHOENIX-6929
> URL: https://issues.apache.org/jira/browse/PHOENIX-6929
> Project: Phoenix
> Issue Type: Bug
> Components: core
>Affects Versions: 5.2.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Critical
> Fix For: 5.2.0
>
>
> Ran the following command:
> mvn clean install -DskipTests -Dhbase.profile=2.5
> This failed with the following error.
> {noformat}
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 4.429 s
> [INFO] Finished at: 2023-04-18T09:09:37-07:00
> [INFO]
>
> [ERROR] Failed to execute goal on project phoenix-hbase-compat-2.5.0: Could
> not resolve dependencies for project
> org.apache.phoenix:phoenix-hbase-compat-2.5.0:jar:5.2.0-SNAPSHOT: Failed to
> collect dependencies at org.apache.hbase:hbase-server:jar:2.5.0 ->
> org.glassfish.web:javax.servlet.jsp:jar:2.3.2 ->
> org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Failed to read artifact
> descriptor for org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Could not
> transfer artifact org.glassfish:javax.el:pom:3.0.1-b06-SNAPSHOT from/to
> jvnet-nexus-snapshots
> (https://maven.java.net/content/repositories/snapshots): transfer failed for
> https://maven.java.net/content/repositories/snapshots/org/glassfish/javax.el/3.0.1-b06-SNAPSHOT/javax.el-3.0.1-b06-SNAPSHOT.pom:
> PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target -> [Help 1]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Assigned] (PHOENIX-6929) Build failing on master branch.
[
https://issues.apache.org/jira/browse/PHOENIX-6929?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Viraj Jasani reassigned PHOENIX-6929:
-
Assignee: Rushabh Shah
> Build failing on master branch.
> ---
>
> Key: PHOENIX-6929
> URL: https://issues.apache.org/jira/browse/PHOENIX-6929
> Project: Phoenix
> Issue Type: Bug
> Components: core
>Affects Versions: 5.2.0
>Reporter: Rushabh Shah
>Assignee: Rushabh Shah
>Priority: Critical
> Fix For: 5.2.0
>
>
> Ran the following command:
> mvn clean install -DskipTests -Dhbase.profile=2.5
> This failed with the following error.
> {noformat}
> [INFO]
>
> [INFO] BUILD FAILURE
> [INFO]
>
> [INFO] Total time: 4.429 s
> [INFO] Finished at: 2023-04-18T09:09:37-07:00
> [INFO]
>
> [ERROR] Failed to execute goal on project phoenix-hbase-compat-2.5.0: Could
> not resolve dependencies for project
> org.apache.phoenix:phoenix-hbase-compat-2.5.0:jar:5.2.0-SNAPSHOT: Failed to
> collect dependencies at org.apache.hbase:hbase-server:jar:2.5.0 ->
> org.glassfish.web:javax.servlet.jsp:jar:2.3.2 ->
> org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Failed to read artifact
> descriptor for org.glassfish:javax.el:jar:3.0.1-b06-SNAPSHOT: Could not
> transfer artifact org.glassfish:javax.el:pom:3.0.1-b06-SNAPSHOT from/to
> jvnet-nexus-snapshots
> (https://maven.java.net/content/repositories/snapshots): transfer failed for
> https://maven.java.net/content/repositories/snapshots/org/glassfish/javax.el/3.0.1-b06-SNAPSHOT/javax.el-3.0.1-b06-SNAPSHOT.pom:
> PKIX path building failed:
> sun.security.provider.certpath.SunCertPathBuilderException: unable to find
> valid certification path to requested target -> [Help 1]
> {noformat}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (PHOENIX-6884) Phoenix to use hbase.rpc.read.timeout and hbase.rpc.write.timeout
[ https://issues.apache.org/jira/browse/PHOENIX-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Khurana updated PHOENIX-6884: --- Fix Version/s: (was: 5.2.0) (was: 5.1.4) > Phoenix to use hbase.rpc.read.timeout and hbase.rpc.write.timeout > - > > Key: PHOENIX-6884 > URL: https://issues.apache.org/jira/browse/PHOENIX-6884 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir Ozdemir >Assignee: Tanuj Khurana >Priority: Major > > Phoenix uses the same RPC timeout, hbase.rpc.timeout, for both read and write > operations currently. HBASE-15866 split hbase.rpc.timeout into > hbase.rpc.read.timeout and hbase.rpc.write.timeout. > The paging feature (PHOENIX-6211, PHOENIX-6207 and PHOENIX-5998) slices > server side operations into small chunks (i.e., pages) and allows all queries > make progress without timeouts. This feature makes Phoenix a better > time-sharing system and thus improves availability. > In order to take advantage of the paging feature fully, we need to set the > timeout for scan RPCs to a small value. While it is reasonable to reduce the > RPC timeout for the read path because of the paging feature, it is not safe > to reduce it for the write path drastically. This is because of the batch > writes and synchronous index updates within a batch write. This means we need > to start separately configuring read and write RPC timeouts. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Reopened] (PHOENIX-6884) Phoenix to use hbase.rpc.read.timeout and hbase.rpc.write.timeout
[ https://issues.apache.org/jira/browse/PHOENIX-6884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tanuj Khurana reopened PHOENIX-6884: Assignee: Tanuj Khurana (was: Kadir Ozdemir) > Phoenix to use hbase.rpc.read.timeout and hbase.rpc.write.timeout > - > > Key: PHOENIX-6884 > URL: https://issues.apache.org/jira/browse/PHOENIX-6884 > Project: Phoenix > Issue Type: Improvement >Reporter: Kadir Ozdemir >Assignee: Tanuj Khurana >Priority: Major > Fix For: 5.2.0, 5.1.4 > > > Phoenix uses the same RPC timeout, hbase.rpc.timeout, for both read and write > operations currently. HBASE-15866 split hbase.rpc.timeout into > hbase.rpc.read.timeout and hbase.rpc.write.timeout. > The paging feature (PHOENIX-6211, PHOENIX-6207 and PHOENIX-5998) slices > server side operations into small chunks (i.e., pages) and allows all queries > make progress without timeouts. This feature makes Phoenix a better > time-sharing system and thus improves availability. > In order to take advantage of the paging feature fully, we need to set the > timeout for scan RPCs to a small value. While it is reasonable to reduce the > RPC timeout for the read path because of the paging feature, it is not safe > to reduce it for the write path drastically. This is because of the batch > writes and synchronous index updates within a batch write. This means we need > to start separately configuring read and write RPC timeouts. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (PHOENIX-6931) Switch queryserver to log4j2
[ https://issues.apache.org/jira/browse/PHOENIX-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth reassigned PHOENIX-6931: Assignee: Istvan Toth > Switch queryserver to log4j2 > > > Key: PHOENIX-6931 > URL: https://issues.apache.org/jira/browse/PHOENIX-6931 > Project: Phoenix > Issue Type: Bug > Components: queryserver >Affects Versions: queryserver-6.0.0 >Reporter: Istvan Toth >Assignee: Istvan Toth >Priority: Blocker > > On the other components we've moved to log4j2. > PQS is still using reload4j. > PQS should not be the odd man out, we need to move to log4j2 there as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (PHOENIX-6931) Switch queryserver to log4j2
[ https://issues.apache.org/jira/browse/PHOENIX-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated PHOENIX-6931: - Description: On the other components we've moved to log4j2. PQS is still using reload4j. PQS should not be the odd man out, we need to move to log4j2 there as well. was: On phoenix we've moved to log4j2. PQS is still using reload4j. PQS should not be the odd man out, we need to move to log4j2 there as well. > Switch queryserver to log4j2 > > > Key: PHOENIX-6931 > URL: https://issues.apache.org/jira/browse/PHOENIX-6931 > Project: Phoenix > Issue Type: Bug > Components: queryserver >Affects Versions: queryserver-6.0.0 >Reporter: Istvan Toth >Priority: Blocker > > On the other components we've moved to log4j2. > PQS is still using reload4j. > PQS should not be the odd man out, we need to move to log4j2 there as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (PHOENIX-6931) Switch queryserver to log4j2
[ https://issues.apache.org/jira/browse/PHOENIX-6931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Istvan Toth updated PHOENIX-6931: - Summary: Switch queryserver to log4j2 (was: Swith queryserver to log4j2) > Switch queryserver to log4j2 > > > Key: PHOENIX-6931 > URL: https://issues.apache.org/jira/browse/PHOENIX-6931 > Project: Phoenix > Issue Type: Bug > Components: queryserver >Affects Versions: queryserver-6.0.0 >Reporter: Istvan Toth >Priority: Blocker > > On phoenix we've moved to log4j2. > PQS is still using reload4j. > PQS should not be the odd man out, we need to move to log4j2 there as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (PHOENIX-6931) Swith queryserver to log4j2
Istvan Toth created PHOENIX-6931: Summary: Swith queryserver to log4j2 Key: PHOENIX-6931 URL: https://issues.apache.org/jira/browse/PHOENIX-6931 Project: Phoenix Issue Type: Bug Components: queryserver Affects Versions: queryserver-6.0.0 Reporter: Istvan Toth On phoenix we've moved to log4j2. PQS is still using reload4j. PQS should not be the odd man out, we need to move to log4j2 there as well. -- This message was sent by Atlassian Jira (v8.20.10#820010)
