+1 to another repo for connectors On Mon, Dec 3, 2018 at 6:27 PM James Taylor <jamestay...@apache.org> wrote:
> +1. Good idea, Thomas. > > On Mon, Dec 3, 2018 at 2:57 PM Thomas D'Silva <tdsi...@salesforce.com> > wrote: > > > I believe we will be maintaining the 4.x branches which support HBase > 1.2, > > 1.3 and 1.4 for a while. > > Should we think about pulling out the connectors and queryserver into > their > > own repo similar to > > what HBase did (see https://issues.apache.org/jira/browse/HBASE-20934). > > They could then have > > their own release schedule and Java support. > > > > > > On Sat, Dec 1, 2018 at 6:53 AM Pedro Boado <pedro.bo...@gmail.com> > wrote: > > > > > Well I don't count with a lot more 4.x releases - maybe I'm > wrong-headed > > . > > > For master branch and cdh6 we'd be looking at spark 2.x > > > > > > Part of the success of a project is about version stability. Not a lot > of > > > corporate projects can afford keep upgrading to the latest versions - > > think > > > about it, you're in production, with a few thousand lines code running > > > spark 1.6 ... And for upgrading to 4.14 you need to review all of this > > > spark code -and maybe recompile to scala 2.11 btw- . It doesn't make > > sense. > > > Until now 4.x was pretty stable and in my opinion it should've never > been > > > migrated to spark 2 and java 8. Minor versions should keep certain > > > stability in terms of dependencies. > > > > > > All these changes should've come with phoenix 5. But you're right it > > needs > > > a sensible solution as 4.14.1 is already out and compiled with java8. > > > > > > > > > > > > On Fri, 30 Nov 2018, 23:28 Thomas D'Silva <tdsi...@salesforce.com > wrote: > > > > > > > Spark 1.6 is really old and doesn't support the newer Datasource v2 > api > > > > that we have been looking at integrating with. > > > > As Alex points out you will might end up having to revert a lot more > > > > commits in the future. > > > > Seems like the queryserver and phoenix-spark modules on the cdh > branch > > > > would end up diverging a lot from the standard open source branch. > > > > > > > > > > > > On Fri, Nov 30, 2018 at 2:23 PM Alex Araujo <alexara...@gmail.com> > > > wrote: > > > > > > > > > > Only a downgrade to spark 1.6 ( > > > > > changes are only needed in a few IT, basically going back from > > Datasets > > > > to > > > > > Dataframes) and going back to Avatica 1.10 ( involving reverting > > > > > PHOENIX-4755, PHOENIX-4750 and PHOENIX-4805 ). > > > > > > > > > > We're talking about the 4.x branches, right? Doesn't seem prudent > to > > do > > > > it > > > > > there as down-streamers may already be relying on the newer > versions. > > > > > > > > > > On Fri, Nov 30, 2018 at 4:18 PM Pedro Boado <pedro.bo...@gmail.com > > > > > > wrote: > > > > > > > > > > > Thinking about typical server installation in a corporate > > environment > > > > I'd > > > > > > keep everything compatible with the same JVM version. > > > > > > > > > > > > I've gone down the route for the cdh branch. Full JDK 7 > > compatibility > > > > > > doesn't require changes in phoenix-core. Only a downgrade to > spark > > > 1.6 > > > > ( > > > > > > changes are only needed in a few IT, basically going back from > > > Datasets > > > > > to > > > > > > Dataframes) and going back to Avatica 1.10 ( involving reverting > > > > > > PHOENIX-4755, PHOENIX-4750 and PHOENIX-4805 ). > > > > > > > > > > > > > > > > > > On Fri, 30 Nov 2018, 18:57 Thomas D'Silva <tsi...@salesforce.com > > > > > > <tdsi...@salesforce.com> wrote: > > > > > > > > > > > > > We could allow individual submodules like the queryserver, or > > > > > > phoenix-spark > > > > > > > to be built with their own compiler configuration (1.8+). > > > > > > > This would allow these modules to use Java 1.8 features. I > think > > > this > > > > > > would > > > > > > > be a good compromise given that they depend on > > > > > > > features that are provided by versions of spark and avatica > that > > no > > > > > > longer > > > > > > > support Java 1.7. > > > > > > > We can still ensure phoenix-core supports Java 1.7. You would > > have > > > to > > > > > > skip > > > > > > > building modules that require Java 1.8, WDYT? > > > > > > > > > > > > > > On Thu, Nov 29, 2018 at 6:11 PM Jaanai Zhang < > > > cloud.pos...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > I'd vote for keep using java7 on 4.x branches. if upgrades to > > > > java8, > > > > > it > > > > > > > > will impact users who want to upgrade the latest 4.x > branches. > > > they > > > > > > must > > > > > > > > consider using java8 in their running environments, maybe > > their > > > > > > > libraries > > > > > > > > do not support java8, then they have to give up to upgrade. > So > > I > > > > > think > > > > > > > that > > > > > > > > drops support java7 is not friendly for some users. > > > > > > > > > > > > > > > > > > > > > > > > ---------------------------------------- > > > > > > > > Jaanai Zhang > > > > > > > > Best regards! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Pedro Boado <pedro.bo...@gmail.com> 于2018年11月30日周五 上午6:13写道： > > > > > > > > > > > > > > > > > I'd vote for keep compiling 4.x branches in java7. It makes > > > sense > > > > > as > > > > > > > it's > > > > > > > > > just a new minor release. > > > > > > > > > > > > > > > > > > It's pretty easy reverting back to spark 1.6 and also > avatica > > > > > > > dependency > > > > > > > > > could be reverted to the previous version. > > > > > > > > > > > > > > > > > > On 29 Nov 2018 21:41, "Thomas D'Silva" < > > tdsi...@salesforce.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > We have traditionally followed HBase's java support (see > > > > > > > > > https://hbase.apache.org/book.html#basic.prerequisites). > The > > > > > > > > > phoenix-queryserver module has a dependency on Avatica > which > > > does > > > > > not > > > > > > > > > support Java 1.7. The phoenix-spark module depends on spark > > > 2.3.2 > > > > > > which > > > > > > > > > also does not support Java 1.7. Do folks feel we should > > > continue > > > > to > > > > > > > > provide > > > > > > > > > support Java 1.7 on the 1.x branches? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Fri, 30 Nov 2018, 18:57 Thomas D'Silva < > tdsi...@salesforce.com > > > > wrote: > > > > > > > > > > > > > We could allow individual submodules like the queryserver, or > > > > > > phoenix-spark > > > > > > > to be built with their own compiler configuration (1.8+). > > > > > > > This would allow these modules to use Java 1.8 features. I > think > > > this > > > > > > would > > > > > > > be a good compromise given that they depend on > > > > > > > features that are provided by versions of spark and avatica > that > > no > > > > > > longer > > > > > > > support Java 1.7. > > > > > > > We can still ensure phoenix-core supports Java 1.7. You would > > have > > > to > > > > > > skip > > > > > > > building modules that require Java 1.8, WDYT? > > > > > > > > > > > > > > On Thu, Nov 29, 2018 at 6:11 PM Jaanai Zhang < > > > cloud.pos...@gmail.com > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > I'd vote for keep using java7 on 4.x branches. if upgrades to > > > > java8, > > > > > it > > > > > > > > will impact users who want to upgrade the latest 4.x > branches. > > > they > > > > > > must > > > > > > > > consider using java8 in their running environments, maybe > > their > > > > > > > libraries > > > > > > > > do not support java8, then they have to give up to upgrade. > So > > I > > > > > think > > > > > > > that > > > > > > > > drops support java7 is not friendly for some users. > > > > > > > > > > > > > > > > > > > > > > > > ---------------------------------------- > > > > > > > > Jaanai Zhang > > > > > > > > Best regards! > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > Pedro Boado <pedro.bo...@gmail.com> 于2018年11月30日周五 上午6:13写道： > > > > > > > > > > > > > > > > > I'd vote for keep compiling 4.x branches in java7. It makes > > > sense > > > > > as > > > > > > > it's > > > > > > > > > just a new minor release. > > > > > > > > > > > > > > > > > > It's pretty easy reverting back to spark 1.6 and also > avatica > > > > > > > dependency > > > > > > > > > could be reverted to the previous version. > > > > > > > > > > > > > > > > > > On 29 Nov 2018 21:41, "Thomas D'Silva" < > > tdsi...@salesforce.com > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > We have traditionally followed HBase's java support (see > > > > > > > > > https://hbase.apache.org/book.html#basic.prerequisites). > The > > > > > > > > > phoenix-queryserver module has a dependency on Avatica > which > > > does > > > > > not > > > > > > > > > support Java 1.7. The phoenix-spark module depends on spark > > > 2.3.2 > > > > > > which > > > > > > > > > also does not support Java 1.7. Do folks feel we should > > > continue > > > > to > > > > > > > > provide > > > > > > > > > support Java 1.7 on the 1.x branches? > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
