I think we have reached consensus about moving these modules into a separate repo. I filed PHOENIX-5062 and PHOENIX-5063.
On Tue, Dec 4, 2018 at 5:20 PM Alex Araujo <[email protected]> wrote: > > If we really want, I bet we could get rid of most of the code we have > for PQS already and just push it into Avatica itself. Just a thought ;) > > +1. Even willing to volunteer myself here. > > On Tue, Dec 4, 2018 at 5:18 PM Josh Elser <[email protected]> wrote: > > > That makes the most sense to me. > > > > If we really want, I bet we could get rid of most of the code we have > > for PQS already and just push it into Avatica itself. Just a thought ;) > > > > There are some other users of Avatica, but we are, far and away, the > > most prevalent. > > > > On 12/4/18 8:10 PM, Thomas D'Silva wrote: > > > Should we have one repo for the connectors (phoenix-flume, > phoenix-hive, > > > phoenix-kafka, phoenix-pig and phoenix-spark) > > > and a separate repo for the queryserver? > > > > > > On Tue, Dec 4, 2018 at 3:58 PM Vincent Poon <[email protected]> > > wrote: > > > > > >> +1 to another repo for connectors > > >> > > >> On Mon, Dec 3, 2018 at 6:27 PM James Taylor <[email protected]> > > >> wrote: > > >> > > >>> +1. Good idea, Thomas. > > >>> > > >>> On Mon, Dec 3, 2018 at 2:57 PM Thomas D'Silva < > [email protected]> > > >>> wrote: > > >>> > > >>>> I believe we will be maintaining the 4.x branches which support > HBase > > >>> 1.2, > > >>>> 1.3 and 1.4 for a while. > > >>>> Should we think about pulling out the connectors and queryserver > into > > >>> their > > >>>> own repo similar to > > >>>> what HBase did (see > https://issues.apache.org/jira/browse/HBASE-20934 > > >> ). > > >>>> They could then have > > >>>> their own release schedule and Java support. > > >>>> > > >>>> > > >>>> On Sat, Dec 1, 2018 at 6:53 AM Pedro Boado <[email protected]> > > >>> wrote: > > >>>> > > >>>>> Well I don't count with a lot more 4.x releases - maybe I'm > > >>> wrong-headed > > >>>> . > > >>>>> For master branch and cdh6 we'd be looking at spark 2.x > > >>>>> > > >>>>> Part of the success of a project is about version stability. Not a > > >> lot > > >>> of > > >>>>> corporate projects can afford keep upgrading to the latest > versions - > > >>>> think > > >>>>> about it, you're in production, with a few thousand lines code > > >> running > > >>>>> spark 1.6 ... And for upgrading to 4.14 you need to review all of > > >> this > > >>>>> spark code -and maybe recompile to scala 2.11 btw- . It doesn't > make > > >>>> sense. > > >>>>> Until now 4.x was pretty stable and in my opinion it should've > never > > >>> been > > >>>>> migrated to spark 2 and java 8. Minor versions should keep certain > > >>>>> stability in terms of dependencies. > > >>>>> > > >>>>> All these changes should've come with phoenix 5. But you're right > it > > >>>> needs > > >>>>> a sensible solution as 4.14.1 is already out and compiled with > java8. > > >>>>> > > >>>>> > > >>>>> > > >>>>> On Fri, 30 Nov 2018, 23:28 Thomas D'Silva <[email protected] > > >>> wrote: > > >>>>> > > >>>>>> Spark 1.6 is really old and doesn't support the newer Datasource > v2 > > >>> api > > >>>>>> that we have been looking at integrating with. > > >>>>>> As Alex points out you will might end up having to revert a lot > > >> more > > >>>>>> commits in the future. > > >>>>>> Seems like the queryserver and phoenix-spark modules on the cdh > > >>> branch > > >>>>>> would end up diverging a lot from the standard open source branch. > > >>>>>> > > >>>>>> > > >>>>>> On Fri, Nov 30, 2018 at 2:23 PM Alex Araujo <[email protected] > > > > >>>>> wrote: > > >>>>>> > > >>>>>>>> Only a downgrade to spark 1.6 ( > > >>>>>>> changes are only needed in a few IT, basically going back from > > >>>> Datasets > > >>>>>> to > > >>>>>>> Dataframes) and going back to Avatica 1.10 ( involving reverting > > >>>>>>> PHOENIX-4755, PHOENIX-4750 and PHOENIX-4805 ). > > >>>>>>> > > >>>>>>> We're talking about the 4.x branches, right? Doesn't seem prudent > > >>> to > > >>>> do > > >>>>>> it > > >>>>>>> there as down-streamers may already be relying on the newer > > >>> versions. > > >>>>>>> > > >>>>>>> On Fri, Nov 30, 2018 at 4:18 PM Pedro Boado < > > >> [email protected] > > >>>> > > >>>>>> wrote: > > >>>>>>> > > >>>>>>>> Thinking about typical server installation in a corporate > > >>>> environment > > >>>>>> I'd > > >>>>>>>> keep everything compatible with the same JVM version. > > >>>>>>>> > > >>>>>>>> I've gone down the route for the cdh branch. Full JDK 7 > > >>>> compatibility > > >>>>>>>> doesn't require changes in phoenix-core. Only a downgrade to > > >>> spark > > >>>>> 1.6 > > >>>>>> ( > > >>>>>>>> changes are only needed in a few IT, basically going back from > > >>>>> Datasets > > >>>>>>> to > > >>>>>>>> Dataframes) and going back to Avatica 1.10 ( involving > > >> reverting > > >>>>>>>> PHOENIX-4755, PHOENIX-4750 and PHOENIX-4805 ). > > >>>>>>>> > > >>>>>>>> > > >>>>>>>> On Fri, 30 Nov 2018, 18:57 Thomas D'Silva < > > >> [email protected] > > >>>>>>>> <[email protected]> wrote: > > >>>>>>>> > > >>>>>>>>> We could allow individual submodules like the queryserver, or > > >>>>>>>> phoenix-spark > > >>>>>>>>> to be built with their own compiler configuration (1.8+). > > >>>>>>>>> This would allow these modules to use Java 1.8 features. I > > >>> think > > >>>>> this > > >>>>>>>> would > > >>>>>>>>> be a good compromise given that they depend on > > >>>>>>>>> features that are provided by versions of spark and avatica > > >>> that > > >>>> no > > >>>>>>>> longer > > >>>>>>>>> support Java 1.7. > > >>>>>>>>> We can still ensure phoenix-core supports Java 1.7. You would > > >>>> have > > >>>>> to > > >>>>>>>> skip > > >>>>>>>>> building modules that require Java 1.8, WDYT? > > >>>>>>>>> > > >>>>>>>>> On Thu, Nov 29, 2018 at 6:11 PM Jaanai Zhang < > > >>>>> [email protected] > > >>>>>>> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> I'd vote for keep using java7 on 4.x branches. if upgrades > > >> to > > >>>>>> java8, > > >>>>>>> it > > >>>>>>>>>> will impact users who want to upgrade the latest 4.x > > >>> branches. > > >>>>> they > > >>>>>>>> must > > >>>>>>>>>> consider using java8 in their running environments, maybe > > >>>> their > > >>>>>>>>> libraries > > >>>>>>>>>> do not support java8, then they have to give up to upgrade. > > >>> So > > >>>> I > > >>>>>>> think > > >>>>>>>>> that > > >>>>>>>>>> drops support java7 is not friendly for some users. > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> ---------------------------------------- > > >>>>>>>>>> Jaanai Zhang > > >>>>>>>>>> Best regards! > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> Pedro Boado <[email protected]> 于2018年11月30日周五 > > >> 上午6:13写道： > > >>>>>>>>>> > > >>>>>>>>>>> I'd vote for keep compiling 4.x branches in java7. It > > >> makes > > >>>>> sense > > >>>>>>> as > > >>>>>>>>> it's > > >>>>>>>>>>> just a new minor release. > > >>>>>>>>>>> > > >>>>>>>>>>> It's pretty easy reverting back to spark 1.6 and also > > >>> avatica > > >>>>>>>>> dependency > > >>>>>>>>>>> could be reverted to the previous version. > > >>>>>>>>>>> > > >>>>>>>>>>> On 29 Nov 2018 21:41, "Thomas D'Silva" < > > >>>> [email protected] > > >>>>>> > > >>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>> We have traditionally followed HBase's java support (see > > >>>>>>>>>>> https://hbase.apache.org/book.html#basic.prerequisites). > > >>> The > > >>>>>>>>>>> phoenix-queryserver module has a dependency on Avatica > > >>> which > > >>>>> does > > >>>>>>> not > > >>>>>>>>>>> support Java 1.7. The phoenix-spark module depends on > > >> spark > > >>>>> 2.3.2 > > >>>>>>>> which > > >>>>>>>>>>> also does not support Java 1.7. Do folks feel we should > > >>>>> continue > > >>>>>> to > > >>>>>>>>>> provide > > >>>>>>>>>>> support Java 1.7 on the 1.x branches? > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>>> On Fri, 30 Nov 2018, 18:57 Thomas D'Silva < > > >>> [email protected] > > >>>>>> wrote: > > >>>>>>>> > > >>>>>>>>> We could allow individual submodules like the queryserver, or > > >>>>>>>> phoenix-spark > > >>>>>>>>> to be built with their own compiler configuration (1.8+). > > >>>>>>>>> This would allow these modules to use Java 1.8 features. I > > >>> think > > >>>>> this > > >>>>>>>> would > > >>>>>>>>> be a good compromise given that they depend on > > >>>>>>>>> features that are provided by versions of spark and avatica > > >>> that > > >>>> no > > >>>>>>>> longer > > >>>>>>>>> support Java 1.7. > > >>>>>>>>> We can still ensure phoenix-core supports Java 1.7. You would > > >>>> have > > >>>>> to > > >>>>>>>> skip > > >>>>>>>>> building modules that require Java 1.8, WDYT? > > >>>>>>>>> > > >>>>>>>>> On Thu, Nov 29, 2018 at 6:11 PM Jaanai Zhang < > > >>>>> [email protected] > > >>>>>>> > > >>>>>>>>> wrote: > > >>>>>>>>> > > >>>>>>>>>> I'd vote for keep using java7 on 4.x branches. if upgrades > > >> to > > >>>>>> java8, > > >>>>>>> it > > >>>>>>>>>> will impact users who want to upgrade the latest 4.x > > >>> branches. > > >>>>> they > > >>>>>>>> must > > >>>>>>>>>> consider using java8 in their running environments, maybe > > >>>> their > > >>>>>>>>> libraries > > >>>>>>>>>> do not support java8, then they have to give up to upgrade. > > >>> So > > >>>> I > > >>>>>>> think > > >>>>>>>>> that > > >>>>>>>>>> drops support java7 is not friendly for some users. > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> ---------------------------------------- > > >>>>>>>>>> Jaanai Zhang > > >>>>>>>>>> Best regards! > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>>> Pedro Boado <[email protected]> 于2018年11月30日周五 > > >> 上午6:13写道： > > >>>>>>>>>> > > >>>>>>>>>>> I'd vote for keep compiling 4.x branches in java7. It > > >> makes > > >>>>> sense > > >>>>>>> as > > >>>>>>>>> it's > > >>>>>>>>>>> just a new minor release. > > >>>>>>>>>>> > > >>>>>>>>>>> It's pretty easy reverting back to spark 1.6 and also > > >>> avatica > > >>>>>>>>> dependency > > >>>>>>>>>>> could be reverted to the previous version. > > >>>>>>>>>>> > > >>>>>>>>>>> On 29 Nov 2018 21:41, "Thomas D'Silva" < > > >>>> [email protected] > > >>>>>> > > >>>>>>>> wrote: > > >>>>>>>>>>> > > >>>>>>>>>>> We have traditionally followed HBase's java support (see > > >>>>>>>>>>> https://hbase.apache.org/book.html#basic.prerequisites). > > >>> The > > >>>>>>>>>>> phoenix-queryserver module has a dependency on Avatica > > >>> which > > >>>>> does > > >>>>>>> not > > >>>>>>>>>>> support Java 1.7. The phoenix-spark module depends on > > >> spark > > >>>>> 2.3.2 > > >>>>>>>> which > > >>>>>>>>>>> also does not support Java 1.7. Do folks feel we should > > >>>>> continue > > >>>>>> to > > >>>>>>>>>> provide > > >>>>>>>>>>> support Java 1.7 on the 1.x branches? > > >>>>>>>>>>> > > >>>>>>>>>> > > >>>>>>>>> > > >>>>>>>> > > >>>>>>> > > >>>>>> > > >>>>> > > >>>> > > >>> > > >> > > > > > >
