> If we really want, I bet we could get rid of most of the code we have for PQS already and just push it into Avatica itself. Just a thought ;)
+1. Even willing to volunteer myself here. On Tue, Dec 4, 2018 at 5:18 PM Josh Elser <els...@apache.org> wrote: > That makes the most sense to me. > > If we really want, I bet we could get rid of most of the code we have > for PQS already and just push it into Avatica itself. Just a thought ;) > > There are some other users of Avatica, but we are, far and away, the > most prevalent. > > On 12/4/18 8:10 PM, Thomas D'Silva wrote: > > Should we have one repo for the connectors (phoenix-flume, phoenix-hive, > > phoenix-kafka, phoenix-pig and phoenix-spark) > > and a separate repo for the queryserver? > > > > On Tue, Dec 4, 2018 at 3:58 PM Vincent Poon <vincentp...@apache.org> > wrote: > > > >> +1 to another repo for connectors > >> > >> On Mon, Dec 3, 2018 at 6:27 PM James Taylor <jamestay...@apache.org> > >> wrote: > >> > >>> +1. Good idea, Thomas. > >>> > >>> On Mon, Dec 3, 2018 at 2:57 PM Thomas D'Silva <tdsi...@salesforce.com> > >>> wrote: > >>> > >>>> I believe we will be maintaining the 4.x branches which support HBase > >>> 1.2, > >>>> 1.3 and 1.4 for a while. > >>>> Should we think about pulling out the connectors and queryserver into > >>> their > >>>> own repo similar to > >>>> what HBase did (see https://issues.apache.org/jira/browse/HBASE-20934 > >> ). > >>>> They could then have > >>>> their own release schedule and Java support. > >>>> > >>>> > >>>> On Sat, Dec 1, 2018 at 6:53 AM Pedro Boado <pedro.bo...@gmail.com> > >>> wrote: > >>>> > >>>>> Well I don't count with a lot more 4.x releases - maybe I'm > >>> wrong-headed > >>>> . > >>>>> For master branch and cdh6 we'd be looking at spark 2.x > >>>>> > >>>>> Part of the success of a project is about version stability. Not a > >> lot > >>> of > >>>>> corporate projects can afford keep upgrading to the latest versions - > >>>> think > >>>>> about it, you're in production, with a few thousand lines code > >> running > >>>>> spark 1.6 ... And for upgrading to 4.14 you need to review all of > >> this > >>>>> spark code -and maybe recompile to scala 2.11 btw- . It doesn't make > >>>> sense. > >>>>> Until now 4.x was pretty stable and in my opinion it should've never > >>> been > >>>>> migrated to spark 2 and java 8. Minor versions should keep certain > >>>>> stability in terms of dependencies. > >>>>> > >>>>> All these changes should've come with phoenix 5. But you're right it > >>>> needs > >>>>> a sensible solution as 4.14.1 is already out and compiled with java8. > >>>>> > >>>>> > >>>>> > >>>>> On Fri, 30 Nov 2018, 23:28 Thomas D'Silva <tdsi...@salesforce.com > >>> wrote: > >>>>> > >>>>>> Spark 1.6 is really old and doesn't support the newer Datasource v2 > >>> api > >>>>>> that we have been looking at integrating with. > >>>>>> As Alex points out you will might end up having to revert a lot > >> more > >>>>>> commits in the future. > >>>>>> Seems like the queryserver and phoenix-spark modules on the cdh > >>> branch > >>>>>> would end up diverging a lot from the standard open source branch. > >>>>>> > >>>>>> > >>>>>> On Fri, Nov 30, 2018 at 2:23 PM Alex Araujo <alexara...@gmail.com> > >>>>> wrote: > >>>>>> > >>>>>>>> Only a downgrade to spark 1.6 ( > >>>>>>> changes are only needed in a few IT, basically going back from > >>>> Datasets > >>>>>> to > >>>>>>> Dataframes) and going back to Avatica 1.10 ( involving reverting > >>>>>>> PHOENIX-4755, PHOENIX-4750 and PHOENIX-4805 ). > >>>>>>> > >>>>>>> We're talking about the 4.x branches, right? Doesn't seem prudent > >>> to > >>>> do > >>>>>> it > >>>>>>> there as down-streamers may already be relying on the newer > >>> versions. > >>>>>>> > >>>>>>> On Fri, Nov 30, 2018 at 4:18 PM Pedro Boado < > >> pedro.bo...@gmail.com > >>>> > >>>>>> wrote: > >>>>>>> > >>>>>>>> Thinking about typical server installation in a corporate > >>>> environment > >>>>>> I'd > >>>>>>>> keep everything compatible with the same JVM version. > >>>>>>>> > >>>>>>>> I've gone down the route for the cdh branch. Full JDK 7 > >>>> compatibility > >>>>>>>> doesn't require changes in phoenix-core. Only a downgrade to > >>> spark > >>>>> 1.6 > >>>>>> ( > >>>>>>>> changes are only needed in a few IT, basically going back from > >>>>> Datasets > >>>>>>> to > >>>>>>>> Dataframes) and going back to Avatica 1.10 ( involving > >> reverting > >>>>>>>> PHOENIX-4755, PHOENIX-4750 and PHOENIX-4805 ). > >>>>>>>> > >>>>>>>> > >>>>>>>> On Fri, 30 Nov 2018, 18:57 Thomas D'Silva < > >> tsi...@salesforce.com > >>>>>>>> <tdsi...@salesforce.com> wrote: > >>>>>>>> > >>>>>>>>> We could allow individual submodules like the queryserver, or > >>>>>>>> phoenix-spark > >>>>>>>>> to be built with their own compiler configuration (1.8+). > >>>>>>>>> This would allow these modules to use Java 1.8 features. I > >>> think > >>>>> this > >>>>>>>> would > >>>>>>>>> be a good compromise given that they depend on > >>>>>>>>> features that are provided by versions of spark and avatica > >>> that > >>>> no > >>>>>>>> longer > >>>>>>>>> support Java 1.7. > >>>>>>>>> We can still ensure phoenix-core supports Java 1.7. You would > >>>> have > >>>>> to > >>>>>>>> skip > >>>>>>>>> building modules that require Java 1.8, WDYT? > >>>>>>>>> > >>>>>>>>> On Thu, Nov 29, 2018 at 6:11 PM Jaanai Zhang < > >>>>> cloud.pos...@gmail.com > >>>>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> I'd vote for keep using java7 on 4.x branches. if upgrades > >> to > >>>>>> java8, > >>>>>>> it > >>>>>>>>>> will impact users who want to upgrade the latest 4.x > >>> branches. > >>>>> they > >>>>>>>> must > >>>>>>>>>> consider using java8 in their running environments, maybe > >>>> their > >>>>>>>>> libraries > >>>>>>>>>> do not support java8, then they have to give up to upgrade. > >>> So > >>>> I > >>>>>>> think > >>>>>>>>> that > >>>>>>>>>> drops support java7 is not friendly for some users. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> ---------------------------------------- > >>>>>>>>>> Jaanai Zhang > >>>>>>>>>> Best regards! > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Pedro Boado <pedro.bo...@gmail.com> 于2018年11月30日周五 > >> 上午6:13写道： > >>>>>>>>>> > >>>>>>>>>>> I'd vote for keep compiling 4.x branches in java7. It > >> makes > >>>>> sense > >>>>>>> as > >>>>>>>>> it's > >>>>>>>>>>> just a new minor release. > >>>>>>>>>>> > >>>>>>>>>>> It's pretty easy reverting back to spark 1.6 and also > >>> avatica > >>>>>>>>> dependency > >>>>>>>>>>> could be reverted to the previous version. > >>>>>>>>>>> > >>>>>>>>>>> On 29 Nov 2018 21:41, "Thomas D'Silva" < > >>>> tdsi...@salesforce.com > >>>>>> > >>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> We have traditionally followed HBase's java support (see > >>>>>>>>>>> https://hbase.apache.org/book.html#basic.prerequisites). > >>> The > >>>>>>>>>>> phoenix-queryserver module has a dependency on Avatica > >>> which > >>>>> does > >>>>>>> not > >>>>>>>>>>> support Java 1.7. The phoenix-spark module depends on > >> spark > >>>>> 2.3.2 > >>>>>>>> which > >>>>>>>>>>> also does not support Java 1.7. Do folks feel we should > >>>>> continue > >>>>>> to > >>>>>>>>>> provide > >>>>>>>>>>> support Java 1.7 on the 1.x branches? > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>>> On Fri, 30 Nov 2018, 18:57 Thomas D'Silva < > >>> tdsi...@salesforce.com > >>>>>> wrote: > >>>>>>>> > >>>>>>>>> We could allow individual submodules like the queryserver, or > >>>>>>>> phoenix-spark > >>>>>>>>> to be built with their own compiler configuration (1.8+). > >>>>>>>>> This would allow these modules to use Java 1.8 features. I > >>> think > >>>>> this > >>>>>>>> would > >>>>>>>>> be a good compromise given that they depend on > >>>>>>>>> features that are provided by versions of spark and avatica > >>> that > >>>> no > >>>>>>>> longer > >>>>>>>>> support Java 1.7. > >>>>>>>>> We can still ensure phoenix-core supports Java 1.7. You would > >>>> have > >>>>> to > >>>>>>>> skip > >>>>>>>>> building modules that require Java 1.8, WDYT? > >>>>>>>>> > >>>>>>>>> On Thu, Nov 29, 2018 at 6:11 PM Jaanai Zhang < > >>>>> cloud.pos...@gmail.com > >>>>>>> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> I'd vote for keep using java7 on 4.x branches. if upgrades > >> to > >>>>>> java8, > >>>>>>> it > >>>>>>>>>> will impact users who want to upgrade the latest 4.x > >>> branches. > >>>>> they > >>>>>>>> must > >>>>>>>>>> consider using java8 in their running environments, maybe > >>>> their > >>>>>>>>> libraries > >>>>>>>>>> do not support java8, then they have to give up to upgrade. > >>> So > >>>> I > >>>>>>> think > >>>>>>>>> that > >>>>>>>>>> drops support java7 is not friendly for some users. > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> ---------------------------------------- > >>>>>>>>>> Jaanai Zhang > >>>>>>>>>> Best regards! > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> Pedro Boado <pedro.bo...@gmail.com> 于2018年11月30日周五 > >> 上午6:13写道： > >>>>>>>>>> > >>>>>>>>>>> I'd vote for keep compiling 4.x branches in java7. It > >> makes > >>>>> sense > >>>>>>> as > >>>>>>>>> it's > >>>>>>>>>>> just a new minor release. > >>>>>>>>>>> > >>>>>>>>>>> It's pretty easy reverting back to spark 1.6 and also > >>> avatica > >>>>>>>>> dependency > >>>>>>>>>>> could be reverted to the previous version. > >>>>>>>>>>> > >>>>>>>>>>> On 29 Nov 2018 21:41, "Thomas D'Silva" < > >>>> tdsi...@salesforce.com > >>>>>> > >>>>>>>> wrote: > >>>>>>>>>>> > >>>>>>>>>>> We have traditionally followed HBase's java support (see > >>>>>>>>>>> https://hbase.apache.org/book.html#basic.prerequisites). > >>> The > >>>>>>>>>>> phoenix-queryserver module has a dependency on Avatica > >>> which > >>>>> does > >>>>>>> not > >>>>>>>>>>> support Java 1.7. The phoenix-spark module depends on > >> spark > >>>>> 2.3.2 > >>>>>>>> which > >>>>>>>>>>> also does not support Java 1.7. Do folks feel we should > >>>>> continue > >>>>>> to > >>>>>>>>>> provide > >>>>>>>>>>> support Java 1.7 on the 1.x branches? > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> > >>>>>>> > >>>>>> > >>>>> > >>>> > >>> > >> > > >
