That makes the most sense to me.
If we really want, I bet we could get rid of most of the code we have
for PQS already and just push it into Avatica itself. Just a thought ;)
There are some other users of Avatica, but we are, far and away, the
most prevalent.
On 12/4/18 8:10 PM, Thomas D'Silva wrote:
Should we have one repo for the connectors (phoenix-flume, phoenix-hive,
phoenix-kafka, phoenix-pig and phoenix-spark)
and a separate repo for the queryserver?
On Tue, Dec 4, 2018 at 3:58 PM Vincent Poon <vincentp...@apache.org> wrote:
+1 to another repo for connectors
On Mon, Dec 3, 2018 at 6:27 PM James Taylor <jamestay...@apache.org>
wrote:
+1. Good idea, Thomas.
On Mon, Dec 3, 2018 at 2:57 PM Thomas D'Silva <tdsi...@salesforce.com>
wrote:
I believe we will be maintaining the 4.x branches which support HBase
1.2,
1.3 and 1.4 for a while.
Should we think about pulling out the connectors and queryserver into
their
own repo similar to
what HBase did (see https://issues.apache.org/jira/browse/HBASE-20934
).
They could then have
their own release schedule and Java support.
On Sat, Dec 1, 2018 at 6:53 AM Pedro Boado <pedro.bo...@gmail.com>
wrote:
Well I don't count with a lot more 4.x releases - maybe I'm
wrong-headed
.
For master branch and cdh6 we'd be looking at spark 2.x
Part of the success of a project is about version stability. Not a
lot
of
corporate projects can afford keep upgrading to the latest versions -
think
about it, you're in production, with a few thousand lines code
running
spark 1.6 ... And for upgrading to 4.14 you need to review all of
this
spark code -and maybe recompile to scala 2.11 btw- . It doesn't make
sense.
Until now 4.x was pretty stable and in my opinion it should've never
been
migrated to spark 2 and java 8. Minor versions should keep certain
stability in terms of dependencies.
All these changes should've come with phoenix 5. But you're right it
needs
a sensible solution as 4.14.1 is already out and compiled with java8.
On Fri, 30 Nov 2018, 23:28 Thomas D'Silva <tdsi...@salesforce.com
wrote:
Spark 1.6 is really old and doesn't support the newer Datasource v2
api
that we have been looking at integrating with.
As Alex points out you will might end up having to revert a lot
more
commits in the future.
Seems like the queryserver and phoenix-spark modules on the cdh
branch
would end up diverging a lot from the standard open source branch.
On Fri, Nov 30, 2018 at 2:23 PM Alex Araujo <alexara...@gmail.com>
wrote:
Only a downgrade to spark 1.6 (
changes are only needed in a few IT, basically going back from
Datasets
to
Dataframes) and going back to Avatica 1.10 ( involving reverting
PHOENIX-4755, PHOENIX-4750 and PHOENIX-4805 ).
We're talking about the 4.x branches, right? Doesn't seem prudent
to
do
it
there as down-streamers may already be relying on the newer
versions.
On Fri, Nov 30, 2018 at 4:18 PM Pedro Boado <
pedro.bo...@gmail.com
wrote:
Thinking about typical server installation in a corporate
environment
I'd
keep everything compatible with the same JVM version.
I've gone down the route for the cdh branch. Full JDK 7
compatibility
doesn't require changes in phoenix-core. Only a downgrade to
spark
1.6
(
changes are only needed in a few IT, basically going back from
Datasets
to
Dataframes) and going back to Avatica 1.10 ( involving
reverting
PHOENIX-4755, PHOENIX-4750 and PHOENIX-4805 ).
On Fri, 30 Nov 2018, 18:57 Thomas D'Silva <
tsi...@salesforce.com
<tdsi...@salesforce.com> wrote:
We could allow individual submodules like the queryserver, or
phoenix-spark
to be built with their own compiler configuration (1.8+).
This would allow these modules to use Java 1.8 features. I
think
this
would
be a good compromise given that they depend on
features that are provided by versions of spark and avatica
that
no
longer
support Java 1.7.
We can still ensure phoenix-core supports Java 1.7. You would
have
to
skip
building modules that require Java 1.8, WDYT?
On Thu, Nov 29, 2018 at 6:11 PM Jaanai Zhang <
cloud.pos...@gmail.com
wrote:
I'd vote for keep using java7 on 4.x branches. if upgrades
to
java8,
it
will impact users who want to upgrade the latest 4.x
branches.
they
must
consider using java8 in their running environments, maybe
their
libraries
do not support java8, then they have to give up to upgrade.
So
I
think
that
drops support java7 is not friendly for some users.
----------------------------------------
Jaanai Zhang
Best regards!
Pedro Boado <pedro.bo...@gmail.com> 于2018年11月30日周五
上午6:13写道：
I'd vote for keep compiling 4.x branches in java7. It
makes
sense
as
it's
just a new minor release.
It's pretty easy reverting back to spark 1.6 and also
avatica
dependency
could be reverted to the previous version.
On 29 Nov 2018 21:41, "Thomas D'Silva" <
tdsi...@salesforce.com
wrote:
We have traditionally followed HBase's java support (see
https://hbase.apache.org/book.html#basic.prerequisites).
The
phoenix-queryserver module has a dependency on Avatica
which
does
not
support Java 1.7. The phoenix-spark module depends on
spark
2.3.2
which
also does not support Java 1.7. Do folks feel we should
continue
to
provide
support Java 1.7 on the 1.x branches?
On Fri, 30 Nov 2018, 18:57 Thomas D'Silva <
tdsi...@salesforce.com
wrote:
We could allow individual submodules like the queryserver, or
phoenix-spark
to be built with their own compiler configuration (1.8+).
This would allow these modules to use Java 1.8 features. I
think
this
would
be a good compromise given that they depend on
features that are provided by versions of spark and avatica
that
no
longer
support Java 1.7.
We can still ensure phoenix-core supports Java 1.7. You would
have
to
skip
building modules that require Java 1.8, WDYT?
On Thu, Nov 29, 2018 at 6:11 PM Jaanai Zhang <
cloud.pos...@gmail.com
wrote:
I'd vote for keep using java7 on 4.x branches. if upgrades
to
java8,
it
will impact users who want to upgrade the latest 4.x
branches.
they
must
consider using java8 in their running environments, maybe
their
libraries
do not support java8, then they have to give up to upgrade.
So
I
think
that
drops support java7 is not friendly for some users.
----------------------------------------
Jaanai Zhang
Best regards!
Pedro Boado <pedro.bo...@gmail.com> 于2018年11月30日周五
上午6:13写道：
I'd vote for keep compiling 4.x branches in java7. It
makes
sense
as
it's
just a new minor release.
It's pretty easy reverting back to spark 1.6 and also
avatica
dependency
could be reverted to the previous version.
On 29 Nov 2018 21:41, "Thomas D'Silva" <
tdsi...@salesforce.com
wrote:
We have traditionally followed HBase's java support (see
https://hbase.apache.org/book.html#basic.prerequisites).
The
phoenix-queryserver module has a dependency on Avatica
which
does
not
support Java 1.7. The phoenix-spark module depends on
spark
2.3.2
which
also does not support Java 1.7. Do folks feel we should
continue
to
provide
support Java 1.7 on the 1.x branches?
