Re: [DISCUSS] Migrate precommit git repos from kgyrtkirk to apache
Thank you, Stamatis! Also, Zoltan for the "donation" :) -Attila On Wed, Aug 23, 2023 at 4:53 PM Ayush Saxena wrote: > +1, > Thanx Stamatis foe initiating this. This was something which was in my > mind as well since long but couldn’t find time. > > -Ayush > > > On 23-Aug-2023, at 6:19 PM, Zoltan Haindrich wrote: > > > > Hey Stamatis! > > > > I'm happy to donate these repos / help with the migration! > > I should have done it earlier - but it was never top priority...thank > you for initiating it! > > > > cheers, > > Zoltan > > > >> On 8/23/23 14:00, Stamatis Zampetakis wrote: > >> Hi all, > >> Our precommit infrastructure uses code that resides in the following > repos. > >> * https://github.com/kgyrtkirk/hive-test-kube > >> * https://github.com/kgyrtkirk/hive-toolbox > >> * https://github.com/kgyrtkirk/hive-dev-box > >> These are mainly maintained by Zoltán Haindrich who is always helpful > >> and kind to investigate and resolve issues. > >> For facilitating contributions from the apache community and also > >> removing some burden from Zoltan's shoulders it may be a good time to > >> migrate those and put them under the apache namespace. > >> For the initial migration, we could have a straightforward 1 to 1 > >> mapping as shown below: > >> * https://github.com/apache/hive-test-kube > >> * https://github.com/apache/hive-toolbox > >> * https://github.com/apache/hive-dev-box > >> How do you feel about this? > >> Best, > >> Stamatis >
Re: [DISCUSS] Migrate precommit git repos from kgyrtkirk to apache
+1, Thanx Stamatis foe initiating this. This was something which was in my mind as well since long but couldn’t find time. -Ayush > On 23-Aug-2023, at 6:19 PM, Zoltan Haindrich wrote: > > Hey Stamatis! > > I'm happy to donate these repos / help with the migration! > I should have done it earlier - but it was never top priority...thank you for > initiating it! > > cheers, > Zoltan > >> On 8/23/23 14:00, Stamatis Zampetakis wrote: >> Hi all, >> Our precommit infrastructure uses code that resides in the following repos. >> * https://github.com/kgyrtkirk/hive-test-kube >> * https://github.com/kgyrtkirk/hive-toolbox >> * https://github.com/kgyrtkirk/hive-dev-box >> These are mainly maintained by Zoltán Haindrich who is always helpful >> and kind to investigate and resolve issues. >> For facilitating contributions from the apache community and also >> removing some burden from Zoltan's shoulders it may be a good time to >> migrate those and put them under the apache namespace. >> For the initial migration, we could have a straightforward 1 to 1 >> mapping as shown below: >> * https://github.com/apache/hive-test-kube >> * https://github.com/apache/hive-toolbox >> * https://github.com/apache/hive-dev-box >> How do you feel about this? >> Best, >> Stamatis
Re: [DISCUSS] Migrate precommit git repos from kgyrtkirk to apache
Hey Stamatis! I'm happy to donate these repos / help with the migration! I should have done it earlier - but it was never top priority...thank you for initiating it! cheers, Zoltan On 8/23/23 14:00, Stamatis Zampetakis wrote: Hi all, Our precommit infrastructure uses code that resides in the following repos. * https://github.com/kgyrtkirk/hive-test-kube * https://github.com/kgyrtkirk/hive-toolbox * https://github.com/kgyrtkirk/hive-dev-box These are mainly maintained by Zoltán Haindrich who is always helpful and kind to investigate and resolve issues. For facilitating contributions from the apache community and also removing some burden from Zoltan's shoulders it may be a good time to migrate those and put them under the apache namespace. For the initial migration, we could have a straightforward 1 to 1 mapping as shown below: * https://github.com/apache/hive-test-kube * https://github.com/apache/hive-toolbox * https://github.com/apache/hive-dev-box How do you feel about this? Best, Stamatis OpenPGP_signature Description: OpenPGP digital signature
[DISCUSS] Migrate precommit git repos from kgyrtkirk to apache
Hi all, Our precommit infrastructure uses code that resides in the following repos. * https://github.com/kgyrtkirk/hive-test-kube * https://github.com/kgyrtkirk/hive-toolbox * https://github.com/kgyrtkirk/hive-dev-box These are mainly maintained by Zoltán Haindrich who is always helpful and kind to investigate and resolve issues. For facilitating contributions from the apache community and also removing some burden from Zoltan's shoulders it may be a good time to migrate those and put them under the apache namespace. For the initial migration, we could have a straightforward 1 to 1 mapping as shown below: * https://github.com/apache/hive-test-kube * https://github.com/apache/hive-toolbox * https://github.com/apache/hive-dev-box How do you feel about this? Best, Stamatis
Re: MiniHS2 and postgresql jars
Hi,
if I understand correctly the descriptions this would be no problem for the
followings ?
- Postgres
- MSSQL
- Derby
"MariaDB, MySQL, and Oracle licenses are not compatible with Apache License
2 "
I created a ticket to solve this issue:
https://issues.apache.org/jira/browse/HIVE-27642
If we can agree on to restore for Postgres then I will create a PR.
Regards,
Zoltán Rátkai
On Wed, Aug 23, 2023 at 11:23 AM Stamatis Zampetakis
wrote:
> Hello,
>
> For those interested about the full background behind the changes that
> partially broke StartMiniHS2Cluster please have a look at [1, 2, 3].
> If something is not clear or we need to revisit those decisions I am
> happy to discuss further.
>
> Summarizing in one sentence, the bin.tar.gz and src.tag.gz release
> artifacts should never contain jars or code which have licenses in
> Category-X [4]. If we want to use or rely on such code things are a
> bit more flexible as described in [4].
>
> Best,
> Stamatis
>
> [1] https://lists.apache.org/thread/xd25nhox103t2zj52lnzbjkm6d41ls94
> [2] https://issues.apache.org/jira/browse/HIVE-25701
> [3] https://issues.apache.org/jira/browse/HIVE-27338
> [4] https://www.apache.org/legal/resolved.html#category-x
>
> On Wed, Aug 23, 2023 at 11:44 AM Zsolt Miskolczi
> wrote:
> >
> > As I understand, they have been removed due to licensing issues. Before
> we
> > re-add the jar or add a possibility to include jars into the classpath, I
> > want to understand the reason. What was the exact licensing issue?
> >
> > Do we have to make sure that we don't provide those jars in production?
> Or
> > is it necessary for tests as well?
> >
> > For me, the solution would be something that I, as a developer, have to
> set
> > up one time only. Based on my current knowledge, it seems the ability to
> > add external jars to the classpath could be more comfortable if we cannot
> > provide the jars into the production.
> > The reason is that we need the jdbc driver at two places:
> > - for MiniHS2
> > - and in the conf directory, next to bin (to be able to easily use it for
> > schema tool, beeline and standalone metastore service).
> >
> >
> > On the other hand, I'm not completely sure if MiniHS2 as a test is the
> most
> > comfortable idea. In the long term, I would prefer to replace it with a
> run
> > script/configuration that runs Hive Server in a 'Mini' mode, for local
> > development purposes.
> >
> > Thanks,
> > Zsolt
> >
> >
> > Stamatis Zampetakis ezt írta (időpont: 2023. aug.
> 22.,
> > K, 18:06):
> >
> > > I am not against restoring itest.jdbc.jars property but for this case
> > > I prefer the explicit declaration of the dependency.
> > >
> > > Adding an optional or test scope dependency is much simpler and works
> > > out of the box. We don't need to download jars manually and we don't
> > > need to remember how the system property is called in order to run the
> > > test.
> > >
> > > Anyways, we all agree that we want StartMiniHS2Cluster to run on
> > > different metastore backends so we can create a JIRA/PR and move this
> > > forward. How we are going to do it is implementation details so we can
> > > continue the discussion under the respective ticket.
> > >
> > > Best,
> > > Stamatis
> > >
> > > On Tue, Aug 22, 2023 at 6:46 PM László Bodor <
> bodorlaszlo0...@gmail.com>
> > > wrote:
> > > >
> > > > Yeah, I think we should simply re-add the possibility to add jars to
> > > > classpath, call it "itest.jdbc.jars" to preserve the old behavior and
> > > > parameter name.
> > > >
> > > > Denys Kuzmenko ezt írta (időpont: 2023. aug.
> > > 22., K,
> > > > 12:43):
> > > >
> > > > > Instead of adding the dependencies, can't we add the possibility to
> > > > > include jdbc jars in the classpath?
> > > > > something like this:
> > > > >
> > > > > org.apache.maven.plugins
> > > > > maven-failsafe-plugin
> > > > >
> > > > >
> > > > >
> > > > > integration-test
> > > > > verify
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > >
> ${itest.jdbc.jars}
> > > > >
> > > > >...
> > > > >
> > > > >
> > > > >
> > >
>
Re: MiniHS2 and postgresql jars
Hello,
For those interested about the full background behind the changes that
partially broke StartMiniHS2Cluster please have a look at [1, 2, 3].
If something is not clear or we need to revisit those decisions I am
happy to discuss further.
Summarizing in one sentence, the bin.tar.gz and src.tag.gz release
artifacts should never contain jars or code which have licenses in
Category-X [4]. If we want to use or rely on such code things are a
bit more flexible as described in [4].
Best,
Stamatis
[1] https://lists.apache.org/thread/xd25nhox103t2zj52lnzbjkm6d41ls94
[2] https://issues.apache.org/jira/browse/HIVE-25701
[3] https://issues.apache.org/jira/browse/HIVE-27338
[4] https://www.apache.org/legal/resolved.html#category-x
On Wed, Aug 23, 2023 at 11:44 AM Zsolt Miskolczi
wrote:
>
> As I understand, they have been removed due to licensing issues. Before we
> re-add the jar or add a possibility to include jars into the classpath, I
> want to understand the reason. What was the exact licensing issue?
>
> Do we have to make sure that we don't provide those jars in production? Or
> is it necessary for tests as well?
>
> For me, the solution would be something that I, as a developer, have to set
> up one time only. Based on my current knowledge, it seems the ability to
> add external jars to the classpath could be more comfortable if we cannot
> provide the jars into the production.
> The reason is that we need the jdbc driver at two places:
> - for MiniHS2
> - and in the conf directory, next to bin (to be able to easily use it for
> schema tool, beeline and standalone metastore service).
>
>
> On the other hand, I'm not completely sure if MiniHS2 as a test is the most
> comfortable idea. In the long term, I would prefer to replace it with a run
> script/configuration that runs Hive Server in a 'Mini' mode, for local
> development purposes.
>
> Thanks,
> Zsolt
>
>
> Stamatis Zampetakis ezt írta (időpont: 2023. aug. 22.,
> K, 18:06):
>
> > I am not against restoring itest.jdbc.jars property but for this case
> > I prefer the explicit declaration of the dependency.
> >
> > Adding an optional or test scope dependency is much simpler and works
> > out of the box. We don't need to download jars manually and we don't
> > need to remember how the system property is called in order to run the
> > test.
> >
> > Anyways, we all agree that we want StartMiniHS2Cluster to run on
> > different metastore backends so we can create a JIRA/PR and move this
> > forward. How we are going to do it is implementation details so we can
> > continue the discussion under the respective ticket.
> >
> > Best,
> > Stamatis
> >
> > On Tue, Aug 22, 2023 at 6:46 PM László Bodor
> > wrote:
> > >
> > > Yeah, I think we should simply re-add the possibility to add jars to
> > > classpath, call it "itest.jdbc.jars" to preserve the old behavior and
> > > parameter name.
> > >
> > > Denys Kuzmenko ezt írta (időpont: 2023. aug.
> > 22., K,
> > > 12:43):
> > >
> > > > Instead of adding the dependencies, can't we add the possibility to
> > > > include jdbc jars in the classpath?
> > > > something like this:
> > > >
> > > > org.apache.maven.plugins
> > > > maven-failsafe-plugin
> > > >
> > > >
> > > >
> > > > integration-test
> > > > verify
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > ${itest.jdbc.jars}
> > > >
> > > >...
> > > >
> > > >
> > > >
> >
Re: MiniHS2 and postgresql jars
As I understand, they have been removed due to licensing issues. Before we
re-add the jar or add a possibility to include jars into the classpath, I
want to understand the reason. What was the exact licensing issue?
Do we have to make sure that we don't provide those jars in production? Or
is it necessary for tests as well?
For me, the solution would be something that I, as a developer, have to set
up one time only. Based on my current knowledge, it seems the ability to
add external jars to the classpath could be more comfortable if we cannot
provide the jars into the production.
The reason is that we need the jdbc driver at two places:
- for MiniHS2
- and in the conf directory, next to bin (to be able to easily use it for
schema tool, beeline and standalone metastore service).
On the other hand, I'm not completely sure if MiniHS2 as a test is the most
comfortable idea. In the long term, I would prefer to replace it with a run
script/configuration that runs Hive Server in a 'Mini' mode, for local
development purposes.
Thanks,
Zsolt
Stamatis Zampetakis ezt írta (időpont: 2023. aug. 22.,
K, 18:06):
> I am not against restoring itest.jdbc.jars property but for this case
> I prefer the explicit declaration of the dependency.
>
> Adding an optional or test scope dependency is much simpler and works
> out of the box. We don't need to download jars manually and we don't
> need to remember how the system property is called in order to run the
> test.
>
> Anyways, we all agree that we want StartMiniHS2Cluster to run on
> different metastore backends so we can create a JIRA/PR and move this
> forward. How we are going to do it is implementation details so we can
> continue the discussion under the respective ticket.
>
> Best,
> Stamatis
>
> On Tue, Aug 22, 2023 at 6:46 PM László Bodor
> wrote:
> >
> > Yeah, I think we should simply re-add the possibility to add jars to
> > classpath, call it "itest.jdbc.jars" to preserve the old behavior and
> > parameter name.
> >
> > Denys Kuzmenko ezt írta (időpont: 2023. aug.
> 22., K,
> > 12:43):
> >
> > > Instead of adding the dependencies, can't we add the possibility to
> > > include jdbc jars in the classpath?
> > > something like this:
> > >
> > > org.apache.maven.plugins
> > > maven-failsafe-plugin
> > >
> > >
> > >
> > > integration-test
> > > verify
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> ${itest.jdbc.jars}
> > >
> > >...
> > >
> > >
> > >
>
