Questions around moving to Attic process

[Attila Szabó]

Hi there,

Just a couple of questions came into my mind after the vote is passed to
send Sqoop to the Attic.

Do we want to do a "final" release ( since 1.4.7 we do have ~ 110 commits
which never reached any Apache Sqoop official release, this never reached
the users over official channels)?

What should we ( I mean devs/commiters/etc. ) if a PR hits the codebase? Do
we still intended to merge it and guide the contributor, or notify them
about the project is technically "dead"?

What about the open JIRA tickets? Is the whole Sqoop project and tickets
gonna be wiped from the JIRA as it's not a toplevel project anymore, or
should we still handle them as existing tasks to be solved ( by anyone in
their free time for fun)?

Thanks for the clarification,
Attila

Re: Your project's website

[Attila Szabó]

Adding Andrew back to the thread

On Sun, Aug 23, 2020, 17:51 Attila Szabó  wrote:

> Hey Andrew ,
> Sorry for the delayed response!
>
> I've personally reached out to the PMC, and received permission to do the
> site migration, which is due to mid September latest
>
> I'll update you as we progress!
>
> If you have any questions please don't hesitate to contact me!
>
> Kind regards ,
> Attila
>
> On Sun, Aug 23, 2020, 16:14 Andrew Wetmore  wrote:
>
>> Hi:
>>
>> I wrote to you two weeks ago about your project website, but have not seen
>> a response. Whom should I contact directly? I am trying to see whether
>> your
>> project still uses the Apache CMS and, if so, what your plans are to
>> migrate off it.
>>
>> Thanks in advance for your help.
>>
>> Andrew
>>
>> On Fri, Aug 7, 2020 at 9:59 AM Andrew Wetmore  wrote:
>>
>> > Hi:
>> >
>> > I am part of the Infrastructure team, and am writing to ask whether your
>> > project is still using the Apache CMS for your project website. As you
>> > know, the CMS is reaching end-of-life, and we need projects to move
>> their
>> > websites onto a different option within the next few weeks.
>> >
>> > There are several alternatives available, including those listed on this
>> > page [1] on managing project websites. Infra is assembling a Wiki page
>> [2]
>> > on migrating a website from the CMS, and is looking forward to helping
>> > projects with this transition.
>> >
>> > Please let me know whether your site is still on the Apache CMS and, if
>> > so, who will be the project point-of-contact with Infra for the
>> migration.
>> >
>> > Thank you!
>> >
>> >
>> >
>> >
>> > [1] https://infra.apache.org/project-site.html
>> >
>> > [2]
>> >
>> https://cwiki.apache.org/confluence/display/INFRA/Migrate+your+project+website+from+the+Apache+CMS
>> >
>> >
>> > --
>> > Andrew Wetmore
>> > Technical Writer-Editor
>> > Infra
>> > *Apache Software Foundation*
>> > andr...@apache.org
>> >
>> >
>> > <
>> https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=webmail>
>> Virus-free.
>> > www.avast.com
>> > <
>> https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=webmail
>> >
>> > <#m_2909482246229832915_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>> >
>>
>>
>> --
>> Andrew Wetmore
>> Technical Writer-Editor
>> Infra
>> *Apache Software Foundation*
>> andr...@apache.org
>>
>

Re: Your project's website

[Attila Szabó]

Hey Andrew ,
Sorry for the delayed response!

I've personally reached out to the PMC, and received permission to do the
site migration, which is due to mid September latest

I'll update you as we progress!

If you have any questions please don't hesitate to contact me!

Kind regards ,
Attila

On Sun, Aug 23, 2020, 16:14 Andrew Wetmore  wrote:

> Hi:
>
> I wrote to you two weeks ago about your project website, but have not seen
> a response. Whom should I contact directly? I am trying to see whether your
> project still uses the Apache CMS and, if so, what your plans are to
> migrate off it.
>
> Thanks in advance for your help.
>
> Andrew
>
> On Fri, Aug 7, 2020 at 9:59 AM Andrew Wetmore  wrote:
>
> > Hi:
> >
> > I am part of the Infrastructure team, and am writing to ask whether your
> > project is still using the Apache CMS for your project website. As you
> > know, the CMS is reaching end-of-life, and we need projects to move their
> > websites onto a different option within the next few weeks.
> >
> > There are several alternatives available, including those listed on this
> > page [1] on managing project websites. Infra is assembling a Wiki page
> [2]
> > on migrating a website from the CMS, and is looking forward to helping
> > projects with this transition.
> >
> > Please let me know whether your site is still on the Apache CMS and, if
> > so, who will be the project point-of-contact with Infra for the
> migration.
> >
> > Thank you!
> >
> >
> >
> >
> > [1] https://infra.apache.org/project-site.html
> >
> > [2]
> >
> https://cwiki.apache.org/confluence/display/INFRA/Migrate+your+project+website+from+the+Apache+CMS
> >
> >
> > --
> > Andrew Wetmore
> > Technical Writer-Editor
> > Infra
> > *Apache Software Foundation*
> > andr...@apache.org
> >
> >
> > <
> https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=webmail>
> Virus-free.
> > www.avast.com
> > <
> https://www.avast.com/sig-email?utm_medium=email_source=link_campaign=sig-email_content=webmail
> >
> > <#m_2909482246229832915_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> >
>
>
> --
> Andrew Wetmore
> Technical Writer-Editor
> Infra
> *Apache Software Foundation*
> andr...@apache.org
>

Re: Mandatory relocation of Sqoop git repository to gitbox

[Attila Szabó]

Hello everyone,

+1

@Szabi:
Thanks for owning the consensus process!!!

Cheers,
Attila

On Mon, Dec 10, 2018, 3:50 PM Fero Szabo  Hi Szabi,
>
> +1 from my side to get this done in the initial phase!
>
> Best Regards,
> Fero
>
> On Mon, Dec 10, 2018 at 2:29 PM Szabolcs Vasas  wrote:
>
> > Hi All,
> >
> > According to this
> > <
> >
> http://mail-archives.apache.org/mod_mbox/incubator-general/201812.mbox/%3c6edbcae6-4eb9-6f61-beac-4198fd750...@apache.org%3E
> > >
> > email
> > the git-wip-us server, which hosts Apache Sqoop git repository too, is
> > going to be decommissioned soon and all the projects are going to be
> > migrated to gitbox.
> > For the detailed description of the planned change please refer to the
> > email linked, but the bottom line is that after the migration we are
> going
> > to be able to merge pull requests on the GitHub UI as well which will
> > greatly simplify our commit process.
> >
> > This relocation is mandatory however we have the option execute it in the
> > initial phase which would be great in my opinion because we could start
> > enjoying the benefits very soon.
> >
> > Please reply to this chain with your opinion because we need a consensus
> to
> > be able to start the migration in the initial phase.
> >
> > Thanks and regards,
> > Szabolcs
> >
>
>
> --
> *Ferenc Szabo* | Software Engineer
> t. (+361) 701 1201 <+361+701+1201>
> cloudera.com 
>
> [image: Cloudera] 
>
> [image: Cloudera on Twitter]  [image:
> Cloudera on Facebook]  [image: Cloudera
> on LinkedIn] 
> --
>

Re: Sqoop build infrastructure improvements

[Attila Szabó]

Hello everyone,

So if I'm not mistaken and according to the INFRA mail we've received
during the weekend gitbox.apache.org is exactly what we wanted to have :
Having both ASF commits and push privileges on Github side..
We would just need to have an official consensus in the dev community (
documented on the mail list), fire an INFRA Jira ticket, and after the
merge update our site with the new "how to contribute" information.

I think this would be a quite good timing for the Sqoop project as we're
right now in the middle of these infrastructural reworks.

Kind regards,
Attila

On Fri, Nov 30, 2018, 2:38 PM Szabolcs Vasas  Thanks for the question, I think we should stick to the commit format we
> had earlier so yes, we should go for squash and push.
> The easiest solution could be to download the diff file instead of the
> patch (e.g. https://github.com/apache/sqoop/pull/60.diff instead of
> https://github.com/apache/sqoop/pull/60.patch) and that does the trick.
>
> The "This closes..." commit message just closes the PR but does not delete
> the feature branch, asfgit most probably does not have delete permission
> for these branches anyway.
>
>
> On Fri, Nov 30, 2018 at 11:45 AM Attila Szabó  wrote:
>
> > Hey Szabi,
> >
> > Thanks for the prompt response!
> >
> > I've thought the repos are connected back and forth and the close PR is
> the
> > official way. In case of we still stack to the patch file and git apply
> > then commit and push approach.
> >
> > My only question in this case :
> > Do we have any agreement on how we commit these PRs. I would vote for
> > Squash and push, but of course I'm open for the discussion.
> >
> > BTW :
> > Is "This closes" message deletes the branch automatically?
> >
> > On front of Github + Jira:
> > I'm aware Github has the feature to connect with Jira so I'm pretty sure
> it
> > doable. Also I'm not sure if any ASF project has done it ever, but I'll
> ask
> > around in other communities.
> >
> > Cheers,
> > Attila
> >
> >
> >
> > On Nov 30, 2018 11:03 AM, "Szabolcs Vasas"  wrote:
> >
> > Hi Attila,
> >
> > I think we won't be able to commit the pull requests on GitHub directly
> > because our GitHub repo is just a mirror of the original Apache Sqoop
> repo
> > <https://git-wip-us.apache.org/repos/asf/sqoop.git>.
> > However the commit process is still simplified, the ASF GitHub Bot JIRA
> > comment contains the patch download link (e.g.
> > https://github.com/apache/sqoop/pull/60.patch) and the commit message
> > (e.g. This
> > closes #60) you need to include to close the pull request. The rest of
> the
> > process remains the same, you need to apply the patch manually and push
> the
> > commit to git-wip-us repo.
> >
> > Regarding the GitHub UI improvement: I am not sure GitHub provides such a
> > feature, do you know an example repository where this is implemented?
> >
> > Regards,
> > Szabolcs
> >
> >
> >
> > On Fri, Nov 30, 2018 at 3:21 AM Attila Szabó  wrote:
> >
> > > Hi Szabi,
> > >
> > > First of all:
> > > Big Kudos for the more mature gradle build! I think this is a great
> step
> > > for the whole project!
> > >
> > > On the front of PRs:
> > > I would only make it official if the user management / authorization
> > > handling could be somehow automatically bound to the id.apache.org +
> > > project privileges.
> > > A good example:
> > > Today I reviewed SQOOP-3396 but I would not had been able to merge it
> > > because it seems on the Github project I do not have push / merge
> > > privileges (regardless that I'm a Sqoop committer and also memeber of
> the
> > > ASF group on github with my user).
> > > So if we can somehow bound these things together, and the majority of
> the
> > > ppl would like to use it instead of the Review Board then let it
> happen!
> > > I'm fine with both tools until there's no difference between the Github
> > and
> > > ASF repos from user management POV.
> > >
> > > On the top of this:
> > > I'm not sure if it belongs to our table, or the ASF INFRA team, but it
> > > would be nice if the PRs and the JIRA tickets would be connected
> > > automatically on the Github UI as well, and thus the navigation to
> > > issues.apache.org would be easier!
> > >
> > > On the front of the gradle build:
> > > I've found a smaller issue with clean+unittest within the same command.

Re: Sqoop build infrastructure improvements

[Attila Szabó]

Hey Szabi,

Thanks for the prompt response!

I've thought the repos are connected back and forth and the close PR is the
official way. In case of we still stack to the patch file and git apply
then commit and push approach.

My only question in this case :
Do we have any agreement on how we commit these PRs. I would vote for
Squash and push, but of course I'm open for the discussion.

BTW :
Is "This closes" message deletes the branch automatically?

On front of Github + Jira:
I'm aware Github has the feature to connect with Jira so I'm pretty sure it
doable. Also I'm not sure if any ASF project has done it ever, but I'll ask
around in other communities.

Cheers,
Attila



On Nov 30, 2018 11:03 AM, "Szabolcs Vasas"  wrote:

Hi Attila,

I think we won't be able to commit the pull requests on GitHub directly
because our GitHub repo is just a mirror of the original Apache Sqoop repo
<https://git-wip-us.apache.org/repos/asf/sqoop.git>.
However the commit process is still simplified, the ASF GitHub Bot JIRA
comment contains the patch download link (e.g.
https://github.com/apache/sqoop/pull/60.patch) and the commit message
(e.g. This
closes #60) you need to include to close the pull request. The rest of the
process remains the same, you need to apply the patch manually and push the
commit to git-wip-us repo.

Regarding the GitHub UI improvement: I am not sure GitHub provides such a
feature, do you know an example repository where this is implemented?

Regards,
Szabolcs



On Fri, Nov 30, 2018 at 3:21 AM Attila Szabó  wrote:

> Hi Szabi,
>
> First of all:
> Big Kudos for the more mature gradle build! I think this is a great step
> for the whole project!
>
> On the front of PRs:
> I would only make it official if the user management / authorization
> handling could be somehow automatically bound to the id.apache.org +
> project privileges.
> A good example:
> Today I reviewed SQOOP-3396 but I would not had been able to merge it
> because it seems on the Github project I do not have push / merge
> privileges (regardless that I'm a Sqoop committer and also memeber of the
> ASF group on github with my user).
> So if we can somehow bound these things together, and the majority of the
> ppl would like to use it instead of the Review Board then let it happen!
> I'm fine with both tools until there's no difference between the Github
and
> ASF repos from user management POV.
>
> On the top of this:
> I'm not sure if it belongs to our table, or the ASF INFRA team, but it
> would be nice if the PRs and the JIRA tickets would be connected
> automatically on the Github UI as well, and thus the navigation to
> issues.apache.org would be easier!
>
> On the front of the gradle build:
> I've found a smaller issue with clean+unittest within the same command.
> I've opened a ticket (SQOOP-3415) and a PR (just the follow the new
> standard) with a solution proposal.
>
> My2cents,
> Attila
>
> On Wed, Nov 28, 2018 at 4:54 PM Szabolcs Vasas  >
> wrote:
>
> > Dear Sqoop community,
> >
> > We have been working on quite a few exciting build infrastructure
> > improvements recently, I am sending this email to summarize them.
> >
> > *Gradle can now execute all the Sqoop tests in a single JVM*
> > This improvement makes the Gradle test tasks significantly faster since
> we
> > do not have to start up a new JVM for every test class. It also made
> > possible to introduce fine grained test categories which were essential
> to
> > be able to parallelize the test execution in our CI systems. For more
> > information please refer to COMPILING.txt
> > <https://github.com/apache/sqoop/blob/trunk/COMPILING.txt>.
> >
> > *Apache Sqoop Jenkins job
> > <https://builds.apache.org/job/Sqoop-hadoop200/> now builds and tests
> with
> > Gradle*
> > Since our Gradle build became much more stable and faster it made sense
> to
> > reconfigure our Jenkins job to benefit from these improvements. The job
> is
> > faster now (~30 minutes instead of ~40) and it executes all of the tests
> > which can be run without external RDBMS or cloud systems (while the old
> Ant
> > based job executed the unit test suite only).
> >
> > *Travis CI is enabled for Apache Sqoop*
> > The new Travis CI job <https://travis-ci.org/apache/sqoop> now runs for
> > every commit and every pull request on Apache Sqoop GitHub repository
and
> > it executes all of the tests except the Oracle third party test cases.
> One
> > of the biggest benefit of Travis CI is that it can be really easily
> > configured for the individual forks as well so contributors get a well
> > configured CI job for their own feature branches for free. For more
> >

Re: Sqoop build infrastructure improvements

[Attila Szabó]

Hi Szabi,

First of all:
Big Kudos for the more mature gradle build! I think this is a great step
for the whole project!

On the front of PRs:
I would only make it official if the user management / authorization
handling could be somehow automatically bound to the id.apache.org +
project privileges.
A good example:
Today I reviewed SQOOP-3396 but I would not had been able to merge it
because it seems on the Github project I do not have push / merge
privileges (regardless that I'm a Sqoop committer and also memeber of the
ASF group on github with my user).
So if we can somehow bound these things together, and the majority of the
ppl would like to use it instead of the Review Board then let it happen!
I'm fine with both tools until there's no difference between the Github and
ASF repos from user management POV.

On the top of this:
I'm not sure if it belongs to our table, or the ASF INFRA team, but it
would be nice if the PRs and the JIRA tickets would be connected
automatically on the Github UI as well, and thus the navigation to
issues.apache.org would be easier!

On the front of the gradle build:
I've found a smaller issue with clean+unittest within the same command.
I've opened a ticket (SQOOP-3415) and a PR (just the follow the new
standard) with a solution proposal.

My2cents,
Attila

On Wed, Nov 28, 2018 at 4:54 PM Szabolcs Vasas 
wrote:

> Dear Sqoop community,
>
> We have been working on quite a few exciting build infrastructure
> improvements recently, I am sending this email to summarize them.
>
> *Gradle can now execute all the Sqoop tests in a single JVM*
> This improvement makes the Gradle test tasks significantly faster since we
> do not have to start up a new JVM for every test class. It also made
> possible to introduce fine grained test categories which were essential to
> be able to parallelize the test execution in our CI systems. For more
> information please refer to COMPILING.txt
> .
>
> *Apache Sqoop Jenkins job
>  now builds and tests with
> Gradle*
> Since our Gradle build became much more stable and faster it made sense to
> reconfigure our Jenkins job to benefit from these improvements. The job is
> faster now (~30 minutes instead of ~40) and it executes all of the tests
> which can be run without external RDBMS or cloud systems (while the old Ant
> based job executed the unit test suite only).
>
> *Travis CI is enabled for Apache Sqoop*
> The new Travis CI job  now runs for
> every commit and every pull request on Apache Sqoop GitHub repository and
> it executes all of the tests except the Oracle third party test cases. One
> of the biggest benefit of Travis CI is that it can be really easily
> configured for the individual forks as well so contributors get a well
> configured CI job for their own feature branches for free. For more
> information please refer to COMPILING.txt
> .
>
>
> Since we have a CI job now which integrates very well with GitHub pull
> requests I suggest deprecating the old Review Board and patch file based
> contribution process and use pull requests in the future. We had a mail
> chain about the same proposal last year and it seemed that the community
> was happy about the idea so I think we can evaluate it for some time and if
> everything goes well we can update our how to contribute wiki.
>
> Feel free to reply to this chain with your questions and suggestions on the
> above!
>
> Regards,
> Szabolcs
>

Re: Next release

[Attila Szabó]

Hi Ferenc,

I did not have any specific plan for the end of the year, but as I did the
very last release I would be happy to help anyone who would like to drive
it ( or if no one volunteers I might own it, but IMHO from community POV it
would be better if someone else would execute it this time).

On the scoping front :
AFAIR there are a few tickets which seems to be abandoned for a while and
targeted to this release. A grooming there would be a great start for the
scoping.

My2cents,
Attila

On Tue, Nov 27, 2018, 2:10 PM Ferenc Szabo  Hi sqoop developers,
>
> Do you have a plan for the next release?
>
> Regards
> Ferenc
>

Fwd: Release to support Hadoop 3

[Attila Szabó]

Dear Sqoop community,

Am I the only one who is missing some formal decision making, announcement
and process here?

 - When did the PMC made decision about that we're going for version 3.0
(instead of any other version alternatives)? When and where was it
announced? How is it possible that some of the contributors know this fact
earlier then the rests of the community?

 - When did Bogi become a PMC member, and why was that fact not announced
to the community as it used to be (and still not yet visible on the project
site)? Of course this is just an assumption, but according to some PMC
chair emails back this February: JIRA administrator privileges are only PMC
members, and I guess we still follow this rule, as no other information was
announced, thus I guess the reason why Bogi was able to administrate the
available versions in the JIRA means she has been lifted  (and here I'd
like to send my congrats, if this is true, and my assumption was valid).

 - When did the PMC made decision about dropping Hadoop 2 compatibility?
When and where was it announced?

If as a community which is officially part of the ASF we do have rules, why
do we look like not following them?

Regards,
Attila

ps:
Objections against dropping 1.5, is clearly the fact that having 1.5 was
decided as a community, and yet we're still not sure if those changes would
be only delivered in a next major version, how they would be backported,
cherry-picked, etc. And as the majority of the users are still on 2.x, I
think we cannot force ppl to upgrade, just to being able to use some of the
originally 1.5 planned changes.
So this item is absolutely -1 on my side!

ps2:
Daniel! I've provided some comments on your ORC Jira.

On Thu, May 10, 2018 at 4:00 PM, Boglarka Egyed  wrote:

> Hi All,
>
> Thank you Daniel for the update! I was also writing one when your email
> arrived so I'm just adding a couple of comments to that.
>
> New major version in JIRA:
> Version 3.0.0 has been created in JIRA
> , please feel free
> to use it on the corresponding JIRAs from now. As per my previous email I
> see no point in doing an 1.5.0 release currently so I'm OK with moving all
> the JIRAs having fix/target version of 1.5.0 to 3.0.0. Any objections?
>
> Update on the dependencies of the release:
> * Gradle patch needs some finalization and can be committed soon:
> https://reviews.apache.org/r/66067/
> * Kite removal effort has been started: SQOOP-3313
> 
> * Hive 3.0.0 release is still in an early phase based on this email thread
>  box/%3c2ec60da6-0a2e-4f3a-92f2-e3ce9d497...@hortonworks.com%3E>
> and has no ETA yet
>
> Thanks Daniel for looking into the Hadoop compatibility question, please
> let us know your findings.
>
> Cheers,
> Bogi
>
>
>
> On Thu, May 10, 2018 at 3:27 PM, Dániel Vörös 
> wrote:
>
> > Dear All,
> >
> > After Bogi has created the 3.0.0 version in Jira I've applied it to a
> > couple of tickets that don't make sense on the 1.x line (without
> > Hadoop3/Hive3).
> >
> > However, as Bogi has mentioned in her previous email, it probably doesn't
> > make sense to work on a 1.5 release in parallel with 3.0.0. How would you
> > feel if we were to move all 1.5 issues [1] to 3.0.0?
> >
> > In the meantime I've experimented with running Sqoop 1.4.7 against Hadoop
> > 3.1.0, and I'm planning to do the opposite, running Sqoop 3.0.0-SNAPSHOT
> > against Hadoop 2.x. That way we'd be able to better assess Attila's
> > question about backward compatibility. Please note, that the hard part
> will
> > be Hive integration I'm afraid, and until there's no Hive 3.0 release
> it's
> > hard to test. If anyone's interested in this topic, check out [2].
> >
> > Regards,
> > Daniel
> >
> > [1]
> > https://issues.apache.org/jira/issues?jql=project%20%3D%20SQ
> > OOP%20and%20fixVersion%20%3D%201.5.0%20and%20resolutionDate%
> > 20is%20not%20%20empty%20order%20by%20resolutiondate%20desc
> > [2] https://github.com/dvoros/docker-sqoop
> >
> > On Mon, Apr 16, 2018 at 2:20 PM Szabolcs Vasas  wrote:
> >
> > > Hi All,
> > >
> > > Sqoop NG/Sqoop 3:
> > > As far as I remember Sqoop NG was an alternative name suggested for
> > Sqoop 2
> > > which has a totally different architecture than Sqoop 1. I would not
> use
> > > now since in this release we do not include changes affecting the
> > > architecture but bumping the versions of the dependencies. However
> since
> > > dependencies are bumped to another major releases I think we should
> also
> > > change the major version number of Sqoop.
> > >
> > > Hadoop 2 support:
> > > I agree with Daniel that we should not introduce extra complexity to
> > > support Hadoop 2 as well. However even if we don't support Hadoop 2 in
> > our
> > > next major Sqoop release some features which do not require Hadoop 3
> > 

HBaseKerberizedConnectivityTest

[Attila Szabó]

Hello everyone,

Yesterday I've faced an issue with HBaseKerberizedConnectivityTest, that
the ant JUNIT runner keeps hanging after the test (it prints out the run
time (40+ seconds), the success statues, etc.), but after no other tests
are executed and it seems to be waiting until infinity. Same problem if I
do execute only this testcase. All other tests (unit+third party as well)
running without any issues.

The strange thing is: it's working on my LXSS Ubuntu, Debian, just not on
my CentOS7.4. I've tried with 1.9.2, 1.9.6, 1.9.10, different JDK versions,
etc.

Has anyone else experienced this kind of problem?
Do you have any suggestions?

Thanks,
Attila

Re: Question: Third party docker support file version

[Attila Szabó]

Hi,

I've also found another strange thing:
It seems oracle/database:12.2.0.1-ee is no longer available on dockerhub,
thus the usage of these scripts are very difficult at the beginning, and
needs lots of workarounds (of course it's feasible to clone github, build
the oracle image, etc., but it's very far from the original goal) to make
it work (nonetheless it's very much not working out of the box).

Do we plan to do anything with this? (e.g. creating an official Sqoop id
for dockerhub, and push their the prebuilt image or so). If yes I'd more
than happy to provide my help, guidance on this front.

Cheers,
Attila

<http://www.avg.com/email-signature?utm_medium=email_source=link_campaign=sig-email_content=webmail>
Virus-free.
www.avg.com
<http://www.avg.com/email-signature?utm_medium=email_source=link_campaign=sig-email_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Wed, Feb 14, 2018 at 12:14 PM, Attila Szabó <mau...@apache.org> wrote:

> Hi all,
>
> A few months ago Szabi created start and stop scripts for docker-compose,
> to run 3rd party DBs for the 3rd party tests, which is great ( it's
> intention is to make the integration testing of Sqoop much easier ).
>
> Though the version of docker-compose file is 3+, thus it's only usable
> with the quite recent docker-compose+docker versions.
>
> And here comes the problem:
> The latest version of CentOS (7.4) by default comes a much older
> Docker+Docker-compose version, which is incompatible with the compose file
> version. So out of the box, this solution won't work on the latest CentOS
> version for example (I've discovered it b/c one of my test servers runs
> CentOS7.4).
>
> I'm not telling it's impossible to install it from the net (first docker
> and then docker compose), but it's just not the standard/convenient/stable
> solution which would come from the RPM package.
>
> Here comes my question:
> What do you think: is it possible, and would it make sense to rewrite the
> docker compose file to version 2, and thus providing an EZ way to use this
> feature by the majority of the Linux users? Is there any vital feature in
> version 3 what is required for this task?
>
> Keeping it simple:
> Would it be possible to provide a much wider usable version of this docker
> based testing solution, or would we like to keep and handle it as an
> experimental testing feature?
>
> Thanks,
> Attila
>

Question: Third party docker support file version

[Attila Szabó]

Hi all,

A few months ago Szabi created start and stop scripts for docker-compose,
to run 3rd party DBs for the 3rd party tests, which is great ( it's
intention is to make the integration testing of Sqoop much easier ).

Though the version of docker-compose file is 3+, thus it's only usable with
the quite recent docker-compose+docker versions.

And here comes the problem:
The latest version of CentOS (7.4) by default comes a much older
Docker+Docker-compose version, which is incompatible with the compose file
version. So out of the box, this solution won't work on the latest CentOS
version for example (I've discovered it b/c one of my test servers runs
CentOS7.4).

I'm not telling it's impossible to install it from the net (first docker
and then docker compose), but it's just not the standard/convenient/stable
solution which would come from the RPM package.

Here comes my question:
What do you think: is it possible, and would it make sense to rewrite the
docker compose file to version 2, and thus providing an EZ way to use this
feature by the majority of the Linux users? Is there any vital feature in
version 3 what is required for this task?

Keeping it simple:
Would it be possible to provide a much wider usable version of this docker
based testing solution, or would we like to keep and handle it as an
experimental testing feature?

Thanks,
Attila

Thank you!

[Attila Szabó]

Dear Community,

I'd like to share my gratitude towards those Sqoop team members who helped
my job as a release manager related to Sqoop 1.4.7.

First of all I'd like to thank to Venkat Ranganathan! Without his wisdom
and help as a PMC member this release would not have been possible.

I'd like to also thank to Kathleen Thing, Anna Szőnyi, Boglárka Egyed and
Szabolcs Vasas. Their help and especially their testing efforts were also
invaluable!

I'm looking forward seeing newer and newer achievements from this great
community!

Best regards,
Attila



Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

[ANNOUNCE] Apache Sqoop 1.4.7 released

[Attila Szabó]

The Apache Sqoop team is pleased to announce the release of Sqoop 1.4.7.

This is the sixth release of Apache Sqoop as a TLP project. Sqoop is a
tool designed
for efficiently transferring bulk data between Apache Hadoop and
structured datastores,
such as relational databases.

The release is available here:http://www.apache.org/dyn/closer.cgi/sqoop/

The full change log is available
here:https://issues.apache.org/jira/secure/ReleaseNote.jspa?projectId=12311320=12329683

Your help and feedback is more than welcome. For more information on
how to report problems
and to get involved, visit the project website at http://sqoop.apache.org/.

The Apache Sqoop Team




Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Re: [VOTE] Release Sqoop version 1.4.7

[Attila Szabó]

Hey Venkat,

Thanks for the reply, it was not clear from the Wiki page that it's a PMC
only duty. ( AFAIR it's not mentioned there )

Two things we have to do :
- Close the release in the jira
- Add 1.4.7 to dist.apache.org and delete 1.4.6.

After I'll be able to send out the announcement email.

Thanks
Attila

On Jan 20, 2018 5:30 AM, "Venkat Ranganathan" <vranganat...@hortonworks.com>
wrote:

> Let me take a look at what needs to be done for the release.As a
> community release manager, you should work with a PMC member to get the
> release.   I can help you with some of those tasks
>
> Let me contact you offline on resolving these
>
> Venkat
>
> On 1/19/18, 7:01 AM, "Attila Szabó" <mau...@apache.org> wrote:
>
> Hi Venkat,
>
> It seems I really need your help/assistance. I've faced two issues I'm
> not
> able to resolve on my side:
> A. I'm not able to "RELEASE" the version 1.4.7 in JIRA, as it seems I
> do
> not have the right level of privileges to do that. If my understanding
> is
> correct I would need an admin access for the project to do this
> according
> to the Wiki site:
>
> https://cwiki.apache.org/confluence/display/SQOOP/How+
> to+Release#HowtoRelease-CloseJIRAversion
>
> "Close JIRA version
> You need to close the release in JIRA so that everyone knows that your
> version should not be used as "fixVersion" for new bugs. Go to JIRA
> "Administer project" page and follow "Versions" in left menu. Table
> with
> list of all releases should appear, click on additional menu on the
> right
> of your release and choose "Release" option. Submit release date and
> you're
> done."
>
> B. I'm still not able to commit anything to the dist.apache.org svn
> repository with my Apache user (maugli). So it seems my access to this
> repository is same as before when I wanted to commit my PGP keys. I
> think
> this is also a privilege/access related issue, but I'm not sure as I've
> never handled these SVN repos, only the git ones.
>
> Could you please help me in solving these two issues?
>
> Thanks in advance,
> Attila
>
> On Wed, Jan 17, 2018 at 3:15 AM, Attila Szabó <mau...@apache.org>
> wrote:
>
> > Hey Venkat,
> >
> > I've just seen your mail. I do really appreciate your help on this
> front.
> > I'll let you know if there is any problem on my side.
> >
> > Thanks in advance!!!
> >
> > Attila
> >
> > On Jan 17, 2018 1:31 AM, "Venkat Ranganathan" <
> > vranganat...@hortonworks.com> wrote:
> >
> >> Attila
> >>
> >> Do you need help in publishing the release?
> >>
> >> Venkat
> >>
> >> On 1/15/18, 4:36 AM, "Anna Szonyi" <szo...@cloudera.com> wrote:
> >>
> >> +1 to continuing the release, also a belated +1 (non-binding)
> for the
> >> release candidate!
> >>
> >> On Mon, Jan 15, 2018 at 10:49 AM, Szabolcs Vasas <
> va...@cloudera.com>
> >> wrote:
> >>
> >> > Hi Attila,
> >> >
> >> > It seems RC0 has received the necessary 3 votes from PMC
> members,
> >> can we
> >> > proceed with the release process?
> >> > Please let us know if you need any help.
> >> >
> >> > Regards,
> >> > Szabolcs
> >> >
> >> > On Thu, Jan 11, 2018 at 5:30 AM, Abraham Fine <
> af...@apache.org>
> >> wrote:
> >> >
> >> > > +1
> >> > >
> >> > > On Tue, Jan 9, 2018, at 17:15, Venkat Ranganathan wrote:
> >> > > > +1
> >> > > > Venkat
> >> > > >
> >> > > > On 1/9/18, 10:25 AM, "Kathleen Ting" <kathl...@apache.org
> >
> >> wrote:
> >> > > >
> >> > > > Thanks Attila for being the release manager.
> >> > > >
> >> > > > +1 on Sqoop 1.4.7 RC0.
> >> > > >
> >> > > > Thanks,
>     >>     > > > Kate
> >> > > >
> >> > > > On Fri, Jan 5, 2018 at 3:17 

Re: [VOTE] Release Sqoop version 1.4.7

[Attila Szabó]

Hi Venkat,

It seems I really need your help/assistance. I've faced two issues I'm not
able to resolve on my side:
A. I'm not able to "RELEASE" the version 1.4.7 in JIRA, as it seems I do
not have the right level of privileges to do that. If my understanding is
correct I would need an admin access for the project to do this according
to the Wiki site:

https://cwiki.apache.org/confluence/display/SQOOP/How+to+Release#HowtoRelease-CloseJIRAversion

"Close JIRA version
You need to close the release in JIRA so that everyone knows that your
version should not be used as "fixVersion" for new bugs. Go to JIRA
"Administer project" page and follow "Versions" in left menu. Table with
list of all releases should appear, click on additional menu on the right
of your release and choose "Release" option. Submit release date and you're
done."

B. I'm still not able to commit anything to the dist.apache.org svn
repository with my Apache user (maugli). So it seems my access to this
repository is same as before when I wanted to commit my PGP keys. I think
this is also a privilege/access related issue, but I'm not sure as I've
never handled these SVN repos, only the git ones.

Could you please help me in solving these two issues?

Thanks in advance,
Attila

On Wed, Jan 17, 2018 at 3:15 AM, Attila Szabó <mau...@apache.org> wrote:

> Hey Venkat,
>
> I've just seen your mail. I do really appreciate your help on this front.
> I'll let you know if there is any problem on my side.
>
> Thanks in advance!!!
>
> Attila
>
> On Jan 17, 2018 1:31 AM, "Venkat Ranganathan" <
> vranganat...@hortonworks.com> wrote:
>
>> Attila
>>
>> Do you need help in publishing the release?
>>
>> Venkat
>>
>> On 1/15/18, 4:36 AM, "Anna Szonyi" <szo...@cloudera.com> wrote:
>>
>> +1 to continuing the release, also a belated +1 (non-binding) for the
>> release candidate!
>>
>> On Mon, Jan 15, 2018 at 10:49 AM, Szabolcs Vasas <va...@cloudera.com>
>> wrote:
>>
>> > Hi Attila,
>> >
>> > It seems RC0 has received the necessary 3 votes from PMC members,
>> can we
>> > proceed with the release process?
>> > Please let us know if you need any help.
>> >
>> > Regards,
>> > Szabolcs
>> >
>> > On Thu, Jan 11, 2018 at 5:30 AM, Abraham Fine <af...@apache.org>
>> wrote:
>> >
>> > > +1
>> > >
>> > > On Tue, Jan 9, 2018, at 17:15, Venkat Ranganathan wrote:
>> > > > +1
>> > > > Venkat
>> > > >
>> > > > On 1/9/18, 10:25 AM, "Kathleen Ting" <kathl...@apache.org>
>> wrote:
>> > > >
>> > > > Thanks Attila for being the release manager.
>> > > >
>> > > > +1 on Sqoop 1.4.7 RC0.
>> > > >
>> > > > Thanks,
>> > > > Kate
>> > > >
>> > > > On Fri, Jan 5, 2018 at 3:17 AM, Boglarka Egyed <
>> b...@apache.org>
>> > > wrote:
>> > > >
>> > > > > Hi Attila,
>> > > > >
>> > > > > Thank you for preparing this release!
>> > > > >
>> > > > > I have checked the followings on RC0:
>> > > > > - checksums OK
>> > > > > - signature OK
>> > > > > - LICENSE and NOTICE files OK
>> > > > > - compile OK
>> > > > > - unit tests OK
>> > > > > - 3rd party tests OK
>> > > > >
>> > > > > Based on this +1 from my side too.
>> > > > >
>> > > > > Many thanks,
>> > > > > Bogi
>> > > > >
>> > > > > On Fri, Jan 5, 2018 at 12:04 PM, Attila Szabó <
>> mau...@apache.org
>> > >
>> > > wrote:
>> > > > >
>> > > > > > Dear Sqoop community,
>> > > > > >
>> > > > > > I'd like to kindly remind and ask you to vote on Sqoop
>> 1.4.7
>> > > RC0, if you
>> > > > > > would have some spare time.
>> > > > > >
>> > > > > > Thanks in advance,
&g

Re: [VOTE] Release Sqoop version 1.4.7

[Attila Szabó]

Hey Venkat,

I've just seen your mail. I do really appreciate your help on this front.
I'll let you know if there is any problem on my side.

Thanks in advance!!!

Attila

On Jan 17, 2018 1:31 AM, "Venkat Ranganathan" <vranganat...@hortonworks.com>
wrote:

> Attila
>
> Do you need help in publishing the release?
>
> Venkat
>
> On 1/15/18, 4:36 AM, "Anna Szonyi" <szo...@cloudera.com> wrote:
>
> +1 to continuing the release, also a belated +1 (non-binding) for the
> release candidate!
>
> On Mon, Jan 15, 2018 at 10:49 AM, Szabolcs Vasas <va...@cloudera.com>
> wrote:
>
> > Hi Attila,
> >
> > It seems RC0 has received the necessary 3 votes from PMC members,
> can we
> > proceed with the release process?
> > Please let us know if you need any help.
> >
> > Regards,
> > Szabolcs
> >
> > On Thu, Jan 11, 2018 at 5:30 AM, Abraham Fine <af...@apache.org>
> wrote:
> >
> > > +1
> > >
> > > On Tue, Jan 9, 2018, at 17:15, Venkat Ranganathan wrote:
> > > > +1
> > > > Venkat
> > > >
> > > > On 1/9/18, 10:25 AM, "Kathleen Ting" <kathl...@apache.org>
> wrote:
> > > >
> > > > Thanks Attila for being the release manager.
> > > >
> > > > +1 on Sqoop 1.4.7 RC0.
> > > >
> > > > Thanks,
> > > > Kate
> > > >
> > > > On Fri, Jan 5, 2018 at 3:17 AM, Boglarka Egyed <
> b...@apache.org>
> > > wrote:
> > > >
> > > > > Hi Attila,
> > > > >
> > > > > Thank you for preparing this release!
> > > > >
> > > > > I have checked the followings on RC0:
> > > > > - checksums OK
> > > > > - signature OK
> > > > > - LICENSE and NOTICE files OK
> > > > > - compile OK
> > > > > - unit tests OK
> > > > > - 3rd party tests OK
> > > > >
> > > > > Based on this +1 from my side too.
> > > > >
> > > > > Many thanks,
> > > > > Bogi
> > > > >
> > > > > On Fri, Jan 5, 2018 at 12:04 PM, Attila Szabó <
> mau...@apache.org
> > >
> > > wrote:
> > > > >
> > > > > > Dear Sqoop community,
> > > > > >
> > > > > > I'd like to kindly remind and ask you to vote on Sqoop
> 1.4.7
> > > RC0, if you
> > > > > > would have some spare time.
> > > > > >
> > > > > > Thanks in advance,
> > > > > > Attila
> > > > > >
> > > > > >
> > > > > > On Dec 21, 2017 6:44 PM, "Attila Szabó" <
> mau...@apache.org>
> > > wrote:
> > > > > >
> > > > > > Hey Szabi,
> > > > > >
> > > > > > AWESOME!
> > > > > >
> > > > > > Thanks for validating my efforts!!
> > > > > >
> > > > > > Cheers,
> > > > > > Attila
> > > > > >
> > > > > > On Dec 21, 2017 6:36 PM, "Szabolcs Vasas" <
> va...@apache.org>
> > > wrote:
> > > > > >
> > > > > > > Hi Attila,
> > > > > > >
> > > > > > > Thank you for updating the artifacts!
> > > > > > >
> > > > > > > I have executed a few test commands, they worked fine.
> > > > > > > The unit tests and the third party tests are passing.
> > > > > > > The signature and hash files are correct.
> > > > > > >
> > > > > > > From my side it looks good, +1
> > > > > > >
> > > > > > > Thank you for preparing this release!
> > > > > > >
> > > > > > > Szabolcs
> > > &

Re: [VOTE] Release Sqoop version 1.4.7

[Attila Szabó]

Hello community,

I think we're on the track after 3 PMC +1's.

Though I totally can understand your eager to have the new release I'd like
to give the possibility to "old" Abe and Jarcec to cast there votes until
Thursday evening.

If we won't face any concerns I'll move forward with the release on
Thursday ( my Sqoop day of the week )

If I would need any help I'll let you know! Many thanks in advance!.

Attila

On Jan 16, 2018 4:17 PM, "Boglarka Egyed" <b...@apache.org> wrote:

> Hi Attila,
>
> Thanks for putting together the release candidate!
>
> Could you maybe give an ETA for finishing the release up now that all the
> +1 votes have been received?
>
> And of course, please let me know if you need any help.
>
> Many thanks.
> Bogi
>
> On Mon, Jan 15, 2018 at 1:35 PM, Anna Szonyi <szo...@cloudera.com> wrote:
>
> > +1 to continuing the release, also a belated +1 (non-binding) for the
> > release candidate!
> >
> > On Mon, Jan 15, 2018 at 10:49 AM, Szabolcs Vasas <va...@cloudera.com>
> > wrote:
> >
> > > Hi Attila,
> > >
> > > It seems RC0 has received the necessary 3 votes from PMC members, can
> we
> > > proceed with the release process?
> > > Please let us know if you need any help.
> > >
> > > Regards,
> > > Szabolcs
> > >
> > > On Thu, Jan 11, 2018 at 5:30 AM, Abraham Fine <af...@apache.org>
> wrote:
> > >
> > > > +1
> > > >
> > > > On Tue, Jan 9, 2018, at 17:15, Venkat Ranganathan wrote:
> > > > > +1
> > > > > Venkat
> > > > >
> > > > > On 1/9/18, 10:25 AM, "Kathleen Ting" <kathl...@apache.org> wrote:
> > > > >
> > > > > Thanks Attila for being the release manager.
> > > > >
> > > > > +1 on Sqoop 1.4.7 RC0.
> > > > >
> > > > > Thanks,
> > > > > Kate
> > > > >
> > > > > On Fri, Jan 5, 2018 at 3:17 AM, Boglarka Egyed <
> b...@apache.org>
> > > > wrote:
> > > > >
> > > > > > Hi Attila,
> > > > > >
> > > > > > Thank you for preparing this release!
> > > > > >
> > > > > > I have checked the followings on RC0:
> > > > > > - checksums OK
> > > > > > - signature OK
> > > > > > - LICENSE and NOTICE files OK
> > > > > > - compile OK
> > > > > > - unit tests OK
> > > > > > - 3rd party tests OK
> > > > > >
> > > > > > Based on this +1 from my side too.
> > > > > >
> > > > > > Many thanks,
> > > > > > Bogi
> > > > > >
> > > > > > On Fri, Jan 5, 2018 at 12:04 PM, Attila Szabó <
> > mau...@apache.org
> > > >
> > > > wrote:
> > > > > >
> > > > > > > Dear Sqoop community,
> > > > > > >
> > > > > > > I'd like to kindly remind and ask you to vote on Sqoop
> 1.4.7
> > > > RC0, if you
> > > > > > > would have some spare time.
> > > > > > >
> > > > > > > Thanks in advance,
> > > > > > > Attila
> > > > > > >
> > > > > > >
> > > > > > > On Dec 21, 2017 6:44 PM, "Attila Szabó" <mau...@apache.org
> >
> > > > wrote:
> > > > > > >
> > > > > > > Hey Szabi,
> > > > > > >
> > > > > > > AWESOME!
> > > > > > >
> > > > > > > Thanks for validating my efforts!!
> > > > > > >
> > > > > > > Cheers,
> > > > > > > Attila
> > > > > > >
> > > > > > > On Dec 21, 2017 6:36 PM, "Szabolcs Vasas" <
> va...@apache.org>
> > > > wrote:
> > > > > > >
> > > > > > > > Hi Attila,
> > > > > > > >
> > > > > > > > Thank you for updating the artifacts!
> > > > > > > >
> > > > > > >

Re: Removing com.cloudera.sqoop packages

[Attila Szabó]

Hey Szabi,

Souds great!

Two comments:
AFAIR Anna Szonyi was also planning to do something similar last January
(when she was busy around the build system, tests, gradle, etc.). It would
make sense on my side, to reach out to her, maybe she's got some useful
feedbacks for you too.
I've opened an issue on the review board (which if I'm not mistaken you've
already fixed meanwhile I've been preparing this email ;-) ).

I'll give my +1 and ship it, as soon as I will be able to execute the
test/releasenotes/etc. on my side as well!

Many thanks for jumping on this initiative, it's a quite old item on my
wishlist!

Cheers,
Attila,

ps: I do not have any pros or cons on the side of Pull Request VS. Review
Board, and I totally can understand if it's better for someone to review
182 commits instead of a big one. Although I would strongly advise and
appreciate if the commits would be squashed into one before merging them to
the trunk, b/c testing the effects of 180+ commits, one by one would
consume tons of efforts. If the commit to the master would/will be up to
me, I would use the patch file instead, or squash first, just for the clean
state of the trunk as well.

My2cents

On Tue, Jan 9, 2018 at 11:49 AM, Ferenc Szabo  wrote:

> Hi Szabi,
>
> I believe this is a great idea.
>
> By removing these packages we will get rid of a great deal of technical
> debt that will simplify future change. I will also help to avoid
> unnecessary conversions like the ones I had to use in my recent
> SqoopOptions related change.
>
> So, also +1 from me!
>
> Cheers,
> Fero
>
> On Mon, Jan 8, 2018 at 5:59 PM, Boglarka Egyed  wrote:
>
> > Hi Szabolcs,
> >
> > I really welcome this initiative, it would be a huge clean up on this
> > project!
> >
> > I already took a look at your pull request and it indeed looks pretty
> > straightforward.
> >
> > I will perform a deeper review and publish it on the Review Board
> otherwise
> > +1 from my side for the idea in general, 1.5 release would be a good
> target
> > for this.
> >
> > Thanks for taking this effort!
> >
> > Cheers.
> > Bogi
> >
> > On Mon, Jan 8, 2018 at 2:01 PM, Szabolcs Vasas  wrote:
> >
> > > Hi All,
> > >
> > > As you probably know we still have dozens of classes in
> > com.cloudera.sqoop
> > > packages which most of the cases just extend their corresponding class
> in
> > > org.apache.sqoop package without adding extra functionality.
> > > These classes make the code harder to read and navigate, they are
> already
> > > deprecated but because of backward compatibility considerations we were
> > not
> > > able to remove them.
> > > The community was planning on a release containing breaking changes (we
> > > called it 1.5) I think it would be a great candidate to include this
> > > cleanup.
> > > I have created a ticket  > jira/browse/SQOOP-3273>
> > > for doing this task and submitted a patch as well. I think the easiest
> > way
> > > to take a look at it is to review the commits in my Sqoop fork:
> > > https://github.com/szvasas/sqoop/tree/cloudera_package_removal
> > > Note that this is change which basically affects all of the files in
> the
> > > Sqoop project, but in the majority of the cases replacing the
> > > com.cloudera.sqoop class to its corresponding org.apache.sqoop class
> was
> > > just a find and replace command so I consider this a relatively low
> risk
> > > change.
> > > Please feel free to reply to this email with your questions and
> concerns
> > > and if you have some time please take a look at the changes.
> > >
> > > Thanks and regards,
> > > Szabolcs
> > >
> >
>
>
>
> --
> Ferenc Szabo
> Software Engineer
>

Re: [VOTE] Release Sqoop version 1.4.7

[Attila Szabó]

Dear Sqoop community,

I'd like to kindly remind and ask you to vote on Sqoop 1.4.7 RC0, if you
would have some spare time.

Thanks in advance,
Attila


On Dec 21, 2017 6:44 PM, "Attila Szabó" <mau...@apache.org> wrote:

Hey Szabi,

AWESOME!

Thanks for validating my efforts!!

Cheers,
Attila

On Dec 21, 2017 6:36 PM, "Szabolcs Vasas" <va...@apache.org> wrote:

> Hi Attila,
>
> Thank you for updating the artifacts!
>
> I have executed a few test commands, they worked fine.
> The unit tests and the third party tests are passing.
> The signature and hash files are correct.
>
> From my side it looks good, +1
>
> Thank you for preparing this release!
>
> Szabolcs
>
>
> On Thu, Dec 21, 2017 at 4:29 PM, Attila Szabó <mau...@apache.org> wrote:
>
> > I've rebuilt the RC0 with the proper line endings (not crlf git options).
> > Updated the tar.gz .asc and .md5 files under the specified location.
> >
> > If you need any further changes please notify me.
> >
> > On Thu, Dec 21, 2017 at 4:08 PM, Attila Szabó <mau...@apache.org> wrote:
> >
> > > Hey Szabi,
> > >
> > > Many thanks! It looks like the core.autocrlf global git config tricked
> > > me... Nice catch!
> > > What would you prefer:
> > > Just rebuild the RC0, or retag the same state as RC1 and do a totally
> new
> > > RC1 build and vote? (I did not see rules for this)
> > >
> > > The proper link for the keys file:
> > > http://www.apache.org/dist/sqoop/KEYS
> > >
> > > Cheers,
> > > Attila
> > >
> > > On Thu, Dec 21, 2017 at 3:21 PM, Szabolcs Vasas <va...@apache.org>
> > wrote:
> > >
> > >> Hi Attila,
> > >>
> > >> I did not have time to run the full testing yet but it seems that the
> > >> scripts in the bin directory have Microsoft-style line endings so I am
> > not
> > >> able to start them.
> > >> Also http://www.apache.org/dist/incubator/sqoop/KEYS seems to be
> > broken,
> > >> can you please update the link?
> > >>
> > >> Thanks and regards,
> > >> Szabolcs
> > >>
> > >> On Tue, Dec 19, 2017 at 11:03 PM, Attila Szabó <mau...@apache.org>
> > wrote:
> > >>
> > >> > This is the first release for Apache Sqoop, version 1.4.7
> > >> >
> > >> > *** Please cast your vote by [3 working days after sending] ***
> > >> >
> > >> > The list of fixed
> > >> > issues:https://git-wip-us.apache.org/repos/asf?p=sqoop.
> > >> > git;a=blob;f=CHANGELOG.txt;h=c3bf922e49bc9bc35f47ca44221b505
> > >> 19202c749;hb=
> > >> > 2328971411f57f0cb683dfb79d19d4d19d185dd8
> > >> >
> > >> > The tarball (*.tar.gz), signature (*.asc), checksum (*.md5),
> > >> > and test result
> > >> > (log/*.ant_test.log):http://people.apache.org/~maugli/
> > sqoop-1.4.7-rc0/
> > >> >
> > >> > The tag to be voted
> > >> > upon:https://git-wip-us.apache.org/repos/asf?p=sqoop.
> > >> > git;a=shortlog;h=refs/tags/release-1.4.7-rc0
> > >> >
> > >> > The KEYS file:http://www.apache.org/dist/incubator/sqoop/KEYS
> > >> >
> > >> >
> > >> > Best regards,
> > >> >
> > >> > Attila Szabo
> > >> >
> > >>
> > >
> > >
> >
>

Re: Please update the team members list on the project site

[Attila Szabó]

Hey Bogi,

AWESOME!

Thanks,
Attila

On Jan 5, 2018 11:54 AM, "Boglarka Egyed" <b...@apache.org> wrote:

> Hi Attila,
>
> Thanks for bringing this up!
>
> I have updated the Team List <http://sqoop.apache.org/team-list.html>
> based
> on the guideline here
> <https://cwiki.apache.org/confluence/display/SQOOP/How+
> to+Update+Project+Website>
> as
> it turned out that committer rights are sufficient for it too.
>
> Cheers,
> Bogi
>
>
> On Wed, Dec 20, 2017 at 1:44 PM, Attila Szabó <mau...@apache.org> wrote:
>
> > Hello everyone,
> >
> > As I've noticed (from the commits) in the past few weeks Szabi (Szabolcs
> > Vasas) has become a committer, and Bogi (Boglarka Egyed) has been a
> > committer since August (BTW: CONGRATS GUYS ONCE MORE!!!), but if I'm not
> > mistaken no one has updated the team list on the project site (
> > http://sqoop.apache.org/team-list.html), thus they are not yet
> > included/printed there.
> >
> > I'd like to kindly ask someone's help (who has the right level of access
> > and permissons for this) to please update the site!
> >
> > Thanks in advance,
> >
> > Attila
> >
>

Re: [VOTE] Release Sqoop version 1.4.7

[Attila Szabó]

Hey Szabi,

AWESOME!

Thanks for validating my efforts!!

Cheers,
Attila

On Dec 21, 2017 6:36 PM, "Szabolcs Vasas" <va...@apache.org> wrote:

> Hi Attila,
>
> Thank you for updating the artifacts!
>
> I have executed a few test commands, they worked fine.
> The unit tests and the third party tests are passing.
> The signature and hash files are correct.
>
> From my side it looks good, +1
>
> Thank you for preparing this release!
>
> Szabolcs
>
>
> On Thu, Dec 21, 2017 at 4:29 PM, Attila Szabó <mau...@apache.org> wrote:
>
> > I've rebuilt the RC0 with the proper line endings (not crlf git options).
> > Updated the tar.gz .asc and .md5 files under the specified location.
> >
> > If you need any further changes please notify me.
> >
> > On Thu, Dec 21, 2017 at 4:08 PM, Attila Szabó <mau...@apache.org> wrote:
> >
> > > Hey Szabi,
> > >
> > > Many thanks! It looks like the core.autocrlf global git config tricked
> > > me... Nice catch!
> > > What would you prefer:
> > > Just rebuild the RC0, or retag the same state as RC1 and do a totally
> new
> > > RC1 build and vote? (I did not see rules for this)
> > >
> > > The proper link for the keys file:
> > > http://www.apache.org/dist/sqoop/KEYS
> > >
> > > Cheers,
> > > Attila
> > >
> > > On Thu, Dec 21, 2017 at 3:21 PM, Szabolcs Vasas <va...@apache.org>
> > wrote:
> > >
> > >> Hi Attila,
> > >>
> > >> I did not have time to run the full testing yet but it seems that the
> > >> scripts in the bin directory have Microsoft-style line endings so I am
> > not
> > >> able to start them.
> > >> Also http://www.apache.org/dist/incubator/sqoop/KEYS seems to be
> > broken,
> > >> can you please update the link?
> > >>
> > >> Thanks and regards,
> > >> Szabolcs
> > >>
> > >> On Tue, Dec 19, 2017 at 11:03 PM, Attila Szabó <mau...@apache.org>
> > wrote:
> > >>
> > >> > This is the first release for Apache Sqoop, version 1.4.7
> > >> >
> > >> > *** Please cast your vote by [3 working days after sending] ***
> > >> >
> > >> > The list of fixed
> > >> > issues:https://git-wip-us.apache.org/repos/asf?p=sqoop.
> > >> > git;a=blob;f=CHANGELOG.txt;h=c3bf922e49bc9bc35f47ca44221b505
> > >> 19202c749;hb=
> > >> > 2328971411f57f0cb683dfb79d19d4d19d185dd8
> > >> >
> > >> > The tarball (*.tar.gz), signature (*.asc), checksum (*.md5),
> > >> > and test result
> > >> > (log/*.ant_test.log):http://people.apache.org/~maugli/
> > sqoop-1.4.7-rc0/
> > >> >
> > >> > The tag to be voted
> > >> > upon:https://git-wip-us.apache.org/repos/asf?p=sqoop.
> > >> > git;a=shortlog;h=refs/tags/release-1.4.7-rc0
> > >> >
> > >> > The KEYS file:http://www.apache.org/dist/incubator/sqoop/KEYS
> > >> >
> > >> >
> > >> > Best regards,
> > >> >
> > >> > Attila Szabo
> > >> >
> > >>
> > >
> > >
> >
>

Re: [VOTE] Release Sqoop version 1.4.7

[Attila Szabó]

I've rebuilt the RC0 with the proper line endings (not crlf git options).
Updated the tar.gz .asc and .md5 files under the specified location.

If you need any further changes please notify me.

On Thu, Dec 21, 2017 at 4:08 PM, Attila Szabó <mau...@apache.org> wrote:

> Hey Szabi,
>
> Many thanks! It looks like the core.autocrlf global git config tricked
> me... Nice catch!
> What would you prefer:
> Just rebuild the RC0, or retag the same state as RC1 and do a totally new
> RC1 build and vote? (I did not see rules for this)
>
> The proper link for the keys file:
> http://www.apache.org/dist/sqoop/KEYS
>
> Cheers,
> Attila
>
> On Thu, Dec 21, 2017 at 3:21 PM, Szabolcs Vasas <va...@apache.org> wrote:
>
>> Hi Attila,
>>
>> I did not have time to run the full testing yet but it seems that the
>> scripts in the bin directory have Microsoft-style line endings so I am not
>> able to start them.
>> Also http://www.apache.org/dist/incubator/sqoop/KEYS seems to be broken,
>> can you please update the link?
>>
>> Thanks and regards,
>> Szabolcs
>>
>> On Tue, Dec 19, 2017 at 11:03 PM, Attila Szabó <mau...@apache.org> wrote:
>>
>> > This is the first release for Apache Sqoop, version 1.4.7
>> >
>> > *** Please cast your vote by [3 working days after sending] ***
>> >
>> > The list of fixed
>> > issues:https://git-wip-us.apache.org/repos/asf?p=sqoop.
>> > git;a=blob;f=CHANGELOG.txt;h=c3bf922e49bc9bc35f47ca44221b505
>> 19202c749;hb=
>> > 2328971411f57f0cb683dfb79d19d4d19d185dd8
>> >
>> > The tarball (*.tar.gz), signature (*.asc), checksum (*.md5),
>> > and test result
>> > (log/*.ant_test.log):http://people.apache.org/~maugli/sqoop-1.4.7-rc0/
>> >
>> > The tag to be voted
>> > upon:https://git-wip-us.apache.org/repos/asf?p=sqoop.
>> > git;a=shortlog;h=refs/tags/release-1.4.7-rc0
>> >
>> > The KEYS file:http://www.apache.org/dist/incubator/sqoop/KEYS
>> >
>> >
>> > Best regards,
>> >
>> > Attila Szabo
>> >
>>
>
>

Re: [VOTE] Release Sqoop version 1.4.7

[Attila Szabó]

Hey Szabi,

Many thanks! It looks like the core.autocrlf global git config tricked
me... Nice catch!
What would you prefer:
Just rebuild the RC0, or retag the same state as RC1 and do a totally new
RC1 build and vote? (I did not see rules for this)

The proper link for the keys file:
http://www.apache.org/dist/sqoop/KEYS

Cheers,
Attila

On Thu, Dec 21, 2017 at 3:21 PM, Szabolcs Vasas <va...@apache.org> wrote:

> Hi Attila,
>
> I did not have time to run the full testing yet but it seems that the
> scripts in the bin directory have Microsoft-style line endings so I am not
> able to start them.
> Also http://www.apache.org/dist/incubator/sqoop/KEYS seems to be broken,
> can you please update the link?
>
> Thanks and regards,
> Szabolcs
>
> On Tue, Dec 19, 2017 at 11:03 PM, Attila Szabó <mau...@apache.org> wrote:
>
> > This is the first release for Apache Sqoop, version 1.4.7
> >
> > *** Please cast your vote by [3 working days after sending] ***
> >
> > The list of fixed
> > issues:https://git-wip-us.apache.org/repos/asf?p=sqoop.
> > git;a=blob;f=CHANGELOG.txt;h=c3bf922e49bc9bc35f47ca44221b50
> 519202c749;hb=
> > 2328971411f57f0cb683dfb79d19d4d19d185dd8
> >
> > The tarball (*.tar.gz), signature (*.asc), checksum (*.md5),
> > and test result
> > (log/*.ant_test.log):http://people.apache.org/~maugli/sqoop-1.4.7-rc0/
> >
> > The tag to be voted
> > upon:https://git-wip-us.apache.org/repos/asf?p=sqoop.
> > git;a=shortlog;h=refs/tags/release-1.4.7-rc0
> >
> > The KEYS file:http://www.apache.org/dist/incubator/sqoop/KEYS
> >
> >
> > Best regards,
> >
> > Attila Szabo
> >
>

Please update the team members list on the project site

[Attila Szabó]

Hello everyone,

As I've noticed (from the commits) in the past few weeks Szabi (Szabolcs
Vasas) has become a committer, and Bogi (Boglarka Egyed) has been a
committer since August (BTW: CONGRATS GUYS ONCE MORE!!!), but if I'm not
mistaken no one has updated the team list on the project site (
http://sqoop.apache.org/team-list.html), thus they are not yet
included/printed there.

I'd like to kindly ask someone's help (who has the right level of access
and permissons for this) to please update the site!

Thanks in advance,

Attila

[VOTE] Release Sqoop version 1.4.7

[Attila Szabó]

This is the first release for Apache Sqoop, version 1.4.7

*** Please cast your vote by [3 working days after sending] ***

The list of fixed
issues:https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=blob;f=CHANGELOG.txt;h=c3bf922e49bc9bc35f47ca44221b50519202c749;hb=2328971411f57f0cb683dfb79d19d4d19d185dd8

The tarball (*.tar.gz), signature (*.asc), checksum (*.md5),
and test result
(log/*.ant_test.log):http://people.apache.org/~maugli/sqoop-1.4.7-rc0/

The tag to be voted
upon:https://git-wip-us.apache.org/repos/asf?p=sqoop.git;a=shortlog;h=refs/tags/release-1.4.7-rc0

The KEYS file:http://www.apache.org/dist/incubator/sqoop/KEYS


Best regards,

Attila Szabo

Re: SQOOP LICENSE.txt question

[Attila Szabó]

Hello everyone,

Though Bogi's discovery made a strong pro on the side to keep the
LICENSE.txt like it is (and as it were before),
after checking other projects (including Apache Hadoop, Beam, Spark, Arrow,
Mesos, HBase, etc.), I've just become even more and more confused if there
is any standard requirement against our product/community on this front.
I've seen several projects including explicit version numbers, I've seen
projects not even mentioning the dependencies, and also I've seen projects
like Flume and Sqoop just having a "" placeholder instead of
explicitly defined version numbers.

Could anyone from the group of the "veteran" community members give us any
insight which behaviour would be the required one in case of Sqoop?

@Abefine, @Kathleen:
What do you think?

Many thanks in advance,
Attila

<http://www.avg.com/email-signature?utm_medium=email_source=link_campaign=sig-email_content=webmail>
Virus-free.
www.avg.com
<http://www.avg.com/email-signature?utm_medium=email_source=link_campaign=sig-email_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Tue, Dec 5, 2017 at 5:10 PM, Attila Szabó <mau...@apache.org> wrote:

> Hey Bogi,
>
> AWESOME!!! Many thanks for taking this initiative, helps a lot.
>
> I'll adjust the changes in a few hours ( right now I'm traveling!)
>
> Many thanks once more,
> Attila
>
> On Dec 5, 2017 12:35 PM, "Boglarka Egyed" <b...@apache.org> wrote:
>
>> Hi Attila,
>>
>> I am not a PMC Member but I took a look and found that e.g. for Flume the
>> version handling is the same, see https://github.com/apache/
>> flume/blob/trunk/LICENSE or for the latest Sqoop2 release too:
>> https://github.com/apache/sqoop/blob/sqoop2/LICENSE.txt
>>
>> Based on this I would say we should follow the existing process and also
>> regarding the binary/source tar.gz files I would do the same as in the
>> previous release.
>>
>> Of course, a confirmation from a PMC Member would be great.
>>
>> Cheers,
>> Bogi
>>
>>
>>
>> On Fri, Nov 3, 2017 at 2:15 AM, Attila Szabó <mau...@apache.org> wrote:
>>
>> > Dear PMC members,
>> >
>> > During the preparation of 1.4.7 I've found something interesting in
>> > connection with the LICENSE.txt file we've used to bound together with
>> the
>> > SQOOP tar.gz files.
>> >
>> > As I've seen since 1.4.0-incubating (before that I've got no
>> information,
>> > as I've checked it from the location http://archive.apache.org/dist
>> /sqoop/
>> > ) we've never filled out the proper version number of the dependencies,
>> but
>> > rather just keeping it in the following format:
>> > lib/xy-.jar - where xy would be the name of the related
>> > dependency, but the "" placeholder never filled with the proper
>> > version number.
>> >
>> > I've also found some discrepancy between the LICENSE.txt convention of
>> the
>> > binary/source tar.gz files from version to version. A good example for
>> > that:
>> >  - in case of 1.4.5 in the source jar only those jars are listed which
>> are
>> > bound within the tar.gz
>> >  - in case of 1.4.6 in the source jar all of the jars are listed which
>> had
>> > been used for the binary version as well.
>> >
>> > My questions are the following:
>> > - Should we follow the existing process with 1.4.7, and not filling the
>> > exact version numbers, or should we ship with a fully LICENSE.txt?
>> > - Should we make a difference between the source and binary tar.gz files
>> > (like in case of 1.4.5) or should we follow the convention of the very
>> last
>> > release (1.4.6)?
>> >
>> > Looking forward reading your answers,
>> >
>> > Yours,
>> > Attila
>> >
>> > <http://www.avg.com/email-signature?utm_medium=email_
>> > source=link_campaign=sig-email_content=webmail>
>> > Virus-free.
>> > www.avg.com
>> > <http://www.avg.com/email-signature?utm_medium=email_
>> > source=link_campaign=sig-email_content=webmail>
>> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
>> >
>>
>

Re: SQOOP LICENSE.txt question

[Attila Szabó]

Hey Bogi,

AWESOME!!! Many thanks for taking this initiative, helps a lot.

I'll adjust the changes in a few hours ( right now I'm traveling!)

Many thanks once more,
Attila

On Dec 5, 2017 12:35 PM, "Boglarka Egyed" <b...@apache.org> wrote:

> Hi Attila,
>
> I am not a PMC Member but I took a look and found that e.g. for Flume the
> version handling is the same, see https://github.com/apache/
> flume/blob/trunk/LICENSE or for the latest Sqoop2 release too:
> https://github.com/apache/sqoop/blob/sqoop2/LICENSE.txt
>
> Based on this I would say we should follow the existing process and also
> regarding the binary/source tar.gz files I would do the same as in the
> previous release.
>
> Of course, a confirmation from a PMC Member would be great.
>
> Cheers,
> Bogi
>
>
>
> On Fri, Nov 3, 2017 at 2:15 AM, Attila Szabó <mau...@apache.org> wrote:
>
> > Dear PMC members,
> >
> > During the preparation of 1.4.7 I've found something interesting in
> > connection with the LICENSE.txt file we've used to bound together with
> the
> > SQOOP tar.gz files.
> >
> > As I've seen since 1.4.0-incubating (before that I've got no information,
> > as I've checked it from the location http://archive.apache.org/
> dist/sqoop/
> > ) we've never filled out the proper version number of the dependencies,
> but
> > rather just keeping it in the following format:
> > lib/xy-.jar - where xy would be the name of the related
> > dependency, but the "" placeholder never filled with the proper
> > version number.
> >
> > I've also found some discrepancy between the LICENSE.txt convention of
> the
> > binary/source tar.gz files from version to version. A good example for
> > that:
> >  - in case of 1.4.5 in the source jar only those jars are listed which
> are
> > bound within the tar.gz
> >  - in case of 1.4.6 in the source jar all of the jars are listed which
> had
> > been used for the binary version as well.
> >
> > My questions are the following:
> > - Should we follow the existing process with 1.4.7, and not filling the
> > exact version numbers, or should we ship with a fully LICENSE.txt?
> > - Should we make a difference between the source and binary tar.gz files
> > (like in case of 1.4.5) or should we follow the convention of the very
> last
> > release (1.4.6)?
> >
> > Looking forward reading your answers,
> >
> > Yours,
> > Attila
> >
> > <http://www.avg.com/email-signature?utm_medium=email_
> > source=link_campaign=sig-email_content=webmail>
> > Virus-free.
> > www.avg.com
> > <http://www.avg.com/email-signature?utm_medium=email_
> > source=link_campaign=sig-email_content=webmail>
> > <#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
> >
>

Re: Sqoop Meetup Page

[Attila Szabó]

Hey Jarcec,

As I've seen the community activity in the past 1-1.5 year, committers were
active only from the EU, and regardless the fact we've received patches
from all over the world (which is more than fascinating dear Sqoop
fellas!!!, keep it up!),
right now I do not know anyone who would be an active evangelist/enthusiast
about Sqoop in CA/PA.

So unless if we have someone, who would be a dedicated for the Sqoop cause
in the US with tons of free time, I would not advise resurrecting the page.

My 2cents,
Attila


Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Mon, Nov 20, 2017 at 4:34 PM,  wrote:

> Dear Sqoop users and developers,,
> as you might know, Sqoop have it’s own Sqoop meetup page:
>
> https://www.meetup.com/Sqoop-User-Meetup/  User-Meetup/>
>
> I’m currently the main organizer, but due to my day job keeping me
> sufficiently occupied, I’m not doing the meetup justice. As a result last
> meetup we organized through this page was more then three years ago
> (October 2014). Hence I wonder if it’s still useful to keep it around - and
> I’m curious what the community think at large? Do you see value in
> resurrecting the page?
>
> Jarcec

SQOOP LICENSE.txt question

[Attila Szabó]

Dear PMC members,

During the preparation of 1.4.7 I've found something interesting in
connection with the LICENSE.txt file we've used to bound together with the
SQOOP tar.gz files.

As I've seen since 1.4.0-incubating (before that I've got no information,
as I've checked it from the location http://archive.apache.org/dist/sqoop/
) we've never filled out the proper version number of the dependencies, but
rather just keeping it in the following format:
lib/xy-.jar - where xy would be the name of the related
dependency, but the "" placeholder never filled with the proper
version number.

I've also found some discrepancy between the LICENSE.txt convention of the
binary/source tar.gz files from version to version. A good example for that:
 - in case of 1.4.5 in the source jar only those jars are listed which are
bound within the tar.gz
 - in case of 1.4.6 in the source jar all of the jars are listed which had
been used for the binary version as well.

My questions are the following:
- Should we follow the existing process with 1.4.7, and not filling the
exact version numbers, or should we ship with a fully LICENSE.txt?
- Should we make a difference between the source and binary tar.gz files
(like in case of 1.4.5) or should we follow the convention of the very last
release (1.4.6)?

Looking forward reading your answers,

Yours,
Attila


Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Help needed with SQOOP-3205 (blocking 1.4.7 release)

[Attila Szabó]

Dear PMC members,

I've been following the wiki

guide, and tried to update the KEYS file with my PGP key (to finally being
able to release RC0), though I'm not permitted to commit any changes.

Could you please help me finding someone who has the right level of
privileges, and would help me with this task?

Attached I'm sending my public PGP key entry, in the required form (and
also an SVN patch file, maybe it's more convenient that way).

Thanks in advance,
Attila


Virus-free.
www.avg.com

<#m_838463530951546137_DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>
Index: KEYS
===
--- KEYS	(revision 22749)
+++ KEYS	(working copy)
@@ -1107,3 +1107,62 @@
 FaekicXfPAk3BWS7q6wD7f1toZ9cLV9WrhkpKTjQeFvN/0wzgQrudmxj4w==
 =yZkk
 -END PGP PUBLIC KEY BLOCK-
+pub   rsa4096 2017-11-01 [SC] [expires: 2027-10-30]
+  2A6F8D0FEF3DAB4B9BC424F99FC8D8321F68EA1B
+uid   [ultimate] Attila Szabo 
+sig 39FC8D8321F68EA1B 2017-11-01  Attila Szabo 
+sub   rsa4096 2017-11-01 [E] [expires: 2027-10-30]
+sig  9FC8D8321F68EA1B 2017-11-01  Attila Szabo 
+
+-BEGIN PGP PUBLIC KEY BLOCK-
+
+mQINBFn50MgBEADBVKL51dfoxQo29XlTPsPasI4uzdXw9OuoofY9rzPpFB1jDNH4
+1LRZLs10ThjaFLVp/6WyK4MKnYE0FLHbmc+eJkwx+H0RyYnaxwcASHq2Cp5LienQ
+PqAvDeEgeUOej/AW4Oc2CQ9e9ZCJTrpM3b2nQZaGmWQk2bqTdlWCGbv2yEGzWTzE
+fZV7nJPkjloBrpOmI6wO6EsW5+fl6GJ/kaFBtA3dVsSE3Ez3QicGSBiG/9kCgXVb
+UPRvUMKxF815jPOewpTUVCLKl3dEwtmfCUrdT9vQfL1faY9qf01eVMUZaplgv/oF
+LlYOXzSB16ROhz+0WSkluVvk5PECgibRlE2Bb53HngRrx0iptnlRNidiDE+xkX2v
+pi4hBx28dDl3UA5HdjdYIgYS14rjAV6wifwwrJH2Wh6Sgo3rFC/WtpemUAYTTYLO
+Doma5QDjq0R8dwP1yvoEW2mtylo7BDuTTb9G3HeGDbVH/wuyZAMmLgoNJE0c2j1l
+Aa74qbPI55hB4d5cvuZq12g7/KPaV3Zo4qnDh8P1+Lvs2kBL6pthgcHJj+yS1aiB
+FaBtkiiFYhfUvjOEacpgjgrAjUsUTNqIz1PsGmBkXgZO/0LStVdFHBAUvunf0tx3
+WDcYIen+GPbEKAOQGUXLO1JjfWDW1h8QckPMc5hmWs3UOn7PFo+G/JiyKwARAQAB
+tCBBdHRpbGEgU3phYm8gPG1hdWdsaUBhcGFjaGUub3JnPokCVAQTAQgAPhYhBCpv
+jQ/vPatLm8Qk+Z/I2DIfaOobBQJZ+dDIAhsDBQkSzAMABQsJCAcCBhUICQoLAgQW
+AgMBAh4BAheAAAoJEJ/I2DIfaOob+3YP/1bj5ia3yT2moc7+SmASsI+tGiGewDCk
+vPeS4kuVbxOiK4XRwAFTPx9XOjMZYlN2erCfnla6Sn7U2Ip/nRSAVL16pr9G+Q/1
+8IuTme+gt+zgnkifMb5+JWYiiZhIYdsP17TuidVtGD2zNiBu7oQ/5aBUV7ImONAp
+qoFrdTznv0op8FFBSsuDoQYTSSV088e7cLXJoBvoIhj63Xc4GTEp9RWtDnB2Kk/s
++zLOJL64wXg28YMGwCtyut0CAqzRZi4ZXJm29rEOrfp/gVN9DXYGLLLJ0PbrNlWr
+zqbO55kGRzm7ck1MqN0wf2Z1eofShrI9MKXhZfFmz4H1HYKRTZW+3i+adIp747t5
+nVAYQUUMC4kSHK2yPIdaR6+jHdY5JohVoL2RvhaFTg2epo0TDbDTIKUrWwh0FBna
+RaF5uSAsKkenK23Xb8G2dRvH0RBin5y0JNWbxo3YtqwPRwI8usrKiI6VAWz39rE6
+5Pz2u3+vbwa9qZzeYQX1249eWBix5XFbLY0e2v5F+G7typVVGhDEQUabZJUBltcW
++H4ivBZcmHGmaxF3I2oyG22Ucp+yqf2sZuXySv1jLvpbRbfU3aGetALRcqh4D1Or
+nOPYNzXLuqctpJ2OkFkvWmxNnFDwm8Mj+vq943F/gAJAXiOCO67885d2RjoCY1EX
+e0ia6X0yIAqFuQINBFn50MgBEADRWctoi7cCL4QDuiRC4Epf25CqJjos062sjlKv
+FA8lCGIUoZct1riFxP5VkmZc2zeu1BtWHXae+n9xXdJBQ/5Xz2Dv6MX/LS7jZik8
+7hfe6rTyxHTtlMixqlxinbqVFUhcOr5WLqhirthk7/o7E7JdP/iwl6qdbBMjHEdv
+G5o72XeEr+LrPyM5CHpARjtlpT8tbgz6sviTCx8nnXidVlmiL7Stf5VpDPgjt7Cx
+rmm+fjQ/Zpi+8PcSyHXiUJGd4kJv3b9D/BvTcoXQrTjxTnWeE0Im2lthxMgwoEGd
+5ZDmMQvmXCNS96b/DpGor2/gjCnVkKgB4DheKt3w6NoaB0i+T1rWJCWmoO+Bo1f9
+tOjO5L8MlcBUEfMMd+ifkOFg2ARlEGPFT61B5fcnyAXCl1m4QyhM6MuET0JCenXD
+1XNDWnYqWFNMeVJxvbzQAO9oyaOa4pV/Dv1yLWdCSw4bE/JIjWfk+XqtDq7oZ/fn
+imBHkFHE8NAzKpNhojjzCBBy8hGCqSmFOv3E4Ta5qLtIp/x6gWiLCz+5LAdlNtIU
+zjUnLrm59hsjg5+20Al2ACRVvE+15b1z3bg8zKnGT5ANce51olRyquV0HcTi66jy
+ZVMQqikO5Oa2rV/hLBxdnvnYwe3I5W6mU2MXCd8O+3nxQA70n/6hU+lebZmIdM73
++OYocQARAQABiQI8BBgBCAAmFiEEKm+ND+89q0ubxCT5n8jYMh9o6hsFAln50MgC
+GwwFCRLMAwAACgkQn8jYMh9o6huvqxAAndEAa9GHvXKjS3rZsyAUYmZqKJbSe0ax
+8fQfa0rLYYt2P5LDGywWM6+ruQcTeUreSsfclclpQyFyFLR1TOasIuTljoZmo/C8
+/xkGZLMwhSNqPtp6UFRxSBq+F09T+o5fLGYFwF2I63VnRi6Xzvk2ws1kFRx6flAM
+PBJaYW/w+YjHXugMOLoSdqJcXgtzKGCyn020uI6SMT8H73xdK5ya8WdPrN11RGSP
+Np3QeCnhVsXRdAssOZA0zHytbz99gTsztr7CboJxwaTmWTlv0XQ7ewEICpvqIRaY
+EYxsL7t9I80V+p3UlOLqWgpGZzkyt9TWnOfYmXs5njvr9MYAbbflVS3r6pzZfTIf
+BnPmOw9N4W2JHi8wRDgCEFrZH8w4+W4ypxV28G3in2DheyMFSD3hcId3qJM7tvic
+Obn+LGisVWj0iTBhxR851eI6MViEGANHrTgPLIFC0vnnHjqHaa9k35G9kRuhf8a2
+X2x4IHxikLWlhARLwItkuEKNPBC3ymjADS6Dipm6CTa4/b00OpsZRdQRITNfzOAc
+nXAoPwmTCFBRPjgHUBWIdiDhpJkyMTDKsdzlYUhToBQV9s9iSw8v2nrZ+8eUQS94
+QFYfJTvwUxZ/v8vDIFAXu1qU+dh4AA8MmzQ2e5h6W0s4QVj/XhvkWbyXiliKEsTZ
+IGEuXlTtLR8=
+=eWZc
+-END PGP PUBLIC KEY BLOCK-


ATTILA_SZABO_PGP.KEY
Description: application/iwork-keynote-sffkey

Re: SQOOP 1.4.7 Release by end of June

[Attila Szabó]

Hey Szabi,

Thanks for the reach out!

I've already seen emails about those commits, and I plan to cherry pick
them to the 1.4.6 release branch this evening.

Until:
Could you please add a comment about these changes on the umbrella JIRA?
(you could also open a subtask wit links if that's easier on your side).

The rest I'll handle.

Many thanks in advance,

Attila

<http://www.avg.com/email-signature?utm_medium=email_source=link_campaign=sig-email_content=webmail>
Virus-free.
www.avg.com
<http://www.avg.com/email-signature?utm_medium=email_source=link_campaign=sig-email_content=webmail>
<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

On Tue, Jun 20, 2017 at 4:11 PM, Szabolcs Vasas <va...@cloudera.com> wrote:

> Hi Attila,
>
> Thank you for creating the branch, I have just one comment.
> It turned out that some third party tests were failing on trunk branch so I
> submitted 2 patches (https://issues.apache.org/jira/browse/SQOOP-3194 ,
> https://issues.apache.org/jira/browse/SQOOP-3198) which were committed
> today. Shall we include these changes in the release too?
>
> Regards,
> Szabolcs
>
> On Fri, Jun 16, 2017 at 9:18 AM, Attila Szabó <mau...@apache.org> wrote:
>
> > Dear community,
> >
> > First of all many thanks for the valuable feedbacks. As agreed the
> > releasing process had been started.
> >
> > We do have an umbrella JIRA for this:
> > https://issues.apache.org/jira/browse/SQOOP-3199
> >
> > All feedbacks, comments more than welcome here.
> >
> > A short status:
> > As you can see sanitize, branching, changelog, doc+version updates is
> > already done.
> >
> > To finalize the scope of 1.4.7 I'd like to kindly ask all of you to check
> > the CHANGELOG.txt (release notes) and check if every JIRA ticket is there
> > what is vital on your side for 1.4.7.
> >
> > According to the current plans/status:
> > all of the issues which are not fixed (TODO, in progress) right now, and
> > connected to 1.4.7 (please check
> > https://issues.apache.org/jira/projects/SQOOP/versions/12329683 for
> > details) are meant to move to 1.5.0 unless there is any specific reason
> to
> > include in 1.4.7 and thus changing the scope/delaying the process.
> >
> > The content of 1.4.7 branch should be changed only in sync with the
> > umbrella JIRA, and all new commits on the trunk from now is considered as
> > part of 1.5.0.
> >
> > In case of any questions, suggestions, change requests please use this
> mail
> > thread or the umbrella JIRA.
> >
> > I'll keep you updated with the progress of the release.
> >
> > Best regards,
> > Attila
> >
> >
> > On Wed, May 31, 2017 at 11:18 AM, Boglarka Egyed <b...@cloudera.com>
> > wrote:
> >
> > > Hi Attila,
> > >
> > > +1 to the idea and many thanks for driving this!
> > >
> > > I don't have any specific requirement or concern so feel free to
> proceed
> > > with the proposed timeline.
> > >
> > > Thanks,
> > > Bogi
> > >
> > > On Wed, May 31, 2017 at 4:51 AM, Ying Cao <angela.caoy...@gmail.com>
> > > wrote:
> > >
> > >> Hi Attila,
> > >>
> > >> This is really great,  +1 to the idea !
> > >>
> > >> 2017-05-30 23:40 GMT+08:00 Szabolcs Vasas <va...@cloudera.com>:
> > >>
> > >> > Hi Attila,
> > >> >
> > >> > Thank you for taking the initiative, big +1 to the idea!
> > >> >
> > >> > Regards,
> > >> > Szabolcs
> > >> >
> > >> > On Tue, May 30, 2017 at 1:40 PM, Anna Szonyi <szo...@cloudera.com>
> > >> wrote:
> > >> >
> > >> > > Hi Attila,
> > >> > >
> > >> > > This would be really great! Please let me know if I can help with
> > >> > anything!
> > >> > >
> > >> > > Thanks,
> > >> > > Anna
> > >> > >
> > >> > > On Thu, May 25, 2017 at 6:20 PM, Attila Szabó <mau...@apache.org>
> > >> wrote:
> > >> > >
> > >> > > > Hello everyone,
> > >> > > >
> > >> > > > In the past few months we've mentioned it a few times that it
> > would
> > >> > make
> > >> > > > sense to create a new release of Sqoop from the current trunk
> > >> version
> > >> > > > (1.4.7).
> > >> > > >
> >

Re: SQOOP 1.4.7 Release by end of June

[Attila Szabó]

Dear community,

First of all many thanks for the valuable feedbacks. As agreed the
releasing process had been started.

We do have an umbrella JIRA for this:
https://issues.apache.org/jira/browse/SQOOP-3199

All feedbacks, comments more than welcome here.

A short status:
As you can see sanitize, branching, changelog, doc+version updates is
already done.

To finalize the scope of 1.4.7 I'd like to kindly ask all of you to check
the CHANGELOG.txt (release notes) and check if every JIRA ticket is there
what is vital on your side for 1.4.7.

According to the current plans/status:
all of the issues which are not fixed (TODO, in progress) right now, and
connected to 1.4.7 (please check
https://issues.apache.org/jira/projects/SQOOP/versions/12329683 for
details) are meant to move to 1.5.0 unless there is any specific reason to
include in 1.4.7 and thus changing the scope/delaying the process.

The content of 1.4.7 branch should be changed only in sync with the
umbrella JIRA, and all new commits on the trunk from now is considered as
part of 1.5.0.

In case of any questions, suggestions, change requests please use this mail
thread or the umbrella JIRA.

I'll keep you updated with the progress of the release.

Best regards,
Attila


On Wed, May 31, 2017 at 11:18 AM, Boglarka Egyed <b...@cloudera.com> wrote:

> Hi Attila,
>
> +1 to the idea and many thanks for driving this!
>
> I don't have any specific requirement or concern so feel free to proceed
> with the proposed timeline.
>
> Thanks,
> Bogi
>
> On Wed, May 31, 2017 at 4:51 AM, Ying Cao <angela.caoy...@gmail.com>
> wrote:
>
>> Hi Attila,
>>
>> This is really great,  +1 to the idea !
>>
>> 2017-05-30 23:40 GMT+08:00 Szabolcs Vasas <va...@cloudera.com>:
>>
>> > Hi Attila,
>> >
>> > Thank you for taking the initiative, big +1 to the idea!
>> >
>> > Regards,
>> > Szabolcs
>> >
>> > On Tue, May 30, 2017 at 1:40 PM, Anna Szonyi <szo...@cloudera.com>
>> wrote:
>> >
>> > > Hi Attila,
>> > >
>> > > This would be really great! Please let me know if I can help with
>> > anything!
>> > >
>> > > Thanks,
>> > > Anna
>> > >
>> > > On Thu, May 25, 2017 at 6:20 PM, Attila Szabó <mau...@apache.org>
>> wrote:
>> > >
>> > > > Hello everyone,
>> > > >
>> > > > In the past few months we've mentioned it a few times that it would
>> > make
>> > > > sense to create a new release of Sqoop from the current trunk
>> version
>> > > > (1.4.7).
>> > > >
>> > > > IMHO this is the right time to move forward with this topic.
>> > > > First of all because the latest release is 2 years old (1.4.6 was
>> > > released
>> > > > in 2015 May), and it would make a great impact for the end users to
>> > > deliver
>> > > > all of the changes we as a community have made since.
>> > > > As a second argument, recently we were able to onboard quite a few
>> new
>> > > > contributor (including but not limited to Illya Yalovyy, Ying Cao,
>> > Eric
>> > > > Lin, Chris Teoh, Dmitry Zagorulkin), and as a recognition of their
>> > work,
>> > > > I'd like to ensure their commits gets into a release as soon as
>> > possible.
>> > > > Third: recently Anna, Bogi and Szabi were super active on the front
>> of
>> > > > stabilizing and improving the tests, the build system and so,
>> however
>> > > > according to my feelings the current status quo (waiting for a minor
>> > > > version change with bigger changes) holding them back from more
>> > > fundamental
>> > > > improvements (I guess this is the reason why the Gradle/Maven new
>> build
>> > > > sytem is still in progress, and the conversation around feature
>> > branches
>> > > > started). I think it would make a great impact for the whole
>> community
>> > > and
>> > > > the product itself, if we would be able to remove this burden from
>> > their
>> > > > shoulders, and thus they would be able to focus on bigger
>> > > > changes/challenges.
>> > > >
>> > > > Because of the reasons mentioned above, on the top of my existing
>> > duties,
>> > > > I'd like to help the community's life by owning the 1.4.7 release
>> > > process,
>> > > > and starting the release with branching the trunk as of 2017-06-01
>> > (next
>> > > > Friday) PST 8PM, and finishing the whole process latest by the end
>> of
>> > > June.
>> > > >
>> > > > Of course before starting even creating the related JIRA tasks,
>> plans,
>> > > the
>> > > > branching, etc. I'd like to collect the inputs from all of you if
>> this
>> > > fits
>> > > > your plan/requirements/etc.
>> > > >
>> > > > Looking forward hearing all of your thoughts, concerns, pros and
>> cons.
>> > > >
>> > > > Best regards,
>> > > > Attila Szabo
>> > > >
>> > >
>> >
>> >
>> >
>> > --
>> > Szabolcs Vasas
>> > Software Engineer
>> > <http://www.cloudera.com>
>> >
>>
>
>

Re: Creating feature branches

[Attila Szabó]

Hey Bogi,

Thanks for raising this questions. I've just opened a new topic around this.
It would be great if you could share your thoughts there.

Many thanks,
Attila

On Tue, May 23, 2017 at 2:41 PM, Boglarka Egyed <b...@cloudera.com> wrote:

> Hi Attila,
>
> You mentioned that you have a plan for releasing 1.4.7, could you please
> give an update on that?
>
> Many thanks,
> Bogi
>
> On Tue, May 16, 2017 at 10:35 AM, Szabolcs Vasas <va...@cloudera.com>
> wrote:
>
>> Hi Attila,
>>
>> I don't have a list of JIRAs yet this was just an idea we can implement in
>> the future.
>>
>> Szabolcs
>>
>> On Mon, May 15, 2017 at 6:23 PM, Attila Szabó <mau...@apache.org> wrote:
>>
>> > Hey Szabi:
>> > Do you already have a plan ( I mean a list of JIRA issues for your
>> first
>> > feature branch(es))
>> >
>> > Thanks,
>> > Attila
>> >
>> > On May 15, 2017 3:15 PM, "Szabolcs Vasas" <va...@cloudera.com> wrote:
>> >
>> > > Hi all,
>> > >
>> > > I also like the idea of creating feature branches. As far as I see we
>> > > cannot create Sqoop epics on issues.apache.org so we could use
>> subtasks
>> > to
>> > > track what do we plan to put in a feature branch.
>> > >
>> > > Regards,
>> > > Szabolcs
>> > >
>> > > On Fri, May 12, 2017 at 2:15 PM, Attila Szabó <mau...@apache.org>
>> wrote:
>> > >
>> > > > Hey Anna,
>> > > >
>> > > >
>> > > > On my side: +1 if it helps your development efforts, and if the
>> feature
>> > > > branch won't be a "long living" one. (and of course you would merge
>> it
>> > > with
>> > > > the trunk frequently not to let it diverge too much from other
>> > changes).
>> > > >
>> > > >
>> > > > Related actions (by me):
>> > > >
>> > > > I would like to support your efforts on this side by finalizing the
>> > 1.4.7
>> > > > release dates with the community soon (I'd like to send an email
>> about
>> > > > those efforts still today).
>> > > >
>> > > >
>> > > > On request:
>> > > >
>> > > > Would you please enlist the JIRA tasks you'd like to work on
>> related to
>> > > > that specific feature branch (thus the whole community would be
>> able to
>> > > see
>> > > > the scope of the features you'd working on).
>> > > >
>> > > >
>> > > > Cheers,
>> > > >
>> > > > Attila
>> > > >
>> > > > On Fri, May 12, 2017 at 11:58 AM, Anna Szonyi <szo...@cloudera.com>
>> > > wrote:
>> > > >
>> > > > > Hi @devs,
>> > > > >
>> > > > > We would like to start the preparation for upgrading to hive 2
>> > (upgrade
>> > > > > version, migrate off of hive cli and onto beeline, etc.).
>> > > > > As we anticipate there being a few patches we were thinking it
>> would
>> > > make
>> > > > > sense to create a feature branch (sqoop_hive2) for this, as we do
>> not
>> > > > want
>> > > > > trunk to become unstable or introduce "breaking changes" before
>> the
>> > > > > proposed 1.4.7 release.
>> > > > >
>> > > > > Does anyone have any concerns about this?
>> > > > >
>> > > > > And in general do we have a policy about feature branches for
>> sqoop
>> > > > (like a
>> > > > > sqoop_kerberos branch, etc.)?
>> > > > >
>> > > > > Thanks,
>> > > > > Anna
>> > > > >
>> > > >
>> > >
>> > >
>> > >
>> > > --
>> > > Szabolcs Vasas
>> > > Software Engineer
>> > > <http://www.cloudera.com>
>> > >
>> >
>>
>>
>>
>> --
>> Szabolcs Vasas
>> Software Engineer
>> <http://www.cloudera.com>
>>
>
>

SQOOP 1.4.7 Release by end of June

[Attila Szabó]

Hello everyone,

In the past few months we've mentioned it a few times that it would make
sense to create a new release of Sqoop from the current trunk version
(1.4.7).

IMHO this is the right time to move forward with this topic.
First of all because the latest release is 2 years old (1.4.6 was released
in 2015 May), and it would make a great impact for the end users to deliver
all of the changes we as a community have made since.
As a second argument, recently we were able to onboard quite a few new
contributor (including but not limited to Illya Yalovyy, Ying Cao,  Eric
Lin, Chris Teoh, Dmitry Zagorulkin), and as a recognition of their work,
I'd like to ensure their commits gets into a release as soon as possible.
Third: recently Anna, Bogi and Szabi were super active on the front of
stabilizing and improving the tests, the build system and so, however
according to my feelings the current status quo (waiting for a minor
version change with bigger changes) holding them back from more fundamental
improvements (I guess this is the reason why the Gradle/Maven new build
sytem is still in progress, and the conversation around feature branches
started). I think it would make a great impact for the whole community and
the product itself, if we would be able to remove this burden from their
shoulders, and thus they would be able to focus on bigger
changes/challenges.

Because of the reasons mentioned above, on the top of my existing duties,
I'd like to help the community's life by owning the 1.4.7 release process,
and starting the release with branching the trunk as of 2017-06-01 (next
Friday) PST 8PM, and finishing the whole process latest by the end of June.

Of course before starting even creating the related JIRA tasks, plans, the
branching, etc. I'd like to collect the inputs from all of you if this fits
your plan/requirements/etc.

Looking forward hearing all of your thoughts, concerns, pros and cons.

Best regards,
Attila Szabo

Re: Creating feature branches

[Attila Szabó]

Hey Szabi:
Do you already have a plan ( I mean a list of JIRA issues for your  first
feature branch(es))

Thanks,
Attila

On May 15, 2017 3:15 PM, "Szabolcs Vasas" <va...@cloudera.com> wrote:

> Hi all,
>
> I also like the idea of creating feature branches. As far as I see we
> cannot create Sqoop epics on issues.apache.org so we could use subtasks to
> track what do we plan to put in a feature branch.
>
> Regards,
> Szabolcs
>
> On Fri, May 12, 2017 at 2:15 PM, Attila Szabó <mau...@apache.org> wrote:
>
> > Hey Anna,
> >
> >
> > On my side: +1 if it helps your development efforts, and if the feature
> > branch won't be a "long living" one. (and of course you would merge it
> with
> > the trunk frequently not to let it diverge too much from other changes).
> >
> >
> > Related actions (by me):
> >
> > I would like to support your efforts on this side by finalizing the 1.4.7
> > release dates with the community soon (I'd like to send an email about
> > those efforts still today).
> >
> >
> > On request:
> >
> > Would you please enlist the JIRA tasks you'd like to work on related to
> > that specific feature branch (thus the whole community would be able to
> see
> > the scope of the features you'd working on).
> >
> >
> > Cheers,
> >
> > Attila
> >
> > On Fri, May 12, 2017 at 11:58 AM, Anna Szonyi <szo...@cloudera.com>
> wrote:
> >
> > > Hi @devs,
> > >
> > > We would like to start the preparation for upgrading to hive 2 (upgrade
> > > version, migrate off of hive cli and onto beeline, etc.).
> > > As we anticipate there being a few patches we were thinking it would
> make
> > > sense to create a feature branch (sqoop_hive2) for this, as we do not
> > want
> > > trunk to become unstable or introduce "breaking changes" before the
> > > proposed 1.4.7 release.
> > >
> > > Does anyone have any concerns about this?
> > >
> > > And in general do we have a policy about feature branches for sqoop
> > (like a
> > > sqoop_kerberos branch, etc.)?
> > >
> > > Thanks,
> > > Anna
> > >
> >
>
>
>
> --
> Szabolcs Vasas
> Software Engineer
> <http://www.cloudera.com>
>

Re: Creating feature branches

[Attila Szabó]

Hey Anna,


On my side: +1 if it helps your development efforts, and if the feature
branch won't be a "long living" one. (and of course you would merge it with
the trunk frequently not to let it diverge too much from other changes).


Related actions (by me):

I would like to support your efforts on this side by finalizing the 1.4.7
release dates with the community soon (I'd like to send an email about
those efforts still today).


On request:

Would you please enlist the JIRA tasks you'd like to work on related to
that specific feature branch (thus the whole community would be able to see
the scope of the features you'd working on).


Cheers,

Attila

On Fri, May 12, 2017 at 11:58 AM, Anna Szonyi  wrote:

> Hi @devs,
>
> We would like to start the preparation for upgrading to hive 2 (upgrade
> version, migrate off of hive cli and onto beeline, etc.).
> As we anticipate there being a few patches we were thinking it would make
> sense to create a feature branch (sqoop_hive2) for this, as we do not want
> trunk to become unstable or introduce "breaking changes" before the
> proposed 1.4.7 release.
>
> Does anyone have any concerns about this?
>
> And in general do we have a policy about feature branches for sqoop (like a
> sqoop_kerberos branch, etc.)?
>
> Thanks,
> Anna
>

Re: Running 3rd party test on the CI

[Attila Szabó]

Hello everyone,

My arguments in this topic are the following:

Having reliable ( ~ 24/7 online, of course not with five 9 constraints )
external database resources sounds great, would solve most of the
problems/concerns right now we do see about making 3rd party tests
automated. If this is the best practice in the Kudu team and we could be
connected to the same donating companies than it's instantly a +2 on my
side. The two drawbacks/concerns I do find here are:
- can we depend on these resources in long term ( ~ years ), and what
should we / the build do if by any reason some of the resources are not
available?
- if the resources are only available for the CI system how will we
handle/avoid the "works for me" situations. If available for all of the
contributors how to avoid the DBs become messy?

On the docker side:
It's advantage is only it's flexibility. If the CI host machine contains
enough resources to fire up a DB instance ( or even all, depends on how do
we group/organize the testing of each db server ), then we could be sure
about that the success of the execution won't depend on any configuration,
network, etc. Issues. ( and even count on having a clean db at every start
). But in this case the contributor setup could have 0 difference compared
to the CI setup. Of course it's drawback is the very same e.g. meaningful
performance testing shouldn't be done in a Docker environment ( according
to my humblest oppinion ), whilst with donated bare metal resources it
would be also achievable.

On my side I would be happy with both of the solution, the only reason why
I would favor Docker based testing is the configuration identity. But this
is an argument I could easily let go.

The only strong argument I do have in this topic is the following:
If possible I would like to have one single and clean working solution, and
not the mixture of any of the ideas listed ( above or later ). Thus we
could avoid several flakey and configuration based testing issues.

Cheers,
Attila

PS: Many thanks for Anna bringing this topic to the dev list!




On Apr 10, 2017 1:50 PM, "Anna Szonyi"  wrote:

Hi All,

We started a discussion about running thirdparty integration tests in our
CI on separate threads (another e-mail chain and a review), and I thought
it might make sense to bring these threads together.

We were discussing the best way of running the thirdparty tests on the
apache CI, as we think it would be very beneficial.

We were floating around the idea of trying to get resources (mysql, cubrid,
etc.) "donated", if that's possible (as far as I understand that's more or
less how Kudu deals with the same issue).

Attila proposed to investigate using docker images.

I would like to bring this up on the dev list - in case anyone has any
experience with this (using external resources from apache CIs), or anyone
has any concerns/negative experiences.

Please chime in here if you have any ideas/workable solutions for this or
any objections.

Thanks,
Anna

Re: Sqoop CI changes

[Attila Szabó]

Hey Anna,

Thanks for your quick reply!

I'm not sure if I can follow you, on the Kudu team topic, but if that means
either we'd like to include some of their working solutions in our CI
system, or we'd like to do a working Kudu integration, for we need better
CI, I would say +1 for both of them, and would be more than happy to help
your efforts on that front. :-)

My original idea briefly would be for the 3rd party automation:
- As a step 0 fire up docker containers with the related DB dependencies
before the whole cycle or before a good organized group of tests (e.g. one
group of MySQL, Oracle, etc.)
- After the tests executed terminate the containers.

Other solution would be possible like:
- Somehow getting access and resources from the Apache community to have
dedicated/global CI DB servers for Sqoop
- Ask any of the Hadoop vendors or the Apache sponsors to provide us pre
installed server infrastructure
- Acquire server instances and deploy them with Terraform+Ansible, etc.

The problem with this "second approach" is that it would need external
resources involved and we would need someone who would sponsor our HW
resources. On the other hand the benefits would be that on real bare metal
resources we would be able to perform meaningful performance tests in the
future as well.

The original plan:
I have no concerns, but still would like to highlight, that IMHO only the
pre commit hook would be urgent, and the rest of your energies would be
better focused on finishing the new build system, or rather on the solution
3rd party resource problem we've discussed above.

And as always: if you would need my help in any of your efforts please do
not hesitate to ping me here, or on the related JIRA task.

My 2cents,
Attila

On Fri, Apr 7, 2017 at 8:34 PM, Anna Szonyi <szo...@cloudera.com> wrote:

> Hey Attila,
>
> Thanks for your input! I agree that adding the 3rd party tests to the CI
> would be really beneficial. As there are some resourcing problems that need
> to be solved for that to happen (I started a discussion with the Kudu team,
> however if you have any specific help to add there, I would be very happy
> to accept any assistance with that) - I'll start a separate thread where we
> can discuss this issue with the community, see if anyone else has any
> inputs on it :).
>
> However if no one has any objections about the original proposal, I will
> follow through with that in the meantime.
>
> Thanks and Regards,
> Anna
>
>
> On Fri, Apr 7, 2017 at 1:04 PM, Attila Szabó <mau...@apache.org> wrote:
>
> > Hello everyone,
> >
> > First of all I'd like to thank that Anna is willing to invest some
> efforts
> > on making our CI system better, and sorry for my delayed answer (
> however I
> > do still hope it helps regardless the stage of the ongoing efforts )
> >
> > I'd like to share the following thoughts here:
> > - Although it would make sense to eliminate the four ( right now totally
> > equal ) CI cycles and create only one, but regardless some "static noise"
> > it doesn't cause any serious issues for our current commit flow.
> > - Creating a precommit hook sounds like a great idea, I would encourage
> the
> > community to move on that path.
> > - However: According to my humblest opinion the biggest problem with the
> > current CI is that it doesn't execute the so called " 3rd party tests" (
> > which is generally our DB integration test layer), and thus it provides
> > only a limited safety belt for us ( and we've seen quite a few regression
> > on this front in the past one year ). Although we do have solution for
> > running those tests manually from command line, it's quite difficult to
> > setup/test those things from a single desktop, thus cause serious
> > difficulties in validating some changeset before commit.
> > - Still an issue we have on this front is our build
> > system/scripts/mechanism, which again could slow down the Dev+commit
> flow.
> > Although we've started efforts on this front, the final solution was not
> > fully delivered yet.
> >
> > As a conclusion of the elements above:
> > Anna! Would you mind first focusing on the build scripts and the "3rd
> > party" CI automation instead of eliminating the obsolete stuff? IMHO that
> > would be a much better usage of your efforts and would provide a much
> > bigger impact for the community.
> >
> > With my kindest regards,
> > Attila
> >
> > On Mar 28, 2017 2:44 PM, "Erzsebet Szilagyi" <liz.szila...@cloudera.com>
> > wrote:
> >
> > > Great ideas!
> > >
> > > I agree with Bogi and Szabolcs on the redundant test jobs.
> > >
> > > Woul

Re: Sqoop CI changes

[Attila Szabó]

Hello everyone,

First of all I'd like to thank that Anna is willing to invest some efforts
on making our CI system better, and sorry for my delayed answer ( however I
do still hope it helps regardless the stage of the ongoing efforts )

I'd like to share the following thoughts here:
- Although it would make sense to eliminate the four ( right now totally
equal ) CI cycles and create only one, but regardless some "static noise"
it doesn't cause any serious issues for our current commit flow.
- Creating a precommit hook sounds like a great idea, I would encourage the
community to move on that path.
- However: According to my humblest opinion the biggest problem with the
current CI is that it doesn't execute the so called " 3rd party tests" (
which is generally our DB integration test layer), and thus it provides
only a limited safety belt for us ( and we've seen quite a few regression
on this front in the past one year ). Although we do have solution for
running those tests manually from command line, it's quite difficult to
setup/test those things from a single desktop, thus cause serious
difficulties in validating some changeset before commit.
- Still an issue we have on this front is our build
system/scripts/mechanism, which again could slow down the Dev+commit flow.
Although we've started efforts on this front, the final solution was not
fully delivered yet.

As a conclusion of the elements above:
Anna! Would you mind first focusing on the build scripts and the "3rd
party" CI automation instead of eliminating the obsolete stuff? IMHO that
would be a much better usage of your efforts and would provide a much
bigger impact for the community.

With my kindest regards,
Attila

On Mar 28, 2017 2:44 PM, "Erzsebet Szilagyi" 
wrote:

> Great ideas!
>
> I agree with Bogi and Szabolcs on the redundant test jobs.
>
> Would this pre-commit hook launch the same process as the current
> post-commit hook, or would this do something different?
> I think in the first case we could rework the post-commit check into the
> pre-commit hook, in the latter I'm curious about what exactly this check
> would add.
> In general I support the idea: we have seen a number of problems that could
> have been avoided, so this shall be a very useful change!
>
> Thank you,
> Liz
>
> On Fri, Mar 24, 2017 at 9:39 AM, Szabolcs Vasas 
> wrote:
>
> > Hi Anna,
> >
> > Removing the redundant test execution jobs sounds great, I think you can
> go
> > ahead with that.
> >
> > Regarding the pre-commit hook: what would be the purpose of it exactly?
> > Would it execute the unit tests before the patch is committed?
> >
> > Regards,
> > Szabolcs
> >
> > On Thu, Mar 23, 2017 at 4:03 PM, Anna Szonyi 
> wrote:
> >
> > > Hi All,
> > >
> > > I would like to make the following changes to the Sqoop CI system:
> > > Disable the SCM polling for the Sqoop-hadoop23 Sqoop-hadoop20 and
> > > Sqoop-hadoop100 jobs (and later delete the jobs themselves),
> > > as the current trunk version of sqoop no longer contains these
> profiles,
> > so
> > > these runs are redundant.
> > >
> > > I would also like to propose the creation of a pre-commit hook for
> Sqoop
> > > (like the existing one for Sqoop2).
> > >
> > > Please let me know if you have any objections.
> > >
> > > Thanks,
> > > Anna
> > >
> >
> >
> >
> > --
> > Szabolcs Vasas
> > Software Engineer
> > 
> >
>

Re: New Sqoop Committer - Anna Szonyi

[Attila Szabó]

Nice job Anna!

 Very much well deserved!!!

Congrats,

Attila

On Mar 16, 2017 7:47 PM, "Abraham Fine"  wrote:

> On behalf of the Apache Sqoop PMC, I am pleased to welcome Anna Szonyi
> as a new committer to Apache Sqoop. Please join me in congratulating her
> for this accomplishment!
>
> Anna has made a number of contributions to the Sqoop test and build code
> in addition to many bug fixes. Anna has also contributed to our
> community by performing code reviews and helping users on our mailing
> list. You can see many of her contributions here:
> https://s.apache.org/PfGo
>
> We are excited too see what Anna contributes next!
>
> Thanks,
> Abraham Fine
>

Re: Secure sqoop transfer

[Attila Szabó]

Hey Sumit,

Thanks for the information. Most probably I've phrased my words in a
confusing way.

I've meant that according to my knowledge there's no single out of the box
free/open source solution for this (just a good example for that the
properties+settings differs from RBMS to RBMDS JDBC to JDBC, etc.).
Of course it would be possible to create such a tool, but IMHO it would be
out of the scope of Sqoop.

Thanks again Sumit for your additions!

Cheers,
Attila

On Thu, Mar 16, 2017 at 3:55 PM, Sumit Sarkar <sumit.sar...@progress.com>
wrote:

> For option B, there are a couple generic solutions available form my
> employer.
>
> 1) Generic Hybrid JDBC Connectors
> <https://www.progress.com/cloud-and-hybrid-data-integration> encrypts
> data over HTTPS and can run locally or hybrid and 2) SequeLink JDBC-JDBC
> Socket <https://www.progress.com/connectors/sequelink> encrypts using
> TCP/IP. We use option #1 to load our internal data lake via Sqoop and
> sample scripts are here
> <https://www.progress.com/blogs/bulk-data-movement-to-sql-databases-in-hybrid-cloud>
> for a hybrid example but works similar if completely run on-premises.
>
> From: Attila Szabó <mau...@apache.org>
> Reply-To: "u...@sqoop.apache.org" <u...@sqoop.apache.org>
> Date: Tuesday, March 14, 2017 at 6:30 PM
> To: "dev@sqoop.apache.org" <dev@sqoop.apache.org>
> Cc: "u...@sqoop.apache.org" <u...@sqoop.apache.org>
> Subject: Re: Secure sqoop transfer
>
> Hey Jagrut,
>
> Do you have a specific RDBMS you'd like to use, or would you like to do it
> in a generic way?
>
> If A. You should check the specific JDBC driver for the driver/transport
> level encryption options/properties, and feed it to the env execution your
> Sqoop command (Sqoop itself right now cannot, and IMHO should not implement
> it's own encryption solutions, as it has to happen on JDBC level).
> if B. AFAIK there is no general solution for this in case of JDBC
> supported Databases, but you could try to setup a secure + encrypted VPN
> network, which will solve this to you out of the box.
>
> My 2 cents,
> Attila
>
>
> On Tue, Mar 14, 2017 at 6:22 PM, Jagrut Sharma <jagrutsha...@gmail.com>
> wrote:
>
>> I'm trying to figure out if there is a way to do encrypted data transfer
>> between the source rdbms system and cluster when importing data via sqoop
>> (version 1.4.6). Any pointers would be helpful.
>>
>> Thanks!
>>
>> --
>> Jagrut
>>
>
>

Re: Secure sqoop transfer

[Attila Szabó]

Hey Jagrut,

Do you have a specific RDBMS you'd like to use, or would you like to do it
in a generic way?

If A. You should check the specific JDBC driver for the driver/transport
level encryption options/properties, and feed it to the env execution your
Sqoop command (Sqoop itself right now cannot, and IMHO should not implement
it's own encryption solutions, as it has to happen on JDBC level).
if B. AFAIK there is no general solution for this in case of JDBC supported
Databases, but you could try to setup a secure + encrypted VPN network,
which will solve this to you out of the box.

My 2 cents,
Attila


On Tue, Mar 14, 2017 at 6:22 PM, Jagrut Sharma 
wrote:

> I'm trying to figure out if there is a way to do encrypted data transfer
> between the source rdbms system and cluster when importing data via sqoop
> (version 1.4.6). Any pointers would be helpful.
>
> Thanks!
>
> --
> Jagrut
>

Re: Support object store

[Attila Szabó]

Hey Angela,

AFAIK if you connect your S3 bucket under HDFS then it works out of the box.
Does this solution fit your requirements or are you looking for something
different/more specific?

Cheers,
Attila

On Tue, Mar 14, 2017 at 2:59 AM, Ying Cao  wrote:

> Hi Expert,
>
> Does Sqoop support object store? For example Amazon S3?
>
> Angela
>