Re: What is Jenkins job "Portable_Python" in PreCommit?

[Mark Liu]

Thank you for improving the test coverage. That's very helpful!

On the other hand, I have a PR  in
review to make time-consuming integration tests to run parallel in
PreCommit. This could benefits tests like portable_python to be enabled in
`:pythonPreCommit` without creating a Jenkins branch and run seed job for
testing. However, if we have other reasons, then it may not applied.

Would love to hear thoughts on this.

Mark

On Fri, Nov 30, 2018 at 12:27 PM Thomas Weise  wrote:

> This is a very valuable addition! Given the execution times of Python and
> Java pre-commit, having this run in parallel should not be an issue.
>
> Thanks,
> Thomas
>
>
> On Fri, Nov 30, 2018 at 11:01 AM Ankur Goenka  wrote:
>
>> We added new precommit which tests a wordcount pipeline on portable flink
>> runner.
>> In recent time we missed catching some obvious issues which broke
>> potability and could have been caught by this test.
>> The current test is fairly light weight and executes in ~5min which seems
>> to be reasonable for a precommit test.
>> This precommit is added just yesterday so we don't actively track it yet.
>> However please let me know if you see any issues with this precommit.
>>
>> On Fri, Nov 30, 2018 at 3:49 AM Maximilian Michels 
>> wrote:
>>
>>> This was merged with https://github.com/apache/beam/pull/6954.
>>>
>>> Eventually we want to run a portable WordCount on PreCommit. We will do
>>> some more testing on Jenkins before it becomes an official PreCommit
>>> task.
>>>
>>> Thanks,
>>> Max
>>>
>>> On 29.11.18 19:03, Mark Liu wrote:
>>> > ah, thanks Boyuan! Probably I created the PR in a bad timing. Looks
>>> like
>>> > a new PR will fix it.
>>> >
>>> > On Thu, Nov 29, 2018 at 9:50 AM Boyuan Zhang >> > > wrote:
>>> >
>>> > I believe it's in this pending PR:
>>> > https://github.com/apache/beam/pull/7157.
>>> >
>>> > On Thu, Nov 29, 2018 at 8:36 AM Mark Liu >> > > wrote:
>>> >
>>> > Hi guys,
>>> >
>>> > I made some changes 
>>> > to Python PreCommit Gradle and then Portable_Python is invoked
>>> > as a PrecCommit test and failed. However, I can't find where
>>> > it's defined / generated in Gradle or Jenkins groovy. Does
>>> > anyone know? My branch is synced to master yesterday.
>>> >
>>> > Thanks!
>>> > Mark
>>> >
>>>
>>

Re: What is Jenkins job "Portable_Python" in PreCommit?

[Thomas Weise]

This is a very valuable addition! Given the execution times of Python and
Java pre-commit, having this run in parallel should not be an issue.

Thanks,
Thomas


On Fri, Nov 30, 2018 at 11:01 AM Ankur Goenka  wrote:

> We added new precommit which tests a wordcount pipeline on portable flink
> runner.
> In recent time we missed catching some obvious issues which broke
> potability and could have been caught by this test.
> The current test is fairly light weight and executes in ~5min which seems
> to be reasonable for a precommit test.
> This precommit is added just yesterday so we don't actively track it yet.
> However please let me know if you see any issues with this precommit.
>
> On Fri, Nov 30, 2018 at 3:49 AM Maximilian Michels  wrote:
>
>> This was merged with https://github.com/apache/beam/pull/6954.
>>
>> Eventually we want to run a portable WordCount on PreCommit. We will do
>> some more testing on Jenkins before it becomes an official PreCommit task.
>>
>> Thanks,
>> Max
>>
>> On 29.11.18 19:03, Mark Liu wrote:
>> > ah, thanks Boyuan! Probably I created the PR in a bad timing. Looks
>> like
>> > a new PR will fix it.
>> >
>> > On Thu, Nov 29, 2018 at 9:50 AM Boyuan Zhang > > > wrote:
>> >
>> > I believe it's in this pending PR:
>> > https://github.com/apache/beam/pull/7157.
>> >
>> > On Thu, Nov 29, 2018 at 8:36 AM Mark Liu > > > wrote:
>> >
>> > Hi guys,
>> >
>> > I made some changes 
>> > to Python PreCommit Gradle and then Portable_Python is invoked
>> > as a PrecCommit test and failed. However, I can't find where
>> > it's defined / generated in Gradle or Jenkins groovy. Does
>> > anyone know? My branch is synced to master yesterday.
>> >
>> > Thanks!
>> > Mark
>> >
>>
>

Re: What is Jenkins job "Portable_Python" in PreCommit?

[Ankur Goenka]

We added new precommit which tests a wordcount pipeline on portable flink
runner.
In recent time we missed catching some obvious issues which broke
potability and could have been caught by this test.
The current test is fairly light weight and executes in ~5min which seems
to be reasonable for a precommit test.
This precommit is added just yesterday so we don't actively track it yet.
However please let me know if you see any issues with this precommit.

On Fri, Nov 30, 2018 at 3:49 AM Maximilian Michels  wrote:

> This was merged with https://github.com/apache/beam/pull/6954.
>
> Eventually we want to run a portable WordCount on PreCommit. We will do
> some more testing on Jenkins before it becomes an official PreCommit task.
>
> Thanks,
> Max
>
> On 29.11.18 19:03, Mark Liu wrote:
> > ah, thanks Boyuan! Probably I created the PR in a bad timing. Looks like
> > a new PR will fix it.
> >
> > On Thu, Nov 29, 2018 at 9:50 AM Boyuan Zhang  > > wrote:
> >
> > I believe it's in this pending PR:
> > https://github.com/apache/beam/pull/7157.
> >
> > On Thu, Nov 29, 2018 at 8:36 AM Mark Liu  > > wrote:
> >
> > Hi guys,
> >
> > I made some changes 
> > to Python PreCommit Gradle and then Portable_Python is invoked
> > as a PrecCommit test and failed. However, I can't find where
> > it's defined / generated in Gradle or Jenkins groovy. Does
> > anyone know? My branch is synced to master yesterday.
> >
> > Thanks!
> > Mark
> >
>

Re: What is Jenkins job "Portable_Python" in PreCommit?

[Maximilian Michels]

This was merged with https://github.com/apache/beam/pull/6954.

Eventually we want to run a portable WordCount on PreCommit. We will do 
some more testing on Jenkins before it becomes an official PreCommit task.


Thanks,
Max

On 29.11.18 19:03, Mark Liu wrote:
ah, thanks Boyuan! Probably I created the PR in a bad timing. Looks like 
a new PR will fix it.


On Thu, Nov 29, 2018 at 9:50 AM Boyuan Zhang > wrote:


I believe it's in this pending PR:
https://github.com/apache/beam/pull/7157.

On Thu, Nov 29, 2018 at 8:36 AM Mark Liu mailto:mark...@google.com>> wrote:

Hi guys,

I made some changes 
to Python PreCommit Gradle and then Portable_Python is invoked
as a PrecCommit test and failed. However, I can't find where
it's defined / generated in Gradle or Jenkins groovy. Does
anyone know? My branch is synced to master yesterday.

Thanks!
Mark

Re: What is Jenkins job "Portable_Python" in PreCommit?

[Mark Liu]

ah, thanks Boyuan! Probably I created the PR in a bad timing. Looks like a
new PR will fix it.

On Thu, Nov 29, 2018 at 9:50 AM Boyuan Zhang  wrote:

> I believe it's in this pending PR:
> https://github.com/apache/beam/pull/7157.
>
> On Thu, Nov 29, 2018 at 8:36 AM Mark Liu  wrote:
>
>> Hi guys,
>>
>> I made some changes  to Python
>> PreCommit Gradle and then Portable_Python is invoked as a PrecCommit test
>> and failed. However, I can't find where it's defined / generated in Gradle
>> or Jenkins groovy. Does anyone know? My branch is synced to master
>> yesterday.
>>
>> Thanks!
>> Mark
>>
>

Re: What is Jenkins job "Portable_Python" in PreCommit?

[Boyuan Zhang]

I believe it's in this pending PR: https://github.com/apache/beam/pull/7157.

On Thu, Nov 29, 2018 at 8:36 AM Mark Liu  wrote:

> Hi guys,
>
> I made some changes  to Python
> PreCommit Gradle and then Portable_Python is invoked as a PrecCommit test
> and failed. However, I can't find where it's defined / generated in Gradle
> or Jenkins groovy. Does anyone know? My branch is synced to master
> yesterday.
>
> Thanks!
> Mark
>

What is Jenkins job "Portable_Python" in PreCommit?

[Mark Liu]

Hi guys,

I made some changes  to Python
PreCommit Gradle and then Portable_Python is invoked as a PrecCommit test
and failed. However, I can't find where it's defined / generated in Gradle
or Jenkins groovy. Does anyone know? My branch is synced to master
yesterday.

Thanks!
Mark