[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

[Scott Wegner (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642078#comment-16642078
 ] 

Scott Wegner commented on BEAM-5680:


Nothing posted to https://status.apache.org/

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5634) Bring Dataflow Java Worker Code into Beam

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5634?focusedWorklogId=152307=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152307
 ]

ASF GitHub Bot logged work on BEAM-5634:


Author: ASF GitHub Bot
Created on: 08/Oct/18 16:37
Start Date: 08/Oct/18 16:37
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #6561: [BEAM-5634] Bring 
dataflow java worker code into beam
URL: https://github.com/apache/beam/pull/6561#issuecomment-427900910
 
 
   run java precommit


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152307)
Time Spent: 1.5h  (was: 1h 20m)

> Bring Dataflow Java Worker Code into Beam
> -
>
> Key: BEAM-5634
> URL: https://issues.apache.org/jira/browse/BEAM-5634
> Project: Beam
>  Issue Type: Task
>  Components: runner-dataflow
>Reporter: Boyuan Zhang
>Assignee: Boyuan Zhang
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> Discussion: 
> https://lists.apache.org/thread.html/89efd3bc1d30f3d43d4b361a5ee05bd52778c9dc3f43ac72354c2bd9@%3Cdev.beam.apache.org%3E
> Vote: 
> https://lists.apache.org/thread.html/2bdc645659e2fbd7e29f3a2758941faefedb01148a2a11558dfe60f8@%3Cdev.beam.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5634) Bring Dataflow Java Worker Code into Beam

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5634?focusedWorklogId=152306=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152306
 ]

ASF GitHub Bot logged work on BEAM-5634:


Author: ASF GitHub Bot
Created on: 08/Oct/18 16:37
Start Date: 08/Oct/18 16:37
Worklog Time Spent: 10m 
  Work Description: kennknowles commented on issue #6561: [BEAM-5634] Bring 
dataflow java worker code into beam
URL: https://github.com/apache/beam/pull/6561#issuecomment-427900875
 
 
   IP clearance is good; going to re-run Java presubmit since it has been a few 
days.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152306)
Time Spent: 1h 20m  (was: 1h 10m)

> Bring Dataflow Java Worker Code into Beam
> -
>
> Key: BEAM-5634
> URL: https://issues.apache.org/jira/browse/BEAM-5634
> Project: Beam
>  Issue Type: Task
>  Components: runner-dataflow
>Reporter: Boyuan Zhang
>Assignee: Boyuan Zhang
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Discussion: 
> https://lists.apache.org/thread.html/89efd3bc1d30f3d43d4b361a5ee05bd52778c9dc3f43ac72354c2bd9@%3Cdev.beam.apache.org%3E
> Vote: 
> https://lists.apache.org/thread.html/2bdc645659e2fbd7e29f3a2758941faefedb01148a2a11558dfe60f8@%3Cdev.beam.apache.org%3E



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

[Scott Wegner (JIRA)]

Scott Wegner created BEAM-5681:
--

 Summary: [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL 
checks
 Key: BEAM-5681
 URL: https://issues.apache.org/jira/browse/BEAM-5681
 Project: Beam
  Issue Type: Bug
  Components: test-failures, website
Reporter: Scott Wegner
Assignee: Alan Myrvold


_Use this form to file an issue for test failure:_
 * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27]
 * [Test source 
code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140]

Initial investigation:

It seems every link is failing. Same error message:

{{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}}
{{  *  http:// is an invalid URL (line 76)}}
{{ }}
{{}}
{{  }}

The _Cron version of this job has been failing consistently since 10/5 
([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/])


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5613) Snapshot Python dependencies and add to Python_NightlySnapshot

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5613?focusedWorklogId=152313=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152313
 ]

ASF GitHub Bot logged work on BEAM-5613:


Author: ASF GitHub Bot
Created on: 08/Oct/18 16:58
Start Date: 08/Oct/18 16:58
Worklog Time Spent: 10m 
  Work Description: chamikaramj commented on a change in pull request 
#6551: [BEAM-5613] Snapshot of Python depedency and add it to nightly snapshot 
job
URL: https://github.com/apache/beam/pull/6551#discussion_r223433598
 
 

 ##
 File path: sdks/python/scripts/run_snapshot_publish.sh
 ##
 @@ -16,19 +16,27 @@
 #limitations under the License.
 #
 
+BUCKET=gs://beam-python-nightly-snapshots
 
 VERSION=$(awk '/__version__/{print $3}' 
$WORKSPACE/src/sdks/python/apache_beam/version.py)
 VERSION=$(echo $VERSION | cut -c 2- | rev | cut -c 2- | rev)
 time=$(date +"%Y-%m-%dT%H:%M:%S")
 SNAPSHOT="apache-beam-$VERSION-$time.zip"
-BUCKET=gs://beam-python-nightly-snapshots
 
+DEP_SNAPSHOT_ROOT="$BUCKET/dependency_requirements_snapshot"
+DEP_SNAPSHOT_FILE_NAME="beam-py-requirements-$time.txt"
+
+# Snapshots are built by Gradle task :beam-sdks-python:depSnapchot
 
 Review comment:
   nit: s/depSnapchot/depSnapshot


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152313)
Time Spent: 2h 20m  (was: 2h 10m)

> Snapshot Python dependencies and add to Python_NightlySnapshot
> --
>
> Key: BEAM-5613
> URL: https://issues.apache.org/jira/browse/BEAM-5613
> Project: Beam
>  Issue Type: Improvement
>  Components: dependencies
>Reporter: Mark Liu
>Assignee: Mark Liu
>Priority: Major
>  Time Spent: 2h 20m
>  Remaining Estimate: 0h
>
> Python SDK depends on a list of libraries without specific versions. 
> Unexpected or unaware version change could happen if there is a new release 
> or range change in setup.py. This can be detected by dependency checking 
> tools which require to track dependency list and their versions periodically. 
> We could snapshot dependencies periodically using pip freeze and python 
> setup.py egg_info and upload them to a public accessible location to benefit 
> people who want to check dependency info or building related tools.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-5621) Several tests fail on Python 3 with TypeError: unorderable types: str() < int()

[Juta Staes (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5621?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Juta Staes resolved BEAM-5621.
--
   Resolution: Fixed
Fix Version/s: Not applicable

> Several tests fail on Python 3 with TypeError: unorderable types: str() < 
> int()
> ---
>
> Key: BEAM-5621
> URL: https://issues.apache.org/jira/browse/BEAM-5621
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Juta Staes
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 10m
>  Remaining Estimate: 0h
>
> ==
> ERROR: test_remove_duplicates 
> (apache_beam.transforms.ptransform_test.PTransformTest)
> --
> Traceback (most recent call last):
>   File 
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/runners/common.py",
>  line 677, in process
> self.do_fn_invoker.invoke_process(windowed_value)
>   File 
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/runners/common.py",
>  line 414, in invoke_process
> windowed_value, self.process_method(windowed_value.value))
>   File 
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/transforms/core.py",
>  line 1068, in 
> wrapper = lambda x: [fn(x)]
>   File 
> "/usr/local/google/home/valentyn/projects/beam/clean_head/beam/sdks/python/apache_beam/testing/util.py",
>  line 115, in _equal
> sorted_expected = sorted(expected)
> TypeError: unorderable types: str() < int()



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

[Scott Wegner (JIRA)]

Scott Wegner created BEAM-5680:
--

 Summary: [beam_PostCommit_Website_Publish] [:publishWebsite] git 
push 'Authentication failed'
 Key: BEAM-5680
 URL: https://issues.apache.org/jira/browse/BEAM-5680
 Project: Beam
  Issue Type: Bug
  Components: test-failures, website
Reporter: Scott Wegner
Assignee: Melissa Pashniak


_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
 * [Test source 
code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]

Initial investigation:

Failing consistently since 10/7 5am until now ([test 
history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

[Scott Wegner (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642077#comment-16642077
 ] 

Scott Wegner commented on BEAM-5680:


The job correctly ran on 
[website1|https://builds.apache.org/computer/websites1] Jenkins node, which is 
the only thing that should be required to push to gitbox.

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

[Scott Wegner (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642075#comment-16642075
 ] 

Scott Wegner commented on BEAM-5680:


/cc [~alanmyrvold]

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

[Scott Wegner (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642096#comment-16642096
 ] 

Scott Wegner commented on BEAM-5680:


I see other successful jobs run in the 
[git-website|https://builds.apache.org/label/git-websites/] label 
([Solr-reference-guide-master|https://builds.apache.org/job/Solr-reference-guide-master/lastSuccessfulBuild/],
 
[hbase_generate_website|https://builds.apache.org/job/hbase_generate_website/lastSuccessfulBuild/],
 
[Solr-reference-guide-7.x|https://builds.apache.org/job/Solr-reference-guide-7.x/lastSuccessfulBuild/],
 [Incubator 
Site|https://builds.apache.org/job/Incubator%20Site/lastSuccessfulBuild/]), but 
none of them seem to be using gitbox.

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

[Scott Wegner (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642098#comment-16642098
 ] 

Scott Wegner commented on BEAM-5680:


Another recent job failure, 
[Mesos-Websitebot|https://builds.apache.org/job/Mesos-Websitebot/lastSuccessfulBuild/],
 uses gitbox and has a similar failure messages:

{{fatal: Authentication failed for 
'https://gitbox.apache.org/repos/asf/mesos-site.git/'}}
{{Build step 'Execute shell' marked build as failure}}

I'm going to reach out to Apache INFRA.

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

[Scott Wegner (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642127#comment-16642127
 ] 

Scott Wegner commented on BEAM-5681:


I wonder if it's due to the \-PgithubPullRequestId=${ghprbPullId} parameter to 
the job. The 'Cron' version of the job doesn't run as part of a pull-request, 
so this parameter will be empty.

I'm able to repro this by running locally and explicitly setting the parameter 
to whitespace (\-PgithubPullRequestId=" "):  
[https://gradle.com/s/th6k66yjmmxj4]

 

[~alanmyrvold] could you please investigate?

> [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
> ---
>
> Key: BEAM-5681
> URL: https://issues.apache.org/jira/browse/BEAM-5681
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27]
>  * [Test source 
> code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140]
> Initial investigation:
> It seems every link is failing. Same error message:
> {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}}
> {{  *  http:// is an invalid URL (line 76)}}
> {{ }}
> {{ src="//images/beam_logo_navbar.png">}}
> {{  }}
> The _Cron version of this job has been failing consistently since 10/5 
> ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/])
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4130) Portable Flink runner JobService entry point in a Docker container

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4130?focusedWorklogId=152367=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152367
 ]

ASF GitHub Bot logged work on BEAM-4130:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:40
Start Date: 08/Oct/18 19:40
Worklog Time Spent: 10m 
  Work Description: angoenka opened a new pull request #6607: [BEAM-4130] 
Use port 0 to pick dynamic port-BugFix
URL: https://github.com/apache/beam/pull/6607
 
 
   **Please** add a meaningful description for your change here
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
 | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152367)
Time Spent: 10h  (was: 9h 50m)

> Portable Flink runner JobService entry point in a Docker container
> --
>
> Key: BEAM-4130
> URL: https://issues.apache.org/jira/browse/BEAM-4130
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-flink
>

[jira] [Work logged] (BEAM-4176) Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner passes

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4176?focusedWorklogId=152369=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152369
 ]

ASF GitHub Bot logged work on BEAM-4176:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:42
Start Date: 08/Oct/18 19:42
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #6592: [BEAM-4176] WIP 
Enable Post Commit JAVA PVR tests for Flink
URL: https://github.com/apache/beam/pull/6592#issuecomment-427955852
 
 
   Run Java Flink PortableValidatesRunner


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152369)
Time Spent: 31h  (was: 30h 50m)

> Java: Portable batch runner passes all ValidatesRunner tests that 
> non-portable runner passes
> 
>
> Key: BEAM-4176
> URL: https://issues.apache.org/jira/browse/BEAM-4176
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Ben Sidhom
>Assignee: Ankur Goenka
>Priority: Major
> Attachments: 81VxNWtFtke.png, Screen Shot 2018-08-14 at 4.18.31 
> PM.png, Screen Shot 2018-09-03 at 11.07.38 AM.png
>
>  Time Spent: 31h
>  Remaining Estimate: 0h
>
> We need this as a sanity check that runner execution is correct.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4130) Portable Flink runner JobService entry point in a Docker container

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4130?focusedWorklogId=152368=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152368
 ]

ASF GitHub Bot logged work on BEAM-4130:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:41
Start Date: 08/Oct/18 19:41
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #6607: [BEAM-4130] Use port 
0 to pick dynamic port-BugFix
URL: https://github.com/apache/beam/pull/6607#issuecomment-427955480
 
 
   R: @mxm 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152368)
Time Spent: 10h 10m  (was: 10h)

> Portable Flink runner JobService entry point in a Docker container
> --
>
> Key: BEAM-4130
> URL: https://issues.apache.org/jira/browse/BEAM-4130
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-flink
>Reporter: Ben Sidhom
>Assignee: Maximilian Michels
>Priority: Minor
> Fix For: 2.7.0
>
>  Time Spent: 10h 10m
>  Remaining Estimate: 0h
>
> The portable Flink runner exists as a Job Service that runs somewhere. We 
> need a main entry point that itself spins up the job service (and artifact 
> staging service). The main program itself should be packaged into an uberjar 
> such that it can be run locally or submitted to a Flink deployment via `flink 
> run`.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5644) make Planner configurable

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5644?focusedWorklogId=152371=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152371
 ]

ASF GitHub Bot logged work on BEAM-5644:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:44
Start Date: 08/Oct/18 19:44
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6598: 
[BEAM-5644] make planner impl configurable
URL: https://github.com/apache/beam/pull/6598#discussion_r223477121
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java
 ##
 @@ -97,6 +102,24 @@
 .typeSystem(connection.getTypeFactory().getTypeSystem())
 .operatorTable(ChainedSqlOperatorTable.of(opTab0, catalogReader))
 .build();
+
+plannerImplConstructor =
+
getPlannerConstructorFromPipelineOptions(PipelineOptionsFactory.create());
+  }
+
+  public static Constructor 
getPlannerConstructorFromPipelineOptions(PipelineOptions options) {
+String plannerImplClassName = null;
+try {
+  plannerImplClassName = 
options.as(BeamSqlPipelineOptions.class).getPlannerImplClassName();
+  return 
Class.forName(plannerImplClassName).getConstructor(FrameworkConfig.class);
 
 Review comment:
   I would probably also support a no-args constructor here. Eventually maybe 
the one with `pipelineOptions` arg as well.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152371)
Time Spent: 1h 20m  (was: 1h 10m)

> make Planner configurable 
> --
>
> Key: BEAM-5644
> URL: https://issues.apache.org/jira/browse/BEAM-5644
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> We can make planner configurable here: 
> [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145]
>  
> By doing so, we can have different planner implementation to support 
> different SQL dialect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5644) make Planner configurable

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5644?focusedWorklogId=152370=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152370
 ]

ASF GitHub Bot logged work on BEAM-5644:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:44
Start Date: 08/Oct/18 19:44
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6598: 
[BEAM-5644] make planner impl configurable
URL: https://github.com/apache/beam/pull/6598#discussion_r223424411
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/SqlTransform.java
 ##
 @@ -91,6 +91,9 @@
 
 registerFunctions(sqlEnv);
 
+// overwrite BeamSQL's planner constructor.
+sqlEnv.setPlannerConstructor(input.getPipeline().getOptions());
 
 Review comment:
   Mutable things are evil, mutability increases complexity, don't do 
mutability. At most you should pass this in the constructor (factory method).


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152370)
Time Spent: 1h 10m  (was: 1h)

> make Planner configurable 
> --
>
> Key: BEAM-5644
> URL: https://issues.apache.org/jira/browse/BEAM-5644
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 10m
>  Remaining Estimate: 0h
>
> We can make planner configurable here: 
> [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145]
>  
> By doing so, we can have different planner implementation to support 
> different SQL dialect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5644) make Planner configurable

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5644?focusedWorklogId=152374=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152374
 ]

ASF GitHub Bot logged work on BEAM-5644:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:44
Start Date: 08/Oct/18 19:44
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6598: 
[BEAM-5644] make planner impl configurable
URL: https://github.com/apache/beam/pull/6598#discussion_r223475582
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java
 ##
 @@ -56,6 +60,7 @@
   private static final Logger LOG = 
LoggerFactory.getLogger(BeamQueryPlanner.class);
 
 Review comment:
   I think at this point `BeamQueryPlanner` needs refactor. Basically it does 
multiple things and now even more than it did, plus violates encapsulation:
   * creates a config;
   * creates an instance of a planner using the config;
   * exposes a bunch of implementation details on how it constructs the planner;
   * exposes an interface to parse the query and convert it to RelNodes;
   
   To me these look like this can be split into a bunch of independent small 
modules with fewer responsibilities and some inversion of control (each is 
simpler, easier to read and test):
   
   ```
   class BeamCalciteFrameworkConfig {
  static FrameworkConfig create(CalciteConnection connection) { ... }
   }
   
   class CalcitePlannerFactory {
  Planner createCalcitePlanner(
 FrameworkConfig calciteFrameworkConfig,
 BeamSqlPipelineOptions pipelineOptions) {
   
 return  hasPlannerOption(pipelineOptions))
  ? createCustomPlanner(calciteFrameworkConfig)
  : Frameworks.getPlanner(calciteFrameworkConfig);
 }
  }
   }
   ```
   
   Rename BeamQueryPlanner to something like `QueryParser`
   
   ```
   class QueryParser {
  static create(Planner calcitePlanner) {
 this.planner = calcitePlanner;
  }
   }
   ```
   
   Then just configure everything within `BeamSqlEnv`:
   
   ```
   class BeamSqlEnv {
  private BeamSqlEnv(...) {
 ...
 calciteFrameworkConfig = BeamCalciteFrameworkConfig.create(connection);
 calcitePlannerImpl = 
CalcitePlannerFactory.create(calciteFrameworkConfig, getPipelineOptions());
 queryParser = QueryParser.create(calcitePlannerImpl);
  }
   }
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152374)
Time Spent: 1.5h  (was: 1h 20m)

> make Planner configurable 
> --
>
> Key: BEAM-5644
> URL: https://issues.apache.org/jira/browse/BEAM-5644
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1.5h
>  Remaining Estimate: 0h
>
> We can make planner configurable here: 
> [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145]
>  
> By doing so, we can have different planner implementation to support 
> different SQL dialect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5644) make Planner configurable

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5644?focusedWorklogId=152373=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152373
 ]

ASF GitHub Bot logged work on BEAM-5644:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:44
Start Date: 08/Oct/18 19:44
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6598: 
[BEAM-5644] make planner impl configurable
URL: https://github.com/apache/beam/pull/6598#discussion_r223476186
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java
 ##
 @@ -97,6 +102,24 @@
 .typeSystem(connection.getTypeFactory().getTypeSystem())
 .operatorTable(ChainedSqlOperatorTable.of(opTab0, catalogReader))
 .build();
+
+plannerImplConstructor =
+
getPlannerConstructorFromPipelineOptions(PipelineOptionsFactory.create());
+  }
+
+  public static Constructor 
getPlannerConstructorFromPipelineOptions(PipelineOptions options) {
+String plannerImplClassName = null;
+try {
+  plannerImplClassName = 
options.as(BeamSqlPipelineOptions.class).getPlannerImplClassName();
+  return 
Class.forName(plannerImplClassName).getConstructor(FrameworkConfig.class);
+} catch (NoSuchMethodException | ClassNotFoundException e) {
 
 Review comment:
   These need to be separate messages:
* Couldn't the planner implementation class '...' specified in the options;
* Couldn't find a constructor that accepts `FrameworkConfig` parameter;


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152373)
Time Spent: 1h 20m  (was: 1h 10m)

> make Planner configurable 
> --
>
> Key: BEAM-5644
> URL: https://issues.apache.org/jira/browse/BEAM-5644
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> We can make planner configurable here: 
> [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145]
>  
> By doing so, we can have different planner implementation to support 
> different SQL dialect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5644) make Planner configurable

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5644?focusedWorklogId=152372=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152372
 ]

ASF GitHub Bot logged work on BEAM-5644:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:44
Start Date: 08/Oct/18 19:44
Worklog Time Spent: 10m 
  Work Description: akedin commented on a change in pull request #6598: 
[BEAM-5644] make planner impl configurable
URL: https://github.com/apache/beam/pull/6598#discussion_r223476909
 
 

 ##
 File path: 
sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java
 ##
 @@ -142,6 +165,11 @@ public BeamRelNode convertToBeamRel(String sqlStatement)
   }
 
   private Planner getPlanner() {
-return Frameworks.getPlanner(config);
 
 Review comment:
   I think we still should call calcite's factory explicitly, no on guarantees 
there's going to be a constructor with this signature


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152372)
Time Spent: 1h 20m  (was: 1h 10m)

> make Planner configurable 
> --
>
> Key: BEAM-5644
> URL: https://issues.apache.org/jira/browse/BEAM-5644
> Project: Beam
>  Issue Type: New Feature
>  Components: dsl-sql
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> We can make planner configurable here: 
> [BeamQueryPlanner.java#L145|https://github.com/apache/beam/blob/master/sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/BeamQueryPlanner.java#L145]
>  
> By doing so, we can have different planner implementation to support 
> different SQL dialect.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4130) Portable Flink runner JobService entry point in a Docker container

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4130?focusedWorklogId=152376=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152376
 ]

ASF GitHub Bot logged work on BEAM-4130:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:45
Start Date: 08/Oct/18 19:45
Worklog Time Spent: 10m 
  Work Description: mxm commented on a change in pull request #6607: 
[BEAM-4130] Use port 0 to pick dynamic port-BugFix
URL: https://github.com/apache/beam/pull/6607#discussion_r223477264
 
 

 ##
 File path: 
runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkJobServerDriver.java
 ##
 @@ -239,7 +239,7 @@ private InMemoryJobService createJobService() throws 
IOException {
   throws IOException {
 BeamFileSystemArtifactStagingService service = new 
BeamFileSystemArtifactStagingService();
 final GrpcFnServer 
artifactStagingService;
-if (Strings.isNullOrEmpty(configuration.host)) {
+if (configuration.artifactPort == 0) {
 
 Review comment:
   Should this be `Strings.isNullOrEmpty(configuration.host) || 
configuration.port == 0`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152376)
Time Spent: 10.5h  (was: 10h 20m)

> Portable Flink runner JobService entry point in a Docker container
> --
>
> Key: BEAM-4130
> URL: https://issues.apache.org/jira/browse/BEAM-4130
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-flink
>Reporter: Ben Sidhom
>Assignee: Maximilian Michels
>Priority: Minor
> Fix For: 2.7.0
>
>  Time Spent: 10.5h
>  Remaining Estimate: 0h
>
> The portable Flink runner exists as a Job Service that runs somewhere. We 
> need a main entry point that itself spins up the job service (and artifact 
> staging service). The main program itself should be packaged into an uberjar 
> such that it can be run locally or submitted to a Flink deployment via `flink 
> run`.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4130) Portable Flink runner JobService entry point in a Docker container

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4130?focusedWorklogId=152375=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152375
 ]

ASF GitHub Bot logged work on BEAM-4130:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:45
Start Date: 08/Oct/18 19:45
Worklog Time Spent: 10m 
  Work Description: mxm commented on a change in pull request #6607: 
[BEAM-4130] Use port 0 to pick dynamic port-BugFix
URL: https://github.com/apache/beam/pull/6607#discussion_r223477139
 
 

 ##
 File path: 
runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkJobServerDriver.java
 ##
 @@ -201,7 +201,7 @@ public void stop() {
   private GrpcFnServer createJobServer() throws 
IOException {
 InMemoryJobService service = createJobService();
 GrpcFnServer jobServiceGrpcFnServer;
-if (Strings.isNullOrEmpty(configuration.host)) {
+if (configuration.port == 0) {
 
 Review comment:
   Should this be `Strings.isNullOrEmpty(configuration.host) || 
configuration.port == 0`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152375)
Time Spent: 10h 20m  (was: 10h 10m)

> Portable Flink runner JobService entry point in a Docker container
> --
>
> Key: BEAM-4130
> URL: https://issues.apache.org/jira/browse/BEAM-4130
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-flink
>Reporter: Ben Sidhom
>Assignee: Maximilian Michels
>Priority: Minor
> Fix For: 2.7.0
>
>  Time Spent: 10h 20m
>  Remaining Estimate: 0h
>
> The portable Flink runner exists as a Job Service that runs somewhere. We 
> need a main entry point that itself spins up the job service (and artifact 
> staging service). The main program itself should be packaged into an uberjar 
> such that it can be run locally or submitted to a Flink deployment via `flink 
> run`.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-5513) Upgrade google-cloud-pubsub to 0.35.4

[Udi Meiri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Udi Meiri resolved BEAM-5513.
-
   Resolution: Fixed
Fix Version/s: 2.8.0

> Upgrade google-cloud-pubsub to 0.35.4
> -
>
> Key: BEAM-5513
> URL: https://issues.apache.org/jira/browse/BEAM-5513
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core
>Reporter: Udi Meiri
>Assignee: Udi Meiri
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>
> Version 0.35.4 is not the latest version out, but it is the latest supported 
> on Dataflow runner.
> [~markflyhigh]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4130) Portable Flink runner JobService entry point in a Docker container

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4130?focusedWorklogId=152377=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152377
 ]

ASF GitHub Bot logged work on BEAM-4130:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:49
Start Date: 08/Oct/18 19:49
Worklog Time Spent: 10m 
  Work Description: angoenka commented on a change in pull request #6607: 
[BEAM-4130] Use port 0 to pick dynamic port-BugFix
URL: https://github.com/apache/beam/pull/6607#discussion_r223478440
 
 

 ##
 File path: 
runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkJobServerDriver.java
 ##
 @@ -201,7 +201,7 @@ public void stop() {
   private GrpcFnServer createJobServer() throws 
IOException {
 InMemoryJobService service = createJobService();
 GrpcFnServer jobServiceGrpcFnServer;
-if (Strings.isNullOrEmpty(configuration.host)) {
+if (configuration.port == 0) {
 
 Review comment:
   This makes the behavior less consistent.
   Example scenarios:
   When host=null : JobPort Dynamic, ArtifactPort Dynamic
   When host=localhost: JobPort 8099, ArtifactPort  8098
   When host=localhost, jobPort=0: JobPort dynamic, ArtifactPort  8098
   
   Just checking if the JobPort is 0 will remove this confusion


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152377)
Time Spent: 10h 40m  (was: 10.5h)

> Portable Flink runner JobService entry point in a Docker container
> --
>
> Key: BEAM-4130
> URL: https://issues.apache.org/jira/browse/BEAM-4130
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-flink
>Reporter: Ben Sidhom
>Assignee: Maximilian Michels
>Priority: Minor
> Fix For: 2.7.0
>
>  Time Spent: 10h 40m
>  Remaining Estimate: 0h
>
> The portable Flink runner exists as a Job Service that runs somewhere. We 
> need a main entry point that itself spins up the job service (and artifact 
> staging service). The main program itself should be packaged into an uberjar 
> such that it can be run locally or submitted to a Flink deployment via `flink 
> run`.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (BEAM-5539) Beam Dependency Update Request: google-cloud-pubsub

[Udi Meiri (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5539?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Udi Meiri resolved BEAM-5539.
-
   Resolution: Fixed
Fix Version/s: Not applicable

> Beam Dependency Update Request: google-cloud-pubsub
> ---
>
> Key: BEAM-5539
> URL: https://issues.apache.org/jira/browse/BEAM-5539
> Project: Beam
>  Issue Type: Bug
>  Components: dependencies
>Reporter: Beam JIRA Bot
>Assignee: Udi Meiri
>Priority: Major
> Fix For: Not applicable
>
>
>  - 2018-10-01 19:17:59.633423 
> -
> Please consider upgrading the dependency google-cloud-pubsub. 
> The current version is 0.26.0. The latest version is 0.38.0 
> cc: [~markflyhigh], 
>  Please refer to [Beam Dependency Guide 
> |https://beam.apache.org/contribute/dependencies/]for more information. 
> Do Not Modify The Description Above. 
>  - 2018-10-08 12:11:22.339342 
> -
> Please consider upgrading the dependency google-cloud-pubsub. 
> The current version is 0.35.4. The latest version is 0.38.0 
> cc: [~markflyhigh], 
>  Please refer to [Beam Dependency Guide 
> |https://beam.apache.org/contribute/dependencies/]for more information. 
> Do Not Modify The Description Above. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4130) Portable Flink runner JobService entry point in a Docker container

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4130?focusedWorklogId=152378=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152378
 ]

ASF GitHub Bot logged work on BEAM-4130:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:50
Start Date: 08/Oct/18 19:50
Worklog Time Spent: 10m 
  Work Description: angoenka commented on a change in pull request #6607: 
[BEAM-4130] Use port 0 to pick dynamic port-BugFix
URL: https://github.com/apache/beam/pull/6607#discussion_r223478440
 
 

 ##
 File path: 
runners/flink/src/main/java/org/apache/beam/runners/flink/FlinkJobServerDriver.java
 ##
 @@ -201,7 +201,7 @@ public void stop() {
   private GrpcFnServer createJobServer() throws 
IOException {
 InMemoryJobService service = createJobService();
 GrpcFnServer jobServiceGrpcFnServer;
-if (Strings.isNullOrEmpty(configuration.host)) {
+if (configuration.port == 0) {
 
 Review comment:
   I think this makes the behavior less consistent.
   Example scenarios:
   When host=null : JobPort Dynamic, ArtifactPort Dynamic
   When host=localhost: JobPort 8099, ArtifactPort  8098
   When host=localhost, jobPort=0: JobPort dynamic, ArtifactPort  8098
   
   Just checking if the JobPort is 0 will remove this confusion


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152378)
Time Spent: 10h 50m  (was: 10h 40m)

> Portable Flink runner JobService entry point in a Docker container
> --
>
> Key: BEAM-4130
> URL: https://issues.apache.org/jira/browse/BEAM-4130
> Project: Beam
>  Issue Type: New Feature
>  Components: runner-flink
>Reporter: Ben Sidhom
>Assignee: Maximilian Michels
>Priority: Minor
> Fix For: 2.7.0
>
>  Time Spent: 10h 50m
>  Remaining Estimate: 0h
>
> The portable Flink runner exists as a Job Service that runs somewhere. We 
> need a main entry point that itself spins up the job service (and artifact 
> staging service). The main program itself should be packaged into an uberjar 
> such that it can be run locally or submitted to a Flink deployment via `flink 
> run`.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5681?focusedWorklogId=152379=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152379
 ]

ASF GitHub Bot logged work on BEAM-5681:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:52
Start Date: 08/Oct/18 19:52
Worklog Time Spent: 10m 
  Work Description: swegner opened a new pull request #6608: [BEAM-5681] 
Fix website tasks when pull-request ID is specified
URL: https://github.com/apache/beam/pull/6608
 
 
   Website gradle tasks have special support for when running within the 
context of a PR, in order to stage files on GCS. However, adding this support 
broke the non-PR mode along the way.
   
   Specifically, using a PR-based `baseUrl` would break the http-proofer 
internal URL checks, because it would expect to find resources at a /1234/ 
prefix path. The fix for this is to actually generate content in that prefixed 
location.
   
   This PR makes all tasks consistent with how they expect and use baseurl 
paths.
   
   
   
   Follow this checklist to help us incorporate your contribution quickly and 
easily:
   
- [ ] Format the pull request title like `[BEAM-XXX] Fixes bug in 
ApproximateQuantiles`, where you replace `BEAM-XXX` with the appropriate JIRA 
issue, if applicable. This will automatically link the pull request to the 
issue.
- [ ] If this contribution is large, please file an Apache [Individual 
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
   
   It will help us expedite review of your Pull Request if you tag someone 
(e.g. `@username`) to look at it.
   
   Post-Commit Tests Status (on master branch)
   

   
   Lang | SDK | Apex | Dataflow | Flink | Gearpump | Samza | Spark
   --- | --- | --- | --- | --- | --- | --- | ---
   Go | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Go_GradleBuild/lastCompletedBuild/)
 | --- | --- | --- | --- | --- | ---
   Java | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Apex_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Flink_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Gearpump_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Samza_Gradle/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Spark_Gradle/lastCompletedBuild/)
   Python | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_Verify/lastCompletedBuild/)
 | --- | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/lastCompletedBuild/)
  [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Py_ValCont/lastCompletedBuild/)
 | [![Build 
Status](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/badge/icon)](https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/lastCompletedBuild/)
 | --- | --- | ---
   
   
   
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[jira] [Work logged] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5681?focusedWorklogId=152380=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152380
 ]

ASF GitHub Bot logged work on BEAM-5681:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:53
Start Date: 08/Oct/18 19:53
Worklog Time Spent: 10m 
  Work Description: swegner commented on issue #6608: [BEAM-5681] Fix 
website tasks when pull-request ID is specified
URL: https://github.com/apache/beam/pull/6608#issuecomment-427958893
 
 
   R: @alanmyrvold 
   CC: @udim 
   
   I'm in the process of validating to ensure that all tasks pass in both PR 
and non-PR scenarios.


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152380)
Time Spent: 20m  (was: 10m)

> [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
> ---
>
> Key: BEAM-5681
> URL: https://issues.apache.org/jira/browse/BEAM-5681
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Labels: currently-failing
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27]
>  * [Test source 
> code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140]
> Initial investigation:
> It seems every link is failing. Same error message:
> {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}}
> {{  *  http:// is an invalid URL (line 76)}}
> {{ }}
> {{ src="//images/beam_logo_navbar.png">}}
> {{  }}
> The _Cron version of this job has been failing consistently since 10/5 
> ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/])
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4176) Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner passes

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4176?focusedWorklogId=152381=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152381
 ]

ASF GitHub Bot logged work on BEAM-4176:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:59
Start Date: 08/Oct/18 19:59
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #6592: [BEAM-4176] WIP 
Enable Post Commit JAVA PVR tests for Flink
URL: https://github.com/apache/beam/pull/6592#issuecomment-427960658
 
 
   Run Java Flink PortableValidatesRunner


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152381)
Time Spent: 31h 10m  (was: 31h)

> Java: Portable batch runner passes all ValidatesRunner tests that 
> non-portable runner passes
> 
>
> Key: BEAM-4176
> URL: https://issues.apache.org/jira/browse/BEAM-4176
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Ben Sidhom
>Assignee: Ankur Goenka
>Priority: Major
> Attachments: 81VxNWtFtke.png, Screen Shot 2018-08-14 at 4.18.31 
> PM.png, Screen Shot 2018-09-03 at 11.07.38 AM.png
>
>  Time Spent: 31h 10m
>  Remaining Estimate: 0h
>
> We need this as a sanity check that runner execution is correct.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5681?focusedWorklogId=152382=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152382
 ]

ASF GitHub Bot logged work on BEAM-5681:


Author: ASF GitHub Bot
Created on: 08/Oct/18 20:03
Start Date: 08/Oct/18 20:03
Worklog Time Spent: 10m 
  Work Description: swegner commented on issue #6608: [BEAM-5681] Fix 
website tasks when pull-request ID is specified
URL: https://github.com/apache/beam/pull/6608#issuecomment-427961666
 
 
   Tested:
   
   | X | No PR | `-PgithubPullRequestId=6595` |
   |--|--|--|
   | `:buildWebsite` | [success](https://gradle.com/s/flruutfkrwj6q) | 
[success](https://gradle.com/s/urafoimblbgj2) |
   | `:serveWebsite` | success | success |
   | `:testWebsite` | [success](https://gradle.com/s/ra37ssnz3g7zc) | 
[**failure**](https://gradle.com/s/zw2w537ax373m) |
   | `:commitWebsite` | [success](https://gradle.com/s/644a2nhzszn36) | |


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152382)
Time Spent: 0.5h  (was: 20m)

> [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
> ---
>
> Key: BEAM-5681
> URL: https://issues.apache.org/jira/browse/BEAM-5681
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Labels: currently-failing
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27]
>  * [Test source 
> code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140]
> Initial investigation:
> It seems every link is failing. Same error message:
> {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}}
> {{  *  http:// is an invalid URL (line 76)}}
> {{ }}
> {{ src="//images/beam_logo_navbar.png">}}
> {{  }}
> The _Cron version of this job has been failing consistently since 10/5 
> ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/])
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-4176) Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner passes

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4176?focusedWorklogId=152383=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152383
 ]

ASF GitHub Bot logged work on BEAM-4176:


Author: ASF GitHub Bot
Created on: 08/Oct/18 20:05
Start Date: 08/Oct/18 20:05
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #6592: [BEAM-4176] WIP 
Enable Post Commit JAVA PVR tests for Flink
URL: https://github.com/apache/beam/pull/6592#issuecomment-427962144
 
 
   R: @mxm @tweise @robertwb @ryan-williams 


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152383)
Time Spent: 31h 20m  (was: 31h 10m)

> Java: Portable batch runner passes all ValidatesRunner tests that 
> non-portable runner passes
> 
>
> Key: BEAM-4176
> URL: https://issues.apache.org/jira/browse/BEAM-4176
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Ben Sidhom
>Assignee: Ankur Goenka
>Priority: Major
> Attachments: 81VxNWtFtke.png, Screen Shot 2018-08-14 at 4.18.31 
> PM.png, Screen Shot 2018-09-03 at 11.07.38 AM.png
>
>  Time Spent: 31h 20m
>  Remaining Estimate: 0h
>
> We need this as a sanity check that runner execution is correct.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5467) Python Flink ValidatesRunner job fixes

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5467?focusedWorklogId=152385=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152385
 ]

ASF GitHub Bot logged work on BEAM-5467:


Author: ASF GitHub Bot
Created on: 08/Oct/18 20:11
Start Date: 08/Oct/18 20:11
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #6532: [BEAM-5467] 
NOT_FOR_REVIEW Use process SDKHarness to run flink PVR tests.
URL: https://github.com/apache/beam/pull/6532#issuecomment-427963766
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152385)
Time Spent: 2h 40m  (was: 2.5h)

> Python Flink ValidatesRunner job fixes
> --
>
> Key: BEAM-5467
> URL: https://issues.apache.org/jira/browse/BEAM-5467
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Minor
>  Labels: portability-flink
>  Time Spent: 2h 40m
>  Remaining Estimate: 0h
>
> Add status to README
> Rename script and job for consistency
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (BEAM-5089) Add an SDK harness container with Python 3 interpreter for portable pipelines.

[Valentyn Tymofieiev (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5089?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642407#comment-16642407
 ] 

Valentyn Tymofieiev commented on BEAM-5089:
---

Remaining comments:
- Beam SDK should use new container when run in Python 3 environments.
- We should exercise new container in a validates container test suite.

> Add an SDK harness container with Python 3 interpreter for portable pipelines.
> --
>
> Key: BEAM-5089
> URL: https://issues.apache.org/jira/browse/BEAM-5089
> Project: Beam
>  Issue Type: Sub-task
>  Components: sdk-py-core
>Reporter: Valentyn Tymofieiev
>Assignee: Valentyn Tymofieiev
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Work logged] (BEAM-5467) Python Flink ValidatesRunner job fixes

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5467?focusedWorklogId=152387=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152387
 ]

ASF GitHub Bot logged work on BEAM-5467:


Author: ASF GitHub Bot
Created on: 08/Oct/18 20:15
Start Date: 08/Oct/18 20:15
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #6532: [BEAM-5467] 
NOT_FOR_REVIEW Use process SDKHarness to run flink PVR tests.
URL: https://github.com/apache/beam/pull/6532#issuecomment-427964901
 
 
   Run Python Flink ValidatesRunner


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152387)
Time Spent: 2h 50m  (was: 2h 40m)

> Python Flink ValidatesRunner job fixes
> --
>
> Key: BEAM-5467
> URL: https://issues.apache.org/jira/browse/BEAM-5467
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Minor
>  Labels: portability-flink
>  Time Spent: 2h 50m
>  Remaining Estimate: 0h
>
> Add status to README
> Rename script and job for consistency
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[beam] 01/01: Merge pull request #6603 from terrytangyuan/patch-1

[apilloud]

This is an automated email from the ASF dual-hosted git repository.

apilloud pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git

commit bd84b4b3ca5ffd0b6c93e7a7d4bf85f31b35720e
Merge: 1c0a0bf 163c495
Author: Andrew Pilloud 
AuthorDate: Mon Oct 8 13:18:25 2018 -0700

Merge pull request #6603 from terrytangyuan/patch-1

Fixed typo tfrcordio -> tfrecordio

 sdks/python/apache_beam/io/tfrecordio.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

[beam] branch master updated (1c0a0bf -> bd84b4b)

[apilloud]

This is an automated email from the ASF dual-hosted git repository.

apilloud pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/beam.git.


from 1c0a0bf  [BEAM-5634] Bring dataflow java worker code into beam (#6561)
 add 163c495  Fixed typo tfrcordio -> tfrecordio
 new bd84b4b  Merge pull request #6603 from terrytangyuan/patch-1

The 1 revisions listed above as "new" are entirely new to this
repository and will be described in separate emails.  The revisions
listed as "add" were already present in the repository and have only
been added to this reference.


Summary of changes:
 sdks/python/apache_beam/io/tfrecordio.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

Build failed in Jenkins: beam_PostCommit_Website_Publish #134

[Apache Jenkins Server]

See 


Changes:

[github] Fixed typo tfrcordio -> tfrecordio

--
[...truncated 7.74 KB...]
:buildSrc:assemble (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
started.

> Task :buildSrc:assemble
Skipping task ':buildSrc:assemble' as it has no actions.
:buildSrc:assemble (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
completed. Took 0.0 secs.
:buildSrc:spotlessGroovy (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
started.

> Task :buildSrc:spotlessGroovy
file or directory 
'
 not found
file or directory 
'
 not found
file or directory 
'
 not found
Caching disabled for task ':buildSrc:spotlessGroovy': Caching has not been 
enabled for the task
Task ':buildSrc:spotlessGroovy' is not up-to-date because:
  No history is available.
All input files are considered out-of-date for incremental task 
':buildSrc:spotlessGroovy'.
file or directory 
'
 not found
:buildSrc:spotlessGroovy (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
completed. Took 1.33 secs.
:buildSrc:spotlessGroovyCheck (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) started.

> Task :buildSrc:spotlessGroovyCheck
Skipping task ':buildSrc:spotlessGroovyCheck' as it has no actions.
:buildSrc:spotlessGroovyCheck (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) completed. Took 0.001 secs.
:buildSrc:spotlessGroovyGradle (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) started.

> Task :buildSrc:spotlessGroovyGradle
Caching disabled for task ':buildSrc:spotlessGroovyGradle': Caching has not 
been enabled for the task
Task ':buildSrc:spotlessGroovyGradle' is not up-to-date because:
  No history is available.
All input files are considered out-of-date for incremental task 
':buildSrc:spotlessGroovyGradle'.
:buildSrc:spotlessGroovyGradle (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) completed. Took 0.022 secs.
:buildSrc:spotlessGroovyGradleCheck (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) started.

> Task :buildSrc:spotlessGroovyGradleCheck
Skipping task ':buildSrc:spotlessGroovyGradleCheck' as it has no actions.
:buildSrc:spotlessGroovyGradleCheck (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) completed. Took 0.0 secs.
:buildSrc:spotlessCheck (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
started.

> Task :buildSrc:spotlessCheck
Skipping task ':buildSrc:spotlessCheck' as it has no actions.
:buildSrc:spotlessCheck (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
completed. Took 0.0 secs.
:buildSrc:compileTestJava (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
started.

> Task :buildSrc:compileTestJava NO-SOURCE
file or directory 
'
 not found
Skipping task ':buildSrc:compileTestJava' as it has no source files and no 
previous output files.
:buildSrc:compileTestJava (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
completed. Took 0.001 secs.
:buildSrc:compileTestGroovy (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) started.

> Task :buildSrc:compileTestGroovy NO-SOURCE
file or directory 
'
 not found
Skipping task ':buildSrc:compileTestGroovy' as it has no source files and no 
previous output files.
:buildSrc:compileTestGroovy (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) completed. Took 0.001 secs.
:buildSrc:processTestResources (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) started.

> Task :buildSrc:processTestResources NO-SOURCE
file or directory 
'
 not found
Skipping task ':buildSrc:processTestResources' as it has no source files and no 
previous output files.
:buildSrc:processTestResources (Thread[Task worker for ':buildSrc' Thread 
3,5,main]) completed. Took 0.001 secs.
:buildSrc:testClasses (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
started.

> Task :buildSrc:testClasses UP-TO-DATE
Skipping task ':buildSrc:testClasses' as it has no actions.
:buildSrc:testClasses (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 
completed. Took 0.0 secs.
:buildSrc:test (Thread[Task worker for ':buildSrc' Thread 3,5,main]) started.

> Task :buildSrc:test NO-SOURCE
Skipping task ':buildSrc:test' as it has no source files and no previous output 
files.
:buildSrc:test (Thread[Task worker for ':buildSrc' Thread 3,5,main]) 

Build failed in Jenkins: beam_PostCommit_Python_VR_Flink #274

[Apache Jenkins Server]

See 


Changes:

[github] Fixed typo tfrcordio -> tfrecordio

--
Started by GitHub push by apilloud
[EnvInject] - Loading node environment variables.
Building remotely on beam15 (beam) in workspace 

 > git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git config remote.origin.url https://github.com/apache/beam.git # timeout=10
Fetching upstream changes from https://github.com/apache/beam.git
 > git --version # timeout=10
 > git fetch --tags --progress https://github.com/apache/beam.git 
 > +refs/heads/*:refs/remotes/origin/* 
 > +refs/pull/${ghprbPullId}/*:refs/remotes/origin/pr/${ghprbPullId}/*
 > git rev-parse origin/master^{commit} # timeout=10
Checking out Revision bd84b4b3ca5ffd0b6c93e7a7d4bf85f31b35720e (origin/master)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f bd84b4b3ca5ffd0b6c93e7a7d4bf85f31b35720e
Commit message: "Merge pull request #6603 from terrytangyuan/patch-1"
 > git rev-list --no-walk 1c0a0bfba4fb45c78b0ade3284d641c67e4688f3 # timeout=10
Cleaning workspace
 > git rev-parse --verify HEAD # timeout=10
Resetting working tree
 > git reset --hard # timeout=10
 > git clean -fdx # timeout=10
[EnvInject] - Executing scripts and injecting environment variables after the 
SCM step.
[EnvInject] - Injecting as environment variables the properties content 
SPARK_LOCAL_IP=127.0.0.1

[EnvInject] - Variables injected successfully.
[Gradle] - Launching build.
[src] $ 
 
--info --continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g 
-Dorg.gradle.jvmargs=-Xmx4g 
:beam-sdks-python:flinkCompatibilityMatrixBatchProcess 
:beam-sdks-python:flinkCompatibilityMatrixStreamingProcess
Initialized native services in: /home/jenkins/.gradle/native
To honour the JVM settings for this build a new JVM will be forked. Please 
consider using the daemon: 
https://docs.gradle.org/4.10.2/userguide/gradle_daemon.html.
Starting process 'Gradle build daemon'. Working directory: 
/home/jenkins/.gradle/daemon/4.10.2 Command: 
/usr/local/asfpackages/java/jdk1.8.0_172/bin/java -Xmx4g -Dfile.encoding=UTF-8 
-Duser.country=US -Duser.language=en -Duser.variant -cp 
/home/jenkins/.gradle/wrapper/dists/gradle-4.10.2-bin/cghg6c4gf4vkiutgsab8yrnwv/gradle-4.10.2/lib/gradle-launcher-4.10.2.jar
 org.gradle.launcher.daemon.bootstrap.GradleDaemon 4.10.2
Successfully started process 'Gradle build daemon'
An attempt to start the daemon took 0.934 secs.
The client will now receive all logging from the daemon (pid: 17666). The 
daemon log file: /home/jenkins/.gradle/daemon/4.10.2/daemon-17666.out.log
Closing daemon's stdin at end of input.
The daemon will no longer process any standard input.
Daemon will be stopped at the end of the build stopping after processing
Using 12 worker leases.
Starting Build
Parallel execution is an incubating feature.

> Configure project :buildSrc
Evaluating project ':buildSrc' using build file 
'
file or directory 
'
 not found
Selected primary task 'build' from project :
file or directory 
'
 not found
:buildSrc:compileJava (Thread[Task worker for ':buildSrc',5,main]) started.
Using local directory build cache for build ':buildSrc' (location = 
/home/jenkins/.gradle/caches/build-cache-1, removeUnusedEntriesAfter = 7 days).

> Task :buildSrc:compileJava NO-SOURCE
file or directory 
'
 not found
Skipping task ':buildSrc:compileJava' as it has no source files and no previous 
output files.
:buildSrc:compileJava (Thread[Task worker for ':buildSrc',5,main]) completed. 
Took 0.088 secs.
:buildSrc:compileGroovy (Thread[Task worker for ':buildSrc',5,main]) started.

> Task :buildSrc:compileGroovy FROM-CACHE
Build cache key for task ':buildSrc:compileGroovy' is 
ffa895a1cf2b1718eb804dc82ad66ea4
Task ':buildSrc:compileGroovy' is not up-to-date because:
  No history is available.
Origin for task ':buildSrc:compileGroovy': {executionTime=2266, 
hostName=apache-beam-jenkins-slave-group-lbw6, operatingSystem=Linux, 
buildInvocationId=67p7cwv2jbh23lrdpokdubwxee, creationTime=1538771065235, 
type=org.gradle.api.tasks.compile.GroovyCompile_Decorated, userName=jenkins, 
gradleVersion=4.10.2, 
rootPath=
 path=:compileGroovy}
Unpacked output for task ':buildSrc:compileGroovy' from cache.
:buildSrc:compileGroovy (Thread[Task worker for 

[jira] [Created] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

[Scott Wegner (JIRA)]

Scott Wegner created BEAM-5683:
--

 Summary: [beam_PostCommit_Py_VR_Dataflow] 
[test_multiple_empty_outputs] Failure summary
 Key: BEAM-5683
 URL: https://issues.apache.org/jira/browse/BEAM-5683
 Project: Beam
  Issue Type: Bug
  Components: sdk-py-harness, test-failures
Reporter: Scott Wegner
Assignee: Robert Bradshaw


_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
 * [Test source code|TODO]

Initial investigation:

Seems to be failing on pip download.


==
ERROR: test_multiple_empty_outputs 
(apache_beam.transforms.ptransform_test.PTransformTest)
--
Traceback (most recent call last):
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
 line 277, in test_multiple_empty_outputs
pipeline.run()
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
 line 104, in run
result = super(TestPipeline, self).run(test_runner_api)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 403, in run
self.to_runner_api(), self.runner, self._options).run(False)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 416, in run
return self.runner.run_pipeline(self)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
 line 50, in run_pipeline
self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
 line 389, in run_pipeline
self.dataflow_client.create_job(self.job), self)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
 line 184, in wrapper
return fun(*args, **kwargs)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 490, in create_job
self.create_job_description(job)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 519, in create_job_description
resources = self._stage_resour
ces(job.options)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 452, in _stage_resources
staging_location=google_cloud_options.staging_location)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 161, in stage_job_resources
requirements_cache_path)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 411, in _populate_requirements_cache
processes.check_call(cmd_args)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
 line 46, in check_call
return subprocess.check_call(*args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command 
'['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
 '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']' 
returned non-zero exit status 1



_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

[Scott Wegner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5683:
---
Description: 
_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
 * [Test source 
code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390]

Initial investigation:

Seems to be failing on pip download.


==
ERROR: test_multiple_empty_outputs 
(apache_beam.transforms.ptransform_test.PTransformTest)
--
Traceback (most recent call last):
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
 line 277, in test_multiple_empty_outputs
pipeline.run()
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
 line 104, in run
result = super(TestPipeline, self).run(test_runner_api)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 403, in run
self.to_runner_api(), self.runner, self._options).run(False)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 416, in run
return self.runner.run_pipeline(self)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
 line 50, in run_pipeline
self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
 line 389, in run_pipeline
self.dataflow_client.create_job(self.job), self)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
 line 184, in wrapper
return fun(*args, **kwargs)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 490, in create_job
self.create_job_description(job)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 519, in create_job_description
resources = self._stage_resour
ces(job.options)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 452, in _stage_resources
staging_location=google_cloud_options.staging_location)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 161, in stage_job_resources
requirements_cache_path)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 411, in _populate_requirements_cache
processes.check_call(cmd_args)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
 line 46, in check_call
return subprocess.check_call(*args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command 
'['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
 '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']' 
returned non-zero exit status 1



_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._

  was:
_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
 * [Test source code|TODO]

Initial investigation:

Seems to be failing on pip download.

Build failed in Jenkins: beam_PostCommit_Website_Publish #133

[Apache Jenkins Server]

See 


Changes:

[kenn] [BEAM-5634] Bring dataflow java worker code into beam (#6561)

--
[...truncated 8.31 KB...]
> Task :buildSrc:assemble
Skipping task ':buildSrc:assemble' as it has no actions.
:buildSrc:assemble (Thread[Task worker for ':buildSrc' Thread 5,5,main]) 
completed. Took 0.0 secs.
:buildSrc:spotlessGroovy (Thread[Task worker for ':buildSrc' Thread 5,5,main]) 
started.

> Task :buildSrc:spotlessGroovy
file or directory 
'
 not found
file or directory 
'
 not found
file or directory 
'
 not found
Caching disabled for task ':buildSrc:spotlessGroovy': Caching has not been 
enabled for the task
Task ':buildSrc:spotlessGroovy' is not up-to-date because:
  No history is available.
All input files are considered out-of-date for incremental task 
':buildSrc:spotlessGroovy'.
file or directory 
'
 not found
:buildSrc:spotlessGroovy (Thread[Task worker for ':buildSrc' Thread 5,5,main]) 
completed. Took 1.332 secs.
:buildSrc:spotlessGroovyCheck (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) started.

> Task :buildSrc:spotlessGroovyCheck
Skipping task ':buildSrc:spotlessGroovyCheck' as it has no actions.
:buildSrc:spotlessGroovyCheck (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) completed. Took 0.0 secs.
:buildSrc:spotlessGroovyGradle (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) started.

> Task :buildSrc:spotlessGroovyGradle
Caching disabled for task ':buildSrc:spotlessGroovyGradle': Caching has not 
been enabled for the task
Task ':buildSrc:spotlessGroovyGradle' is not up-to-date because:
  No history is available.
All input files are considered out-of-date for incremental task 
':buildSrc:spotlessGroovyGradle'.
:buildSrc:spotlessGroovyGradle (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) completed. Took 0.023 secs.
:buildSrc:spotlessGroovyGradleCheck (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) started.

> Task :buildSrc:spotlessGroovyGradleCheck
Skipping task ':buildSrc:spotlessGroovyGradleCheck' as it has no actions.
:buildSrc:spotlessGroovyGradleCheck (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) completed. Took 0.0 secs.
:buildSrc:spotlessCheck (Thread[Task worker for ':buildSrc' Thread 5,5,main]) 
started.

> Task :buildSrc:spotlessCheck
Skipping task ':buildSrc:spotlessCheck' as it has no actions.
:buildSrc:spotlessCheck (Thread[Task worker for ':buildSrc' Thread 5,5,main]) 
completed. Took 0.0 secs.
:buildSrc:compileTestJava (Thread[Task worker for ':buildSrc' Thread 5,5,main]) 
started.

> Task :buildSrc:compileTestJava NO-SOURCE
file or directory 
'
 not found
Skipping task ':buildSrc:compileTestJava' as it has no source files and no 
previous output files.
:buildSrc:compileTestJava (Thread[Task worker for ':buildSrc' Thread 5,5,main]) 
completed. Took 0.001 secs.
:buildSrc:compileTestGroovy (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) started.

> Task :buildSrc:compileTestGroovy NO-SOURCE
file or directory 
'
 not found
Skipping task ':buildSrc:compileTestGroovy' as it has no source files and no 
previous output files.
:buildSrc:compileTestGroovy (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) completed. Took 0.001 secs.
:buildSrc:processTestResources (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) started.

> Task :buildSrc:processTestResources NO-SOURCE
file or directory 
'
 not found
Skipping task ':buildSrc:processTestResources' as it has no source files and no 
previous output files.
:buildSrc:processTestResources (Thread[Task worker for ':buildSrc' Thread 
5,5,main]) completed. Took 0.001 secs.
:buildSrc:testClasses (Thread[Task worker for ':buildSrc' Thread 5,5,main]) 
started.

> Task :buildSrc:testClasses UP-TO-DATE
Skipping task ':buildSrc:testClasses' as it has no actions.
:buildSrc:testClasses (Thread[Task worker for ':buildSrc' Thread 5,5,main]) 
completed. Took 0.0 secs.
:buildSrc:test (Thread[Task worker for ':buildSrc' Thread 5,5,main]) started.

> Task :buildSrc:test NO-SOURCE
Skipping task ':buildSrc:test' as it has no source files and no previous output 
files.
:buildSrc:test (Thread[Task worker for ':buildSrc' Thread 5,5,main]) completed. 
Took 0.002 secs.
:buildSrc:check (Thread[Task worker 

Build failed in Jenkins: beam_PostCommit_Python_VR_Flink #273

[Apache Jenkins Server]

See 


Changes:

[kenn] [BEAM-5634] Bring dataflow java worker code into beam (#6561)

--
[...truncated 51.09 MB...]
[ToKeyedWorkItem (15/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Ensuring all FileSystem streams are closed for task ToKeyedWorkItem (15/16) 
(87dbda8352708192f4a53564f7c29ad5) [FINISHED]
[flink-akka.actor.default-dispatcher-5] INFO 
org.apache.flink.runtime.executiongraph.ExecutionGraph - GroupByKey -> 
24GroupByKey/GroupByWindow.None/beam:env:docker:v1:0 (4/16) 
(01528b81d669a34d384f7882e99df747) switched from RUNNING to FINISHED.
[ToKeyedWorkItem (1/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Freeing task resources for ToKeyedWorkItem (1/16) 
(e746691bb9d7a9c3804bd1136784cb00).
[ToKeyedWorkItem (13/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Freeing task resources for ToKeyedWorkItem (13/16) 
(444d02a9181958e0217d1e5cd7eb057f).
[ToKeyedWorkItem (2/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
ToKeyedWorkItem (2/16) (25903b2259afb5b0e1f4415fa875c0ff) switched from RUNNING 
to FINISHED.
[ToKeyedWorkItem (2/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Freeing task resources for ToKeyedWorkItem (2/16) 
(25903b2259afb5b0e1f4415fa875c0ff).
[ToKeyedWorkItem (1/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Ensuring all FileSystem streams are closed for task ToKeyedWorkItem (1/16) 
(e746691bb9d7a9c3804bd1136784cb00) [FINISHED]
[ToKeyedWorkItem (5/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
ToKeyedWorkItem (5/16) (dbd535ef7e0b52259909a5b6f4021b14) switched from RUNNING 
to FINISHED.
[ToKeyedWorkItem (5/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Freeing task resources for ToKeyedWorkItem (5/16) 
(dbd535ef7e0b52259909a5b6f4021b14).
[ToKeyedWorkItem (13/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Ensuring all FileSystem streams are closed for task ToKeyedWorkItem (13/16) 
(444d02a9181958e0217d1e5cd7eb057f) [FINISHED]
[ToKeyedWorkItem (3/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
ToKeyedWorkItem (3/16) (2f2b3ef9e1aaf95c8dc8401214b45f50) switched from RUNNING 
to FINISHED.
[ToKeyedWorkItem (3/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Freeing task resources for ToKeyedWorkItem (3/16) 
(2f2b3ef9e1aaf95c8dc8401214b45f50).
[ToKeyedWorkItem (2/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Ensuring all FileSystem streams are closed for task ToKeyedWorkItem (2/16) 
(25903b2259afb5b0e1f4415fa875c0ff) [FINISHED]
[ToKeyedWorkItem (5/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Ensuring all FileSystem streams are closed for task ToKeyedWorkItem (5/16) 
(dbd535ef7e0b52259909a5b6f4021b14) [FINISHED]
[flink-akka.actor.default-dispatcher-5] INFO 
org.apache.flink.runtime.executiongraph.ExecutionGraph - ToKeyedWorkItem (8/16) 
(ff9df2cb2565524dc0fd1e96e1cfc067) switched from RUNNING to FINISHED.
[flink-akka.actor.default-dispatcher-3] INFO 
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and 
sending final execution state FINISHED to JobManager for task GroupByKey -> 
24GroupByKey/GroupByWindow.None/beam:env:docker:v1:0 
c6a9e6c2019105f5f82d8d0f020c7044.
[ToKeyedWorkItem (3/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Ensuring all FileSystem streams are closed for task ToKeyedWorkItem (3/16) 
(2f2b3ef9e1aaf95c8dc8401214b45f50) [FINISHED]
[flink-akka.actor.default-dispatcher-5] INFO 
org.apache.flink.runtime.executiongraph.ExecutionGraph - GroupByKey -> 
24GroupByKey/GroupByWindow.None/beam:env:docker:v1:0 (10/16) 
(4934fa1f00309a30aaaf612216dd3b2f) switched from RUNNING to FINISHED.
[ToKeyedWorkItem (14/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
ToKeyedWorkItem (14/16) (a9ffeb86a995b4c7e36adfb294650850) switched from 
RUNNING to FINISHED.
[flink-akka.actor.default-dispatcher-3] INFO 
org.apache.flink.runtime.taskexecutor.TaskExecutor - Un-registering task and 
sending final execution state FINISHED to JobManager for task GroupByKey -> 
24GroupByKey/GroupByWindow.None/beam:env:docker:v1:0 
5f4378d4695feaf14aa03919562110ce.
[flink-akka.actor.default-dispatcher-5] INFO 
org.apache.flink.runtime.executiongraph.ExecutionGraph - ToKeyedWorkItem (4/16) 
(11bcd21304f23a32afdd6760831be2f5) switched from RUNNING to FINISHED.
[ToKeyedWorkItem (16/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
ToKeyedWorkItem (16/16) (26c51afb02a373dd3d5cb11b65e8f4c0) switched from 
RUNNING to FINISHED.
[ToKeyedWorkItem (16/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Freeing task resources for ToKeyedWorkItem (16/16) 
(26c51afb02a373dd3d5cb11b65e8f4c0).
[ToKeyedWorkItem (14/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Freeing task resources for ToKeyedWorkItem (14/16) 
(a9ffeb86a995b4c7e36adfb294650850).
[ToKeyedWorkItem (16/16)] INFO org.apache.flink.runtime.taskmanager.Task - 
Ensuring all 

[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

[Scott Wegner (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642309#comment-16642309
 ] 

Scott Wegner commented on BEAM-5683:


[~angoenka] I see you [previously worked on the 
[_populate_requirements_cache|https://github.com/apache/beam/pull/5251] code. 
Can you help investigate this flake or find a more appropriate owner?

> [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
> --
>
> Key: BEAM-5683
> URL: https://issues.apache.org/jira/browse/BEAM-5683
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-harness, test-failures
>Reporter: Scott Wegner
>Assignee: Robert Bradshaw
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
>  * [Test source 
> code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390]
> Initial investigation:
> Seems to be failing on pip download.
> ==
> ERROR: test_multiple_empty_outputs 
> (apache_beam.transforms.ptransform_test.PTransformTest)
> --
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
>  line 277, in test_multiple_empty_outputs
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
>  line 104, in run
> result = super(TestPipeline, self).run(test_runner_api)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 403, in run
> self.to_runner_api(), self.runner, self._options).run(False)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 416, in run
> return self.runner.run_pipeline(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
>  line 50, in run_pipeline
> self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
>  line 389, in run_pipeline
> self.dataflow_client.create_job(self.job), self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
>  line 184, in wrapper
> return fun(*args, **kwargs)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 490, in create_job
> self.create_job_description(job)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 519, in create_job_description
> resources = self._stage_resour
> ces(job.options)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 452, in _stage_resources
> staging_location=google_cloud_options.staging_location)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 161, in stage_job_resources
> requirements_cache_path)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 411, in _populate_requirements_cache
> processes.check_call(cmd_args)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
>  line 46, in check_call
> return subprocess.check_call(*args, **kwargs)
>   File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
> raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command 
> '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
>  '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
> 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', 

[jira] [Commented] (BEAM-4505) Archive/Retire apache/beam-site repository

[Udi Meiri (JIRA)]

[ 
https://issues.apache.org/jira/browse/BEAM-4505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642310#comment-16642310
 ] 

Udi Meiri commented on BEAM-4505:
-

Things left to do:
1. Make 'master' the default branch on apache/beam-site.
2. Delete 'asf-git' branch from apache/beam-site.

I don't have permission to do any of these steps.

> Archive/Retire apache/beam-site repository
> --
>
> Key: BEAM-4505
> URL: https://issues.apache.org/jira/browse/BEAM-4505
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: beam-site-automation-reliability
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Assigned] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

[Scott Wegner (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5683:
--

Assignee: Ankur Goenka  (was: Robert Bradshaw)

> [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
> --
>
> Key: BEAM-5683
> URL: https://issues.apache.org/jira/browse/BEAM-5683
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-harness, test-failures
>Reporter: Scott Wegner
>Assignee: Ankur Goenka
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
>  * [Test source 
> code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390]
> Initial investigation:
> Seems to be failing on pip download.
> ==
> ERROR: test_multiple_empty_outputs 
> (apache_beam.transforms.ptransform_test.PTransformTest)
> --
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
>  line 277, in test_multiple_empty_outputs
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
>  line 104, in run
> result = super(TestPipeline, self).run(test_runner_api)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 403, in run
> self.to_runner_api(), self.runner, self._options).run(False)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 416, in run
> return self.runner.run_pipeline(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
>  line 50, in run_pipeline
> self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
>  line 389, in run_pipeline
> self.dataflow_client.create_job(self.job), self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
>  line 184, in wrapper
> return fun(*args, **kwargs)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 490, in create_job
> self.create_job_description(job)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 519, in create_job_description
> resources = self._stage_resour
> ces(job.options)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 452, in _stage_resources
> staging_location=google_cloud_options.staging_location)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 161, in stage_job_resources
> requirements_cache_path)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 411, in _populate_requirements_cache
> processes.check_call(cmd_args)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
>  line 46, in check_call
> return subprocess.check_call(*args, **kwargs)
>   File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
> raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command 
> '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
>  '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
> 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', 
> ':all:']' returned non-zero exit status 1
> 
> _After you've filled out the above details, please [assign the issue to an 
> 

Jenkins build is back to normal : beam_PostCommit_Py_VR_Dataflow #1290

[Apache Jenkins Server]

See 

[jira] [Work logged] (BEAM-4176) Java: Portable batch runner passes all ValidatesRunner tests that non-portable runner passes

[ASF GitHub Bot (JIRA)]

 [ 
https://issues.apache.org/jira/browse/BEAM-4176?focusedWorklogId=152365=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-152365
 ]

ASF GitHub Bot logged work on BEAM-4176:


Author: ASF GitHub Bot
Created on: 08/Oct/18 19:32
Start Date: 08/Oct/18 19:32
Worklog Time Spent: 10m 
  Work Description: angoenka commented on issue #6592: [BEAM-4176] WIP 
Enable Post Commit JAVA PVR tests for Flink
URL: https://github.com/apache/beam/pull/6592#issuecomment-427953217
 
 
   Run Seed Job


This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


Issue Time Tracking
---

Worklog Id: (was: 152365)
Time Spent: 30h 50m  (was: 30h 40m)

> Java: Portable batch runner passes all ValidatesRunner tests that 
> non-portable runner passes
> 
>
> Key: BEAM-4176
> URL: https://issues.apache.org/jira/browse/BEAM-4176
> Project: Beam
>  Issue Type: Bug
>  Components: runner-flink
>Reporter: Ben Sidhom
>Assignee: Ankur Goenka
>Priority: Major
> Attachments: 81VxNWtFtke.png, Screen Shot 2018-08-14 at 4.18.31 
> PM.png, Screen Shot 2018-09-03 at 11.07.38 AM.png
>
>  Time Spent: 30h 50m
>  Remaining Estimate: 0h
>
> We need this as a sanity check that runner execution is correct.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)