[jira] [Commented] (BEAM-5735) Contributor Guide Improvements

2018-10-12 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648557#comment-16648557
 ] 

Scott Wegner commented on BEAM-5735:


I made some small boilerplate changes while we were discussing these in person. 
It's not nearly ready to check-in, but in case it's useful: 
https://github.com/swegner/beam/commit/06aee6b3903020e04b7ce4000ba71be525e04721

> Contributor Guide Improvements
> --
>
> Key: BEAM-5735
> URL: https://issues.apache.org/jira/browse/BEAM-5735
> Project: Beam
>  Issue Type: Improvement
>  Components: website
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>
> This is a wish-list for improvements to the Beam contributor guide.
> Many thanks to [~rohdesam] for the feedback which helped shape this list.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (BEAM-5740) Refactor permissions section into bullet-points

2018-10-12 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5740:
--

Assignee: Scott Wegner

> Refactor permissions section into bullet-points
> ---
>
> Key: BEAM-5740
> URL: https://issues.apache.org/jira/browse/BEAM-5740
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>
> The permissions section has good content, but it's not easily browseable if 
> you're looking for a specific thing (i.e. Slack permissions). We should 
> refactor it into bullet points.
> For permissions that require reaching out via email/Slack, we should link to 
> some previous example. It lowers the barrier to entry if a new contributor 
> can copy/paste some existing template.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (BEAM-5743) Move "works in progress" out of getting started guide

2018-10-12 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5743:
--

Assignee: Scott Wegner

> Move "works in progress" out of getting started guide
> -
>
> Key: BEAM-5743
> URL: https://issues.apache.org/jira/browse/BEAM-5743
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>
> These aren't relevant to a new contributor, and it makes the contributor 
> guide less focused. They should be moved to a separate page.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (BEAM-5741) Move "Contact Us" to a top-level link

2018-10-12 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5741:
--

Assignee: (was: Melissa Pashniak)

> Move "Contact Us" to a top-level link
> -
>
> Key: BEAM-5741
> URL: https://issues.apache.org/jira/browse/BEAM-5741
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Priority: Major
>
> It should be very easy to figure out how to get in touch with community. 
> "Contact Us" should be a top-level link on the page.
> The page can also be improved with:
> * Some basic text on how to use subscribe / unsubscribe links
> * Recommendations on how to use various communications channels (Slack for 
> quick questions, dev@ for longer conversations. And all decisions should make 
> it back to dev@)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5740) Refactor permissions section into bullet-points

2018-10-12 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5740:
---
Description: 
The permissions section has good content, but it's not easily browseable if 
you're looking for a specific thing (i.e. Slack permissions). We should 
refactor it into bullet points.

For permissions that require reaching out via email/Slack, we should link to 
some previous example. It lowers the barrier to entry if a new contributor can 
copy/paste some existing template.

  was:The permissions section has good content, but it's not easily browseable 
if you're looking for a specific thing (i.e. Slack permissions). We should 
refactor it into bullet points.


> Refactor permissions section into bullet-points
> ---
>
> Key: BEAM-5740
> URL: https://issues.apache.org/jira/browse/BEAM-5740
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Priority: Major
>
> The permissions section has good content, but it's not easily browseable if 
> you're looking for a specific thing (i.e. Slack permissions). We should 
> refactor it into bullet points.
> For permissions that require reaching out via email/Slack, we should link to 
> some previous example. It lowers the barrier to entry if a new contributor 
> can copy/paste some existing template.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5743) Move "works in progress" out of getting started guide

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5743:
--

 Summary: Move "works in progress" out of getting started guide
 Key: BEAM-5743
 URL: https://issues.apache.org/jira/browse/BEAM-5743
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner


These aren't relevant to a new contributor, and it makes the contributor guide 
less focused. They should be moved to a separate page.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5742) Add expectations for code reviews

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5742:
--

 Summary: Add expectations for code reviews
 Key: BEAM-5742
 URL: https://issues.apache.org/jira/browse/BEAM-5742
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner


There should be a dedicated section about how we to code reviews, and 
expectations for contributors / reviewers. Things like:
* PR's should have linked JIRA
* Code changes should also include tests
* Small changes are easier to review
* How to find a reviewer, when you can expect reviewer to engage
* How automatic test runs work (PreCommits run by on each commit; build / test 
should be passing before asking for a reviewer)
* How to re-run tests, and how/when to run Post-Commits



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5741) Move "Contact Us" to a top-level link

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5741:
--

 Summary: Move "Contact Us" to a top-level link
 Key: BEAM-5741
 URL: https://issues.apache.org/jira/browse/BEAM-5741
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner
Assignee: Melissa Pashniak


It should be very easy to figure out how to get in touch with community. 
"Contact Us" should be a top-level link on the page.

The page can also be improved with:
* Some basic text on how to use subscribe / unsubscribe links
* Recommendations on how to use various communications channels (Slack for 
quick questions, dev@ for longer conversations. And all decisions should make 
it back to dev@)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5740) Refactor permissions section into bullet-points

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5740:
--

 Summary: Refactor permissions section into bullet-points
 Key: BEAM-5740
 URL: https://issues.apache.org/jira/browse/BEAM-5740
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner


The permissions section has good content, but it's not easily browseable if 
you're looking for a specific thing (i.e. Slack permissions). We should 
refactor it into bullet points.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5739) Contributor Story: "Submitting your first PR"

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5739:
--

 Summary: Contributor Story: "Submitting your first PR"
 Key: BEAM-5739
 URL: https://issues.apache.org/jira/browse/BEAM-5739
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner


We should write the user story for "Submitting your first PR", with 
prescriptive steps on getting started. It should include:

* Forking the repo and setting up the dev environment
* How to build/test
* Choosing an IDE
* language / SDK-specific tips + website
* "When will my changes go live?"



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5738) Refactor Contributor Guide into "Stories"

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5738:
--

 Summary: Refactor Contributor Guide into "Stories"
 Key: BEAM-5738
 URL: https://issues.apache.org/jira/browse/BEAM-5738
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner


The Contributor Guide has become a dumping ground for topics that are maybe 
relevant to contributors. It's long and unorganized and it doesn't really tell 
a contributor how to get started.

We should refactor the contributor guide into "stories", like "How to submit 
your first PR", "How to contribute to the website", "How to propose a new 
feature", etc.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5737) Create a contributor FAQ page

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5737:
--

 Summary: Create a contributor FAQ page
 Key: BEAM-5737
 URL: https://issues.apache.org/jira/browse/BEAM-5737
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner


This should be an easy "dumping ground" place to add documentation, pretty much 
about anything. It should be low-barrier to add new content or update existing. 
High-quality or frequently-used topics can be promoted to top level pages.

It probably makes sense to put this on the wiki. But it should be linked very 
prominently from the top of the Contributor Guide



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5736) List prerequisite knowledge with links to external docs

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5736:
--

 Summary: List prerequisite knowledge with links to external docs
 Key: BEAM-5736
 URL: https://issues.apache.org/jira/browse/BEAM-5736
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner
Assignee: Scott Wegner


Our contributor guide makes some assumptions about prior knowledge. We should 
be explicit about what we expect contributors to already know, and give links 
to places where they can learn more if necessary.

Examples:
* Git & GitHub workflows
* Beam (understand what it does, what it's for, how it fits into ecosystem)
* JDK 8 installed
* Windows / Mac / Linux dev environment (be explicit about what we support)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5735) Contributor Guide Improvements

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5735:
--

 Summary: Contributor Guide Improvements
 Key: BEAM-5735
 URL: https://issues.apache.org/jira/browse/BEAM-5735
 Project: Beam
  Issue Type: Improvement
  Components: website
Reporter: Scott Wegner
Assignee: Alan Myrvold


This is a wish-list for improvements to the Beam contributor guide.

Many thanks to [~rohdesam] for the feedback which helped shape this list.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (BEAM-5672) Staged URL should be posted to GitHub PR

2018-10-12 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5672:
--

Assignee: Scott Wegner  (was: Alan Myrvold)

> Staged URL should be posted to GitHub PR
> 
>
> Key: BEAM-5672
> URL: https://issues.apache.org/jira/browse/BEAM-5672
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Website PRs are automatically tested and staged via Jenkins to a location on 
> GCS: https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Commit/
> However, it's difficult to find the URL for the staged job; it's buried 
> inside the Jenkins job, inside the Gradle scan, inside the log.
> I believe there's an API for posting back a context message to the pull 
> request. If possible, we should push the URL so it's easily viewable from the 
> pull request.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5672) Staged URL should be posted to GitHub PR

2018-10-12 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648082#comment-16648082
 ] 

Scott Wegner commented on BEAM-5672:


re: Add it to the pr template: We have an existing PR template in GitHub, but 
it's not specific to website changes. GitHub allows creating multiple PR 
templates for a repository, but it seems the way to select a template is via 
URL query parameters and is not triggered automatically i.e. based on changed 
file path. I don't think this is a viable solution.

[PR 6669|https://github.com/apache/beam/pull/6669] pulls the website URL up to 
the top-level Jenkins job page; I think this is sufficient until we can think 
of nicer solutions.

> Staged URL should be posted to GitHub PR
> 
>
> Key: BEAM-5672
> URL: https://issues.apache.org/jira/browse/BEAM-5672
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> Website PRs are automatically tested and staged via Jenkins to a location on 
> GCS: https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Commit/
> However, it's difficult to find the URL for the staged job; it's buried 
> inside the Jenkins job, inside the Gradle scan, inside the log.
> I believe there's an API for posting back a context message to the pull 
> request. If possible, we should push the URL so it's easily viewable from the 
> pull request.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5727) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No terminal state was returned"

2018-10-12 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648065#comment-16648065
 ] 

Scott Wegner commented on BEAM-5727:


[~rohdesam] could you help triage this? I don't see any additional logs 
captured here that would be helpful. Perhaps we can try to correlate it with 
some service logs; if not another idea would be to add more failure logging 
here so we have a better picture for next time.

> [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No 
> terminal state was returned"
> 
>
> Key: BEAM-5727
> URL: https://issues.apache.org/jira/browse/BEAM-5727
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, test-failures
>Reporter: Scott Wegner
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w]
>  * [Test source 
> code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352]
> * [Dataflow 
> job|https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-11_15_19_52-2781743142799123679?project=apache-beam-testing]
> Initial investigation:
> The Dataflow job succeeded, but it seems that the service response was 
> missing completion state / metrics:
> Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob 
> waitUntilFinish
> WARNING: No terminal state was returned. State value UNKNOWN
> Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner 
> checkForPAssertSuccess
> WARNING: Metrics not present for Dataflow job 
> 2018-10-11_15_19_52-2781743142799123679.
> Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner 
> run
> WARNING: Dataflow job 2018-10-11_15_19_52-2781743142799123679 did not output 
> a success or failure metric.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (BEAM-5727) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No terminal state was returned"

2018-10-12 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5727:
--

Assignee: Sam Rohde

> [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No 
> terminal state was returned"
> 
>
> Key: BEAM-5727
> URL: https://issues.apache.org/jira/browse/BEAM-5727
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, test-failures
>Reporter: Scott Wegner
>Assignee: Sam Rohde
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w]
>  * [Test source 
> code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352]
> * [Dataflow 
> job|https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-11_15_19_52-2781743142799123679?project=apache-beam-testing]
> Initial investigation:
> The Dataflow job succeeded, but it seems that the service response was 
> missing completion state / metrics:
> Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob 
> waitUntilFinish
> WARNING: No terminal state was returned. State value UNKNOWN
> Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner 
> checkForPAssertSuccess
> WARNING: Metrics not present for Dataflow job 
> 2018-10-11_15_19_52-2781743142799123679.
> Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner 
> run
> WARNING: Dataflow job 2018-10-11_15_19_52-2781743142799123679 did not output 
> a success or failure metric.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5727) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No terminal state was returned"

2018-10-12 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648063#comment-16648063
 ] 

Scott Wegner commented on BEAM-5727:


The failure here comes from the {{DataflowPipelineJob}} 
[waitUntilFinish()|https://github.com/apache/beam/blame/42a03a6bd2e6cfdffab02752a31f3139a08a8d94/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L309]
 loop. Th call to 
[getJobWithRetries()|https://github.com/apache/beam/blame/42a03a6bd2e6cfdffab02752a31f3139a08a8d94/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L497]
 seems to have returned without a valid job status. 

The code for this hasn't changed in a very long time. I suspect this is either 
a rare flake or some regression in the Dataflow service.

> [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No 
> terminal state was returned"
> 
>
> Key: BEAM-5727
> URL: https://issues.apache.org/jira/browse/BEAM-5727
> Project: Beam
>  Issue Type: Bug
>  Components: runner-dataflow, test-failures
>Reporter: Scott Wegner
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w]
>  * [Test source 
> code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352]
> * [Dataflow 
> job|https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-11_15_19_52-2781743142799123679?project=apache-beam-testing]
> Initial investigation:
> The Dataflow job succeeded, but it seems that the service response was 
> missing completion state / metrics:
> Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob 
> waitUntilFinish
> WARNING: No terminal state was returned. State value UNKNOWN
> Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner 
> checkForPAssertSuccess
> WARNING: Metrics not present for Dataflow job 
> 2018-10-11_15_19_52-2781743142799123679.
> Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner 
> run
> WARNING: Dataflow job 2018-10-11_15_19_52-2781743142799123679 did not output 
> a success or failure metric.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5727) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No terminal state was returned"

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5727:
--

 Summary: [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] 
[testKvSwap] "No terminal state was returned"
 Key: BEAM-5727
 URL: https://issues.apache.org/jira/browse/BEAM-5727
 Project: Beam
  Issue Type: Bug
  Components: runner-dataflow, test-failures
Reporter: Scott Wegner


_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w]
 * [Test source 
code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352]
* [Dataflow 
job|https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-11_15_19_52-2781743142799123679?project=apache-beam-testing]

Initial investigation:

The Dataflow job succeeded, but it seems that the service response was missing 
completion state / metrics:

Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob 
waitUntilFinish
WARNING: No terminal state was returned. State value UNKNOWN
Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner 
checkForPAssertSuccess
WARNING: Metrics not present for Dataflow job 
2018-10-11_15_19_52-2781743142799123679.
Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner run
WARNING: Dataflow job 2018-10-11_15_19_52-2781743142799123679 did not output a 
success or failure metric.


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5726) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] No terminal state was returned

2018-10-12 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5726:
--

 Summary: [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] 
[testKvSwap] No terminal state was returned
 Key: BEAM-5726
 URL: https://issues.apache.org/jira/browse/BEAM-5726
 Project: Beam
  Issue Type: Bug
  Components: test-failures
Reporter: Scott Wegner


_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w]
 * [Test source 
code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352]

Initial investigation:

(Add any investigation notes so far) 


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5672) Staged URL should be posted to GitHub PR

2018-10-11 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16647187#comment-16647187
 ] 

Scott Wegner commented on BEAM-5672:


The functionality I was thinking of is powered by the ghprb plugin via the 
[buildStatus|https://github.com/jenkinsci/ghprb-plugin#job-dsl-support] hook. 
However, this only supports a static string. It wouldn't be possible to set a 
status with a different URL per pull request.

However, we could possible add it to the github PR template?

> Staged URL should be posted to GitHub PR
> 
>
> Key: BEAM-5672
> URL: https://issues.apache.org/jira/browse/BEAM-5672
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>
> Website PRs are automatically tested and staged via Jenkins to a location on 
> GCS: https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Commit/
> However, it's difficult to find the URL for the staged job; it's buried 
> inside the Jenkins job, inside the Gradle scan, inside the log.
> I believe there's an API for posting back a context message to the pull 
> request. If possible, we should push the URL so it's easily viewable from the 
> pull request.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Reopened] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

2018-10-11 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reopened BEAM-5681:

  Assignee: Scott Wegner  (was: Alan Myrvold)

The fix from https://github.com/apache/beam/pull/6655 is incorrect; the PR ID 
binding happens at Seed Job runtime, rather than during the actual job 
execution. As a result, it was always using the PR context from the Seed Job 
run.

> [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
> ---
>
> Key: BEAM-5681
> URL: https://issues.apache.org/jira/browse/BEAM-5681
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: currently-failing
> Fix For: Not applicable
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27]
>  * [Test source 
> code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140]
> Initial investigation:
> It seems every link is failing. Same error message:
> {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}}
> {{  *  http:// is an invalid URL (line 76)}}
> {{ }}
> {{ src="//images/beam_logo_navbar.png">}}
> {{  }}
> The _Cron version of this job has been failing consistently since 10/5 
> ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/])
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5717) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination bucket gs://apache-beam-website-pull-requests does not exist or the write to the destination mu

2018-10-11 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5717.

   Resolution: Fixed
Fix Version/s: Not applicable

> [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination 
> bucket gs://apache-beam-website-pull-requests does not exist or the write to 
> the destination must be restarted
> -
>
> Key: BEAM-5717
> URL: https://issues.apache.org/jira/browse/BEAM-5717
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: currently-failing
> Fix For: Not applicable
>
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/3a7ujky4ekojw/console-log?task=:beam-website:stageWebsite#L4610]
>  * [Test source 
> code|https://github.com/apache/beam/blob/280277e7788b4c28680dd8ca02d54a55195b24ba/website/build.gradle#L271]
> Initial investigation:
> I haven't seen this issue before; it's not clear if this is a persistent 
> failure or a flake. I will investigate further.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5719) Misconfigured GCS staging jobs can nuke entire top-level bucket

2018-10-11 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5719.

   Resolution: Fixed
Fix Version/s: Not applicable

> Misconfigured GCS staging jobs can nuke entire top-level bucket
> ---
>
> Key: BEAM-5719
> URL: https://issues.apache.org/jira/browse/BEAM-5719
> Project: Beam
>  Issue Type: Sub-task
>  Components: build-system, website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> See: https://github.com/apache/beam/pull/6655#issuecomment-429037632
> A bug in the Website_Stage_GCS job was causing the Jenkins job to wipe out 
> the top-level GCS bucket used to stage and serve the website from. This ends 
> up blocking all future runs of the job and takes manual effort to recreate 
> and configure the bucket.
> The linked PR should fix the immediate issue, but it's still the case that a 
> bug in the job/configuration could potentially wipe it out again. We should 
> refactor the job or modify the command so that there is no longer danger of 
> wiping it out.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5719) Misconfigured GCS staging jobs can nuke entire top-level bucket

2018-10-11 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5719:
--

 Summary: Misconfigured GCS staging jobs can nuke entire top-level 
bucket
 Key: BEAM-5719
 URL: https://issues.apache.org/jira/browse/BEAM-5719
 Project: Beam
  Issue Type: Sub-task
  Components: build-system, website
Reporter: Scott Wegner
Assignee: Scott Wegner


See: https://github.com/apache/beam/pull/6655#issuecomment-429037632

A bug in the Website_Stage_GCS job was causing the Jenkins job to wipe out the 
top-level GCS bucket used to stage and serve the website from. This ends up 
blocking all future runs of the job and takes manual effort to recreate and 
configure the bucket.

The linked PR should fix the immediate issue, but it's still the case that a 
bug in the job/configuration could potentially wipe it out again. We should 
refactor the job or modify the command so that there is no longer danger of 
wiping it out.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job fixes

2018-10-11 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646845#comment-16646845
 ] 

Scott Wegner commented on BEAM-5467:


Thanks for the additional context. I'm not an expert on diagnosing memory 
issues, but here's what I can pull out of there:

* The build scan shows [some stats on memory 
usage|https://scans.gradle.com/s/f2u3q2obrgaqu/performance/build#memory], and 
for this build I see "PS Eden Space" of 1.36/1.36 GB (99.5%). I would deduce 
that the JVM ran out of allotted memory causing the segfault.
* The [infrastructure 
tab|https://scans.gradle.com/s/f2u3q2obrgaqu#infrastructure] shows the "Max JVM 
memory heap size" for the job: 3824 MB
* In the [timeline|https://scans.gradle.com/s/f2u3q2obrgaqu/timeline] I can see 
that the task that failed was 
{{:beam-sdks-python:flinkCompatibilityMatrixBatch}}. Nothing was running 
concurrently as part of the build, so either this task ate up the entire heap 
space, or some previous task is leaking memory.

My recommendation would be to work towards getting a local repro so that you 
can attach a memory profiler and validate potential fixes. The Jenkins job 
shows the full command-line used to launch the job, including JVM memory 
configuration:

{{gradlew --info --continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g 
-Dorg.gradle.jvmargs=-Xmx4g :beam-sdks-python:flinkCompatibilityMatrixBatch 
:beam-sdks-python:flinkCompatibilityMatrixStreaming}}

> Python Flink ValidatesRunner job fixes
> --
>
> Key: BEAM-5467
> URL: https://issues.apache.org/jira/browse/BEAM-5467
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Minor
>  Labels: portability-flink
>  Time Spent: 9h 40m
>  Remaining Estimate: 0h
>
> Add status to README
> Rename script and job for consistency
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job fixes

2018-10-11 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646757#comment-16646757
 ] 

Scott Wegner commented on BEAM-5467:


[~thw] offhand this doesn't look familiar to me. Can you link to a Jenkin job 
run / Gradle build scan with more details?

> Python Flink ValidatesRunner job fixes
> --
>
> Key: BEAM-5467
> URL: https://issues.apache.org/jira/browse/BEAM-5467
> Project: Beam
>  Issue Type: Improvement
>  Components: runner-flink
>Reporter: Thomas Weise
>Assignee: Thomas Weise
>Priority: Minor
>  Labels: portability-flink
>  Time Spent: 9h 40m
>  Remaining Estimate: 0h
>
> Add status to README
> Rename script and job for consistency
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job

2018-10-11 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5669.

   Resolution: Fixed
Fix Version/s: (was: 2.8.0)
   Not applicable

> Empty commits pushed by PostCommit_Website_Publish jenkins job
> --
>
> Key: BEAM-5669
> URL: https://issues.apache.org/jira/browse/BEAM-5669
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: 2.8.0
>Reporter: Alan Myrvold
>Assignee: Scott Wegner
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> The isClean() call in 
> https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165
>  is returning false due to unstaged .gradle and build files.
> The asf-site branch needs a .gitignore, which can be the same as the master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job

2018-10-11 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646738#comment-16646738
 ] 

Scott Wegner commented on BEAM-5669:


The above PR is now merged, and I'm no longer seeing empty commits.

I've also removed previous empty commits from the branch using {{git rebase}}: 
https://github.com/apache/beam/commits/asf-site

> Empty commits pushed by PostCommit_Website_Publish jenkins job
> --
>
> Key: BEAM-5669
> URL: https://issues.apache.org/jira/browse/BEAM-5669
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: 2.8.0
>Reporter: Alan Myrvold
>Assignee: Scott Wegner
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> The isClean() call in 
> https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165
>  is returning false due to unstaged .gradle and build files.
> The asf-site branch needs a .gitignore, which can be the same as the master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5717) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination bucket gs://apache-beam-website-pull-requests does not exist or the write to the destination m

2018-10-11 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646607#comment-16646607
 ] 

Scott Wegner commented on BEAM-5717:


It appears the root-cause of this is that from the Jenkins job we attempt to 
pass a {{-PgithubPullRequestId}} even when there is no associated PR (i.e. for 
the cron job). 

In the [Jenkins 
consoleFull|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/consoleFull]
 log, I see the commandline:

{{$ 
/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Website_Stage_GCS_Cron/src/gradlew
 --info --continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g 
-Dorg.gradle.jvmargs=-Xmx4g -PgithubPullRequestId=${ghprbPullId} 
:beam-website:stageWebsite}}

The easiest fix would probably be to remove the parameter from the Jenkins job 
if there is no PR.

> [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination 
> bucket gs://apache-beam-website-pull-requests does not exist or the write to 
> the destination must be restarted
> -
>
> Key: BEAM-5717
> URL: https://issues.apache.org/jira/browse/BEAM-5717
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/3a7ujky4ekojw/console-log?task=:beam-website:stageWebsite#L4610]
>  * [Test source 
> code|https://github.com/apache/beam/blob/280277e7788b4c28680dd8ca02d54a55195b24ba/website/build.gradle#L271]
> Initial investigation:
> I haven't seen this issue before; it's not clear if this is a persistent 
> failure or a flake. I will investigate further.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5717) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination bucket gs://apache-beam-website-pull-requests does not exist or the write to the destination m

2018-10-11 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646608#comment-16646608
 ] 

Scott Wegner commented on BEAM-5717:


/cc [~alanmyrvold]

> [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination 
> bucket gs://apache-beam-website-pull-requests does not exist or the write to 
> the destination must be restarted
> -
>
> Key: BEAM-5717
> URL: https://issues.apache.org/jira/browse/BEAM-5717
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/3a7ujky4ekojw/console-log?task=:beam-website:stageWebsite#L4610]
>  * [Test source 
> code|https://github.com/apache/beam/blob/280277e7788b4c28680dd8ca02d54a55195b24ba/website/build.gradle#L271]
> Initial investigation:
> I haven't seen this issue before; it's not clear if this is a persistent 
> failure or a flake. I will investigate further.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5717) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination bucket gs://apache-beam-website-pull-requests does not exist or the write to the destination mus

2018-10-11 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5717:
--

 Summary: [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 
404 The destination bucket gs://apache-beam-website-pull-requests does not 
exist or the write to the destination must be restarted
 Key: BEAM-5717
 URL: https://issues.apache.org/jira/browse/BEAM-5717
 Project: Beam
  Issue Type: Bug
  Components: test-failures
Reporter: Scott Wegner
Assignee: Scott Wegner


_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/3a7ujky4ekojw/console-log?task=:beam-website:stageWebsite#L4610]
 * [Test source 
code|https://github.com/apache/beam/blob/280277e7788b4c28680dd8ca02d54a55195b24ba/website/build.gradle#L271]

Initial investigation:

I haven't seen this issue before; it's not clear if this is a persistent 
failure or a flake. I will investigate further.


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation

2018-10-10 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5678.

   Resolution: Fixed
Fix Version/s: 2.8.0

> Jeckyll redirect_from incompatible with HTMLProofer url validation
> --
>
> Key: BEAM-5678
> URL: https://issues.apache.org/jira/browse/BEAM-5678
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Minor
> Fix For: 2.8.0
>
>
> Renaming a URL and using the Jekyll {{redirect_from}} feature breaks 
> pre-commit validation.
> We hit this on 
> [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]:
> bq. The scenario for using a redirect is if you're moving {{pageA.md}} to 
> {{pageB.md}}; you would rename the file to {{pageB.md}} and add 
> {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} 
> with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} 
> doesn't exist on the livesite yet, so it will always fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation

2018-10-10 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645734#comment-16645734
 ] 

Scott Wegner commented on BEAM-5678:


As of https://github.com/apache/beam/pull/6608, this no longer seems to be an 
issue. The locally-generated HTML used for testing no longer specifies as site 
{{url}} in the _config.yml, so the redirect plugin generates relative 
redirects. Huzzah!

> Jeckyll redirect_from incompatible with HTMLProofer url validation
> --
>
> Key: BEAM-5678
> URL: https://issues.apache.org/jira/browse/BEAM-5678
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Minor
> Fix For: 2.8.0
>
>
> Renaming a URL and using the Jekyll {{redirect_from}} feature breaks 
> pre-commit validation.
> We hit this on 
> [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]:
> bq. The scenario for using a redirect is if you're moving {{pageA.md}} to 
> {{pageB.md}}; you would rename the file to {{pageB.md}} and add 
> {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} 
> with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} 
> doesn't exist on the livesite yet, so it will always fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-4503) Drain/migrate in-progress PR's from apache/beam-site

2018-10-10 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645690#comment-16645690
 ] 

Scott Wegner commented on BEAM-4503:


Only 3 PR's remaining; I pinged each of them today. 
https://github.com/apache/beam-site/pulls

> Drain/migrate in-progress PR's from apache/beam-site
> 
>
> Key: BEAM-4503
> URL: https://issues.apache.org/jira/browse/BEAM-4503
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: beam-site-automation-reliability
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation

2018-10-10 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5678:
--

Assignee: Scott Wegner

> Jeckyll redirect_from incompatible with HTMLProofer url validation
> --
>
> Key: BEAM-5678
> URL: https://issues.apache.org/jira/browse/BEAM-5678
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Minor
>
> Renaming a URL and using the Jekyll {{redirect_from}} feature breaks 
> pre-commit validation.
> We hit this on 
> [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]:
> bq. The scenario for using a redirect is if you're moving {{pageA.md}} to 
> {{pageB.md}}; you would rename the file to {{pageB.md}} and add 
> {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} 
> with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} 
> doesn't exist on the livesite yet, so it will always fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (BEAM-4497) Add pages for master Javadocs / Pydocs and incorporate into post-commit job

2018-10-10 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-4497:
--

Assignee: (was: Scott Wegner)

> Add pages for master Javadocs / Pydocs and incorporate into post-commit job
> ---
>
> Key: BEAM-4497
> URL: https://issues.apache.org/jira/browse/BEAM-4497
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Priority: Major
>  Labels: beam-site-automation-reliability
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job

2018-10-10 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5669:
--

Assignee: Scott Wegner  (was: Alan Myrvold)

> Empty commits pushed by PostCommit_Website_Publish jenkins job
> --
>
> Key: BEAM-5669
> URL: https://issues.apache.org/jira/browse/BEAM-5669
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: 2.8.0
>Reporter: Alan Myrvold
>Assignee: Scott Wegner
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The isClean() call in 
> https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165
>  is returning false due to unstaged .gradle and build files.
> The asf-site branch needs a .gitignore, which can be the same as the master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Reopened] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job

2018-10-10 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reopened BEAM-5669:


I believe this is still an issue. Here's an example empty commit from today: 
https://github.com/apache/beam/commit/5eb09607e52b621a3bed6a868a324465781d3e56

> Empty commits pushed by PostCommit_Website_Publish jenkins job
> --
>
> Key: BEAM-5669
> URL: https://issues.apache.org/jira/browse/BEAM-5669
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: 2.8.0
>Reporter: Alan Myrvold
>Assignee: Alan Myrvold
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The isClean() call in 
> https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165
>  is returning false due to unstaged .gradle and build files.
> The asf-site branch needs a .gitignore, which can be the same as the master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5688) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on githubPullRequestId assert

2018-10-10 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5688.

   Resolution: Fixed
Fix Version/s: Not applicable

This should be fixed with: https://github.com/apache/beam/pull/6608

> [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on 
> githubPullRequestId assert
> --
>
> Key: BEAM-5688
> URL: https://issues.apache.org/jira/browse/BEAM-5688
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Labels: currently-failing
> Fix For: Not applicable
>
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/20/]
>  * [Gradle Build Scan|https://gradle.com/s/7mqwgjegf5hge]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L234]
> Initial investigation:
> This is a problem with how the website gradle scripts are implemented to 
> accept an githubPullRequestId. The Cron job will not have an associated PR, 
> so this currently fails.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

2018-10-10 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5681.

   Resolution: Fixed
Fix Version/s: Not applicable

This should be fixed with: https://github.com/apache/beam/pull/6608

> [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
> ---
>
> Key: BEAM-5681
> URL: https://issues.apache.org/jira/browse/BEAM-5681
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Labels: currently-failing
> Fix For: Not applicable
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27]
>  * [Test source 
> code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140]
> Initial investigation:
> It seems every link is failing. Same error message:
> {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}}
> {{  *  http:// is an invalid URL (line 76)}}
> {{ }}
> {{ src="//images/beam_logo_navbar.png">}}
> {{  }}
> The _Cron version of this job has been failing consistently since 10/5 
> ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/])
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Closed] (BEAM-5704) Update release guide for beam-site changes

2018-10-10 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner closed BEAM-5704.
--
   Resolution: Not A Problem
Fix Version/s: (was: 2.8.0)
   Not applicable

> Update release guide for beam-site changes
> --
>
> Key: BEAM-5704
> URL: https://issues.apache.org/jira/browse/BEAM-5704
> Project: Beam
>  Issue Type: Bug
>  Components: website
>Reporter: Ahmet Altay
>Assignee: Scott Wegner
>Priority: Major
> Fix For: Not applicable
>
>
> [https://beam.apache.org/contribute/release-guide] has instructions related 
> to updating beam-site. They need to be updated to reflect the beam-site 
> deprecation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5704) Update release guide for beam-site changes

2018-10-10 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645412#comment-16645412
 ] 

Scott Wegner commented on BEAM-5704:


I spoke to [~altay] and we believe the docs are up-to-date.

> Update release guide for beam-site changes
> --
>
> Key: BEAM-5704
> URL: https://issues.apache.org/jira/browse/BEAM-5704
> Project: Beam
>  Issue Type: Bug
>  Components: website
>Reporter: Ahmet Altay
>Assignee: Scott Wegner
>Priority: Major
> Fix For: Not applicable
>
>
> [https://beam.apache.org/contribute/release-guide] has instructions related 
> to updating beam-site. They need to be updated to reflect the beam-site 
> deprecation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5704) Update release guide for beam-site changes

2018-10-10 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645406#comment-16645406
 ] 

Scott Wegner commented on BEAM-5704:


This was tracked via BEAM-4501, [~udim] worked on this. Are there specific 
parts that still need updating?

> Update release guide for beam-site changes
> --
>
> Key: BEAM-5704
> URL: https://issues.apache.org/jira/browse/BEAM-5704
> Project: Beam
>  Issue Type: Bug
>  Components: website
>Reporter: Ahmet Altay
>Assignee: Scott Wegner
>Priority: Major
> Fix For: 2.8.0
>
>
> [https://beam.apache.org/contribute/release-guide] has instructions related 
> to updating beam-site. They need to be updated to reflect the beam-site 
> deprecation.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to pip download flake

2018-10-09 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643939#comment-16643939
 ] 

Scott Wegner commented on BEAM-5683:


This failed again today: 
https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1303/

> [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to 
> pip download flake
> --
>
> Key: BEAM-5683
> URL: https://issues.apache.org/jira/browse/BEAM-5683
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-harness, test-failures
>Reporter: Scott Wegner
>Assignee: Ankur Goenka
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
>  * [Test source 
> code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390]
> Initial investigation:
> Seems to be failing on pip download.
> ==
> ERROR: test_multiple_empty_outputs 
> (apache_beam.transforms.ptransform_test.PTransformTest)
> --
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
>  line 277, in test_multiple_empty_outputs
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
>  line 104, in run
> result = super(TestPipeline, self).run(test_runner_api)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 403, in run
> self.to_runner_api(), self.runner, self._options).run(False)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 416, in run
> return self.runner.run_pipeline(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
>  line 50, in run_pipeline
> self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
>  line 389, in run_pipeline
> self.dataflow_client.create_job(self.job), self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
>  line 184, in wrapper
> return fun(*args, **kwargs)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 490, in create_job
> self.create_job_description(job)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 519, in create_job_description
> resources = self._stage_resour
> ces(job.options)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 452, in _stage_resources
> staging_location=google_cloud_options.staging_location)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 161, in stage_job_resources
> requirements_cache_path)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 411, in _populate_requirements_cache
> processes.check_call(cmd_args)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
>  line 46, in check_call
> return subprocess.check_call(*args, **kwargs)
>   File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
> raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command 
> '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
>  '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
> 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', 
> ':all:']' returned non-zero exit status 1
> 
> _After you've 

[jira] [Resolved] (BEAM-5294) [beam_Release_Gradle_NightlySnapshot] Failing due to website test.

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5294.

   Resolution: Fixed
Fix Version/s: Not applicable

> [beam_Release_Gradle_NightlySnapshot] Failing due to website test.
> --
>
> Key: BEAM-5294
> URL: https://issues.apache.org/jira/browse/BEAM-5294
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Boyuan Zhang
>Assignee: Alan Myrvold
>Priority: Major
> Fix For: Not applicable
>
>
> Build link: 
> [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/]
> [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/185/]
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5522) beam_PostRelease_NightlySnapshot timed out

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5522.

   Resolution: Cannot Reproduce
Fix Version/s: Not applicable

> beam_PostRelease_NightlySnapshot timed out
> --
>
> Key: BEAM-5522
> URL: https://issues.apache.org/jira/browse/BEAM-5522
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, testing
>Reporter: Ahmet Altay
>Assignee: Alan Myrvold
>Priority: Major
> Fix For: Not applicable
>
>
> https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/383/console



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5407) [beam_PostCommit_Go_GradleBuild][testE2ETopWikiPages][RolledBack] Breaks post commit

2018-10-09 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643833#comment-16643833
 ] 

Scott Wegner commented on BEAM-5407:


Is this resolved? Can this be closed?

> [beam_PostCommit_Go_GradleBuild][testE2ETopWikiPages][RolledBack] Breaks post 
> commit
> 
>
> Key: BEAM-5407
> URL: https://issues.apache.org/jira/browse/BEAM-5407
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Batkhuyag Batsaikhan
>Assignee: Pablo Estrada
>Priority: Major
>
> Failing job url: 
> https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1482/testReport/



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Closed] (BEAM-5534) AutocompleteIT and WikiTopSessionsIT fail in google testing

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner closed BEAM-5534.
--
   Resolution: Fixed
Fix Version/s: Not applicable

> AutocompleteIT and WikiTopSessionsIT fail in google testing
> ---
>
> Key: BEAM-5534
> URL: https://issues.apache.org/jira/browse/BEAM-5534
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Pablo Estrada
>Assignee: Pablo Estrada
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 1h 20m
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Closed] (BEAM-5125) beam_PostCommit_Java_GradleBuild org.apache.beam.runners.flink PortableExecutionTest testExecution_1_ flaky

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner closed BEAM-5125.
--
   Resolution: Cannot Reproduce
Fix Version/s: Not applicable

> beam_PostCommit_Java_GradleBuild org.apache.beam.runners.flink 
> PortableExecutionTest testExecution_1_ flaky
> ---
>
> Key: BEAM-5125
> URL: https://issues.apache.org/jira/browse/BEAM-5125
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Mikhail Gryzykhin
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 1h 50m
>  Remaining Estimate: 0h
>
> Test fails in both: post and precommit tests. Fails more often in pre-commits.
> Pre-commit history: 
> [https://builds.apache.org/job/beam_PreCommit_Java_Phrase/180/testReport/junit/org.apache.beam.runners.flink/PortableExecutionTest/testExecution_1_/history/]
> Post-commit history: 
> [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1223/testReport/junit/org.apache.beam.runners.flink/PortableExecutionTest/testExecution_1_/history/?start=75]
> Sample job:
> https://builds.apache.org/job/beam_PreCommit_Java_Phrase/180/testReport/junit/org.apache.beam.runners.flink/PortableExecutionTest/testExecution_1_/
> Log:
> java.lang.AssertionError: job state expected: but was: at 
> org.junit.Assert.fail(Assert.java:88) at 
> org.junit.Assert.failNotEquals(Assert.java:834) at 
> org.junit.Assert.assertEquals(Assert.java:118) at 
> org.apache.beam.runners.flink.PortableExecutionTest.testExecution(PortableExecutionTest.java:177)
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5367) [beam_Release_Gradle_NightlySnapshot] is broken due to apache/beam website test failure

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5367.

Resolution: Fixed

> [beam_Release_Gradle_NightlySnapshot] is broken due to apache/beam website 
> test failure
> ---
>
> Key: BEAM-5367
> URL: https://issues.apache.org/jira/browse/BEAM-5367
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Batkhuyag Batsaikhan
>Assignee: Batkhuyag Batsaikhan
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> apache/website test is breaking nightly build. Since we don't have stable 
> apache/beam website, we should remove it from the build for now. Until we 
> have an actual plan to fully migrate from asf beam-site, we should disable 
> apache/website checks from Beam build.
> Failing job url: [https://scans.gradle.com/s/uxb6mdigqj4n4/console-log#L23465]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5367) [beam_Release_Gradle_NightlySnapshot] is broken due to apache/beam website test failure

2018-10-09 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643830#comment-16643830
 ] 

Scott Wegner commented on BEAM-5367:


This is now fixed, latest runs have been successful: 
https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/395/

> [beam_Release_Gradle_NightlySnapshot] is broken due to apache/beam website 
> test failure
> ---
>
> Key: BEAM-5367
> URL: https://issues.apache.org/jira/browse/BEAM-5367
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Batkhuyag Batsaikhan
>Assignee: Batkhuyag Batsaikhan
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 1h
>  Remaining Estimate: 0h
>
> apache/website test is breaking nightly build. Since we don't have stable 
> apache/beam website, we should remove it from the build for now. Until we 
> have an actual plan to fully migrate from asf beam-site, we should disable 
> apache/website checks from Beam build.
> Failing job url: [https://scans.gradle.com/s/uxb6mdigqj4n4/console-log#L23465]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5243) beam_Release_Gradle_NightlySnapshot InvocationError py27-cython/bin/python setup.py nosetests

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5243:
---
Labels:   (was: currently-failing)

> beam_Release_Gradle_NightlySnapshot InvocationError py27-cython/bin/python 
> setup.py nosetests
> -
>
> Key: BEAM-5243
> URL: https://issues.apache.org/jira/browse/BEAM-5243
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-core, test-failures
>Reporter: Andrew Pilloud
>Assignee: Ahmet Altay
>Priority: Major
>
> It isn't clear to me what exactly failed, logs are full of stack traces.
>  [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/151/]
> [https://builds.apache.org/job/beam_PostCommit_Python_Verify/5844/]
>  
>  *01:00:38* ERROR: InvocationError for command 
> '/home/jenkins/jenkins-slave/workspace/beam_Release_Gradle_NightlySnapshot/src/sdks/python/target/.tox/py27-cython/bin/python
>  setup.py nosetests' (exited with code -11)*01:00:38* 
> ___ summary 
> *01:00:38* ERROR: py27-cython: commands 
> failed*01:00:38*



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5665) [beam_PreCommit_Website_Commit] [:testWebsite] External link http://www.atrato.io/blog/2017/04/08/apache-apex-cli/ failed: response code 0 means something's wrong.

2018-10-09 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643826#comment-16643826
 ] 

Scott Wegner commented on BEAM-5665:


http://www.atrato.io/ still 404's. If this doesn't resolve by next week we 
should remove the dead link.

> [beam_PreCommit_Website_Commit] [:testWebsite]  External link 
> http://www.atrato.io/blog/2017/04/08/apache-apex-cli/ failed: response code 0 
> means something's wrong.
> 
>
> Key: BEAM-5665
> URL: https://issues.apache.org/jira/browse/BEAM-5665
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PreCommit_Website_Commit/249/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/2sqxa7hlvdeum/console-log?task=:beam-website:testWebsite#L12]
>  * [Test source 
> code|https://github.com/apache/beam-site/blob/asf-site/Rakefile]
> Initial investigation:
> It seems http://www.atrato.io/ is down or no longer available. We should 
> disable the test for now and perhaps remove the dead link if it no longer 
> works.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5661) [beam_PostCommit_Py_ValCont] [:beam-sdks-python-container:docker] no such file or directory

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5661:
---
Labels:   (was: currently-failing)

> [beam_PostCommit_Py_ValCont] [:beam-sdks-python-container:docker] no such 
> file or directory
> ---
>
> Key: BEAM-5661
> URL: https://issues.apache.org/jira/browse/BEAM-5661
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Assignee: Henning Rohde
>Priority: Major
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Py_ValCont/902/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/pmjbkhxaeryx4/console-log?task=:beam-sdks-python-container:docker#L2]
>  * [Test source 
> code|https://github.com/apache/beam/blob/845f8d0abcc5a8d7f93457c27aff0feeb1a867d5/sdks/python/container/build.gradle#L61]
> Initial investigation:
> {{failed to get digest 
> sha256:4ee4ea2f0113e98b49d8e376ce847feb374ddf2b8ea775502459d8a1b8a3eaed: open 
> /var/lib/docker/image/aufs/imagedb/content/sha256/4ee4ea2f0113e98b49d8e376ce847feb374ddf2b8ea775502459d8a1b8a3eaed:
>  no such file or directory}}
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5665) [beam_PreCommit_Website_Commit] [:testWebsite] External link http://www.atrato.io/blog/2017/04/08/apache-apex-cli/ failed: response code 0 means something's wrong.

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5665:
---
Labels:   (was: currently-failing)

> [beam_PreCommit_Website_Commit] [:testWebsite]  External link 
> http://www.atrato.io/blog/2017/04/08/apache-apex-cli/ failed: response code 0 
> means something's wrong.
> 
>
> Key: BEAM-5665
> URL: https://issues.apache.org/jira/browse/BEAM-5665
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PreCommit_Website_Commit/249/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/2sqxa7hlvdeum/console-log?task=:beam-website:testWebsite#L12]
>  * [Test source 
> code|https://github.com/apache/beam-site/blob/asf-site/Rakefile]
> Initial investigation:
> It seems http://www.atrato.io/ is down or no longer available. We should 
> disable the test for now and perhaps remove the dead link if it no longer 
> works.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5662) [beam_PostCommit_Website_Publish] [:testWebsite] External link http://wiki.apache.org/incubator/BeamProposal failed: got a time out

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5662:
---
Labels:   (was: currently-failing)

> [beam_PostCommit_Website_Publish] [:testWebsite] External link 
> http://wiki.apache.org/incubator/BeamProposal failed: got a time out
> ---
>
> Key: BEAM-5662
> URL: https://issues.apache.org/jira/browse/BEAM-5662
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/94/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/h4mayefon7v7q/console-log?task=:beam-website:testWebsite#L12]
>  * [Test source 
> code|https://github.com/apache/beam/blob/845f8d0abcc5a8d7f93457c27aff0feeb1a867d5/website/Rakefile#L6]
> Initial investigation:
> The failed link is http://wiki.apache.org/incubator/BeamProposal 
> When I visit this link, it works for me. This is likely a flake.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5662) [beam_PostCommit_Website_Publish] [:testWebsite] External link http://wiki.apache.org/incubator/BeamProposal failed: got a time out

2018-10-09 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643825#comment-16643825
 ] 

Scott Wegner commented on BEAM-5662:


I haven't seen this in a while, but I'll keep open for a bit just in case.

> [beam_PostCommit_Website_Publish] [:testWebsite] External link 
> http://wiki.apache.org/incubator/BeamProposal failed: got a time out
> ---
>
> Key: BEAM-5662
> URL: https://issues.apache.org/jira/browse/BEAM-5662
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/94/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/h4mayefon7v7q/console-log?task=:beam-website:testWebsite#L12]
>  * [Test source 
> code|https://github.com/apache/beam/blob/845f8d0abcc5a8d7f93457c27aff0feeb1a867d5/website/Rakefile#L6]
> Initial investigation:
> The failed link is http://wiki.apache.org/incubator/BeamProposal 
> When I visit this link, it works for me. This is likely a flake.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-09 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643824#comment-16643824
 ] 

Scott Wegner commented on BEAM-5680:


Latest run succeeded, this appears to be fixed: 
https://builds.apache.org/job/beam_PostCommit_Website_Publish/146/

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: currently-failing
> Fix For: Not applicable
>
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]).
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5680.

   Resolution: Fixed
Fix Version/s: Not applicable

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: currently-failing
> Fix For: Not applicable
>
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]).
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5688) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on githubPullRequestId assert

2018-10-09 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643817#comment-16643817
 ] 

Scott Wegner commented on BEAM-5688:


[~udim] fyi

> [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on 
> githubPullRequestId assert
> --
>
> Key: BEAM-5688
> URL: https://issues.apache.org/jira/browse/BEAM-5688
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/20/]
>  * [Gradle Build Scan|https://gradle.com/s/7mqwgjegf5hge]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L234]
> Initial investigation:
> This is a problem with how the website gradle scripts are implemented to 
> accept an githubPullRequestId. The Cron job will not have an associated PR, 
> so this currently fails.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5691) testWebsite failed owing to several url 404 not found

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5691.

   Resolution: Duplicate
Fix Version/s: Not applicable

> testWebsite failed owing to several url 404 not found
> -
>
> Key: BEAM-5691
> URL: https://issues.apache.org/jira/browse/BEAM-5691
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Boyuan Zhang
>Priority: Major
> Fix For: Not applicable
>
>
> test log: 
> https://builds.apache.org/job/beam_PreCommit_Website_Commit/262/console



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to pip download flake

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5683:
---
Summary: [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] 
Fails due to pip download flake  (was: [beam_PostCommit_Py_VR_Dataflow] 
[test_multiple_empty_outputs] Failure summary)

> [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to 
> pip download flake
> --
>
> Key: BEAM-5683
> URL: https://issues.apache.org/jira/browse/BEAM-5683
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-harness, test-failures
>Reporter: Scott Wegner
>Assignee: Ankur Goenka
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
>  * [Test source 
> code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390]
> Initial investigation:
> Seems to be failing on pip download.
> ==
> ERROR: test_multiple_empty_outputs 
> (apache_beam.transforms.ptransform_test.PTransformTest)
> --
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
>  line 277, in test_multiple_empty_outputs
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
>  line 104, in run
> result = super(TestPipeline, self).run(test_runner_api)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 403, in run
> self.to_runner_api(), self.runner, self._options).run(False)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 416, in run
> return self.runner.run_pipeline(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
>  line 50, in run_pipeline
> self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
>  line 389, in run_pipeline
> self.dataflow_client.create_job(self.job), self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
>  line 184, in wrapper
> return fun(*args, **kwargs)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 490, in create_job
> self.create_job_description(job)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 519, in create_job_description
> resources = self._stage_resour
> ces(job.options)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 452, in _stage_resources
> staging_location=google_cloud_options.staging_location)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 161, in stage_job_resources
> requirements_cache_path)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 411, in _populate_requirements_cache
> processes.check_call(cmd_args)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
>  line 46, in check_call
> return subprocess.check_call(*args, **kwargs)
>   File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
> raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command 
> '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
>  '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
> 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', 
> 

[jira] [Commented] (BEAM-5688) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on githubPullRequestId assert

2018-10-09 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643659#comment-16643659
 ] 

Scott Wegner commented on BEAM-5688:


I have https://github.com/apache/beam/pull/6608 out to fix this, but the 
current version doesn't work.

> [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on 
> githubPullRequestId assert
> --
>
> Key: BEAM-5688
> URL: https://issues.apache.org/jira/browse/BEAM-5688
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/20/]
>  * [Gradle Build Scan|https://gradle.com/s/7mqwgjegf5hge]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L234]
> Initial investigation:
> This is a problem with how the website gradle scripts are implemented to 
> accept an githubPullRequestId. The Cron job will not have an associated PR, 
> so this currently fails.
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5688) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on githubPullRequestId assert

2018-10-09 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5688:
--

 Summary: [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 
Fails on githubPullRequestId assert
 Key: BEAM-5688
 URL: https://issues.apache.org/jira/browse/BEAM-5688
 Project: Beam
  Issue Type: Bug
  Components: test-failures
Reporter: Scott Wegner
Assignee: Alan Myrvold


_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/20/]
 * [Gradle Build Scan|https://gradle.com/s/7mqwgjegf5hge]
 * [Test source 
code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L234]

Initial investigation:

This is a problem with how the website gradle scripts are implemented to accept 
an githubPullRequestId. The Cron job will not have an associated PR, so this 
currently fails.


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5281) There are several deadlinks in beam-site, please removed.

2018-10-09 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5281.

   Resolution: Duplicate
Fix Version/s: Not applicable

Dupe of BEAM-5681; there's something broken in the pre-commit scripts.

> There are several deadlinks in beam-site, please removed.
> -
>
> Key: BEAM-5281
> URL: https://issues.apache.org/jira/browse/BEAM-5281
> Project: Beam
>  Issue Type: Bug
>  Components: website
>Reporter: Boyuan Zhang
>Assignee: Melissa Pashniak
>Priority: Major
> Fix For: Not applicable
>
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> Deadlinks in beam-site cause nightly build failed: 
> https://scans.gradle.com/s/nzwfwj6iqlgrg/console-log?task=:beam-website:testWebsite#L13



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

2018-10-09 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643606#comment-16643606
 ] 

Scott Wegner commented on BEAM-5683:


[~pabloem] / [~robertwb] can either of you help out?

bq. Can we access pip subprocess logs?

> [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
> --
>
> Key: BEAM-5683
> URL: https://issues.apache.org/jira/browse/BEAM-5683
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-harness, test-failures
>Reporter: Scott Wegner
>Assignee: Ankur Goenka
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
>  * [Test source 
> code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390]
> Initial investigation:
> Seems to be failing on pip download.
> ==
> ERROR: test_multiple_empty_outputs 
> (apache_beam.transforms.ptransform_test.PTransformTest)
> --
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
>  line 277, in test_multiple_empty_outputs
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
>  line 104, in run
> result = super(TestPipeline, self).run(test_runner_api)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 403, in run
> self.to_runner_api(), self.runner, self._options).run(False)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 416, in run
> return self.runner.run_pipeline(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
>  line 50, in run_pipeline
> self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
>  line 389, in run_pipeline
> self.dataflow_client.create_job(self.job), self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
>  line 184, in wrapper
> return fun(*args, **kwargs)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 490, in create_job
> self.create_job_description(job)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 519, in create_job_description
> resources = self._stage_resour
> ces(job.options)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 452, in _stage_resources
> staging_location=google_cloud_options.staging_location)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 161, in stage_job_resources
> requirements_cache_path)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 411, in _populate_requirements_cache
> processes.check_call(cmd_args)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
>  line 46, in check_call
> return subprocess.check_call(*args, **kwargs)
>   File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
> raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command 
> '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
>  '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
> 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', 
> ':all:']' returned non-zero exit status 1
> 
> _After you've filled out the above details, please 

[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642309#comment-16642309
 ] 

Scott Wegner commented on BEAM-5683:


[~angoenka] I see you [previously worked on the 
[_populate_requirements_cache|https://github.com/apache/beam/pull/5251] code. 
Can you help investigate this flake or find a more appropriate owner?

> [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
> --
>
> Key: BEAM-5683
> URL: https://issues.apache.org/jira/browse/BEAM-5683
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-harness, test-failures
>Reporter: Scott Wegner
>Assignee: Robert Bradshaw
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
>  * [Test source 
> code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390]
> Initial investigation:
> Seems to be failing on pip download.
> ==
> ERROR: test_multiple_empty_outputs 
> (apache_beam.transforms.ptransform_test.PTransformTest)
> --
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
>  line 277, in test_multiple_empty_outputs
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
>  line 104, in run
> result = super(TestPipeline, self).run(test_runner_api)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 403, in run
> self.to_runner_api(), self.runner, self._options).run(False)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 416, in run
> return self.runner.run_pipeline(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
>  line 50, in run_pipeline
> self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
>  line 389, in run_pipeline
> self.dataflow_client.create_job(self.job), self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
>  line 184, in wrapper
> return fun(*args, **kwargs)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 490, in create_job
> self.create_job_description(job)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 519, in create_job_description
> resources = self._stage_resour
> ces(job.options)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 452, in _stage_resources
> staging_location=google_cloud_options.staging_location)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 161, in stage_job_resources
> requirements_cache_path)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 411, in _populate_requirements_cache
> processes.check_call(cmd_args)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
>  line 46, in check_call
> return subprocess.check_call(*args, **kwargs)
>   File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
> raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command 
> '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
>  '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
> 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', 

[jira] [Assigned] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

2018-10-08 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5683:
--

Assignee: Ankur Goenka  (was: Robert Bradshaw)

> [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
> --
>
> Key: BEAM-5683
> URL: https://issues.apache.org/jira/browse/BEAM-5683
> Project: Beam
>  Issue Type: Bug
>  Components: sdk-py-harness, test-failures
>Reporter: Scott Wegner
>Assignee: Ankur Goenka
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
>  * [Test source 
> code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390]
> Initial investigation:
> Seems to be failing on pip download.
> ==
> ERROR: test_multiple_empty_outputs 
> (apache_beam.transforms.ptransform_test.PTransformTest)
> --
> Traceback (most recent call last):
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
>  line 277, in test_multiple_empty_outputs
> pipeline.run()
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
>  line 104, in run
> result = super(TestPipeline, self).run(test_runner_api)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 403, in run
> self.to_runner_api(), self.runner, self._options).run(False)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
>  line 416, in run
> return self.runner.run_pipeline(self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
>  line 50, in run_pipeline
> self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
>  line 389, in run_pipeline
> self.dataflow_client.create_job(self.job), self)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
>  line 184, in wrapper
> return fun(*args, **kwargs)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 490, in create_job
> self.create_job_description(job)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 519, in create_job_description
> resources = self._stage_resour
> ces(job.options)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
>  line 452, in _stage_resources
> staging_location=google_cloud_options.staging_location)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 161, in stage_job_resources
> requirements_cache_path)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
>  line 411, in _populate_requirements_cache
> processes.check_call(cmd_args)
>   File 
> "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
>  line 46, in check_call
> return subprocess.check_call(*args, **kwargs)
>   File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
> raise CalledProcessError(retcode, cmd)
> CalledProcessError: Command 
> '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
>  '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
> 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', 
> ':all:']' returned non-zero exit status 1
> 
> _After you've filled out the above details, please [assign the issue to an 
> 

[jira] [Updated] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

2018-10-08 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5683:
---
Description: 
_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
 * [Test source 
code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390]

Initial investigation:

Seems to be failing on pip download.


==
ERROR: test_multiple_empty_outputs 
(apache_beam.transforms.ptransform_test.PTransformTest)
--
Traceback (most recent call last):
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
 line 277, in test_multiple_empty_outputs
pipeline.run()
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
 line 104, in run
result = super(TestPipeline, self).run(test_runner_api)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 403, in run
self.to_runner_api(), self.runner, self._options).run(False)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 416, in run
return self.runner.run_pipeline(self)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
 line 50, in run_pipeline
self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
 line 389, in run_pipeline
self.dataflow_client.create_job(self.job), self)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
 line 184, in wrapper
return fun(*args, **kwargs)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 490, in create_job
self.create_job_description(job)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 519, in create_job_description
resources = self._stage_resour
ces(job.options)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 452, in _stage_resources
staging_location=google_cloud_options.staging_location)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 161, in stage_job_resources
requirements_cache_path)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 411, in _populate_requirements_cache
processes.check_call(cmd_args)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
 line 46, in check_call
return subprocess.check_call(*args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command 
'['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
 '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']' 
returned non-zero exit status 1



_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._

  was:
_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
 * [Test source code|TODO]

Initial investigation:

Seems to be failing on pip download.



[jira] [Created] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary

2018-10-08 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5683:
--

 Summary: [beam_PostCommit_Py_VR_Dataflow] 
[test_multiple_empty_outputs] Failure summary
 Key: BEAM-5683
 URL: https://issues.apache.org/jira/browse/BEAM-5683
 Project: Beam
  Issue Type: Bug
  Components: sdk-py-harness, test-failures
Reporter: Scott Wegner
Assignee: Robert Bradshaw


_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests]
 * [Test source code|TODO]

Initial investigation:

Seems to be failing on pip download.


==
ERROR: test_multiple_empty_outputs 
(apache_beam.transforms.ptransform_test.PTransformTest)
--
Traceback (most recent call last):
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py",
 line 277, in test_multiple_empty_outputs
pipeline.run()
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py",
 line 104, in run
result = super(TestPipeline, self).run(test_runner_api)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 403, in run
self.to_runner_api(), self.runner, self._options).run(False)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py",
 line 416, in run
return self.runner.run_pipeline(self)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py",
 line 50, in run_pipeline
self.result = super(TestDataflowRunner, self).run_pipeline(pipeline)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py",
 line 389, in run_pipeline
self.dataflow_client.create_job(self.job), self)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py",
 line 184, in wrapper
return fun(*args, **kwargs)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 490, in create_job
self.create_job_description(job)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 519, in create_job_description
resources = self._stage_resour
ces(job.options)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py",
 line 452, in _stage_resources
staging_location=google_cloud_options.staging_location)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 161, in stage_job_resources
requirements_cache_path)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py",
 line 411, in _populate_requirements_cache
processes.check_call(cmd_args)
  File 
"/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py",
 line 46, in check_call
return subprocess.check_call(*args, **kwargs)
  File "/usr/lib/python2.7/subprocess.py", line 541, in check_call
raise CalledProcessError(retcode, cmd)
CalledProcessError: Command 
'['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python',
 '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 
'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']' 
returned non-zero exit status 1



_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642132#comment-16642132
 ] 

Scott Wegner commented on BEAM-5681:


[beam_PreCommit_Website_Stage_GCS_Cron|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/]
 seems to be [failing for the same 
reason|https://scans.gradle.com/s/k2ggapk6a3hmm/console-log?task=:beam-website:stageWebsite],
 but with a better error message:

{{Build file 
'/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Website_Stage_GCS_Cron/src/website/build.gradle'
 line: 234}}
{{assert githubPullRequestId.isInteger()}}
{{   |   |}}
{{   ${ghprbPullId}  false}}

> [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
> ---
>
> Key: BEAM-5681
> URL: https://issues.apache.org/jira/browse/BEAM-5681
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27]
>  * [Test source 
> code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140]
> Initial investigation:
> It seems every link is failing. Same error message:
> {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}}
> {{  *  http:// is an invalid URL (line 76)}}
> {{ }}
> {{ src="//images/beam_logo_navbar.png">}}
> {{  }}
> The _Cron version of this job has been failing consistently since 10/5 
> ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/])
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642127#comment-16642127
 ] 

Scott Wegner commented on BEAM-5681:


I wonder if it's due to the \-PgithubPullRequestId=${ghprbPullId} parameter to 
the job. The 'Cron' version of the job doesn't run as part of a pull-request, 
so this parameter will be empty.

I'm able to repro this by running locally and explicitly setting the parameter 
to whitespace (\-PgithubPullRequestId=" "):  
[https://gradle.com/s/th6k66yjmmxj4]

 

[~alanmyrvold] could you please investigate?

> [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
> ---
>
> Key: BEAM-5681
> URL: https://issues.apache.org/jira/browse/BEAM-5681
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Alan Myrvold
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27]
>  * [Test source 
> code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140]
> Initial investigation:
> It seems every link is failing. Same error message:
> {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}}
> {{  *  http:// is an invalid URL (line 76)}}
> {{ }}
> {{ src="//images/beam_logo_navbar.png">}}
> {{  }}
> The _Cron version of this job has been failing consistently since 10/5 
> ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/])
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks

2018-10-08 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5681:
--

 Summary: [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL 
checks
 Key: BEAM-5681
 URL: https://issues.apache.org/jira/browse/BEAM-5681
 Project: Beam
  Issue Type: Bug
  Components: test-failures, website
Reporter: Scott Wegner
Assignee: Alan Myrvold


_Use this form to file an issue for test failure:_
 * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27]
 * [Test source 
code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140]

Initial investigation:

It seems every link is failing. Same error message:

{{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}}
{{  *  http:// is an invalid URL (line 76)}}
{{ }}
{{}}
{{  }}

The _Cron version of this job has been failing consistently since 10/5 
([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/])


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642113#comment-16642113
 ] 

Scott Wegner commented on BEAM-5680:


I've filed an issue with INFRA.

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-08 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5680:
---
Description: 
_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
 * [Test source 
code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]

Initial investigation:

Failing consistently since 10/7 5am until now ([test 
history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]).


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._

  was:
_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
 * [Test source 
code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]

Initial investigation:

Failing consistently since 10/7 5am until now ([test 
history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._


> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]).
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Assigned] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-08 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reassigned BEAM-5680:
--

Assignee: Scott Wegner  (was: Melissa Pashniak)

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642098#comment-16642098
 ] 

Scott Wegner commented on BEAM-5680:


Another recent job failure, 
[Mesos-Websitebot|https://builds.apache.org/job/Mesos-Websitebot/lastSuccessfulBuild/],
 uses gitbox and has a similar failure messages:

{{fatal: Authentication failed for 
'https://gitbox.apache.org/repos/asf/mesos-site.git/'}}
{{Build step 'Execute shell' marked build as failure}}

I'm going to reach out to Apache INFRA.

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642096#comment-16642096
 ] 

Scott Wegner commented on BEAM-5680:


I see other successful jobs run in the 
[git-website|https://builds.apache.org/label/git-websites/] label 
([Solr-reference-guide-master|https://builds.apache.org/job/Solr-reference-guide-master/lastSuccessfulBuild/],
 
[hbase_generate_website|https://builds.apache.org/job/hbase_generate_website/lastSuccessfulBuild/],
 
[Solr-reference-guide-7.x|https://builds.apache.org/job/Solr-reference-guide-7.x/lastSuccessfulBuild/],
 [Incubator 
Site|https://builds.apache.org/job/Incubator%20Site/lastSuccessfulBuild/]), but 
none of them seem to be using gitbox.

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642078#comment-16642078
 ] 

Scott Wegner commented on BEAM-5680:


Nothing posted to https://status.apache.org/

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642075#comment-16642075
 ] 

Scott Wegner commented on BEAM-5680:


/cc [~alanmyrvold]

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-08 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5680:
--

 Summary: [beam_PostCommit_Website_Publish] [:publishWebsite] git 
push 'Authentication failed'
 Key: BEAM-5680
 URL: https://issues.apache.org/jira/browse/BEAM-5680
 Project: Beam
  Issue Type: Bug
  Components: test-failures, website
Reporter: Scott Wegner
Assignee: Melissa Pashniak


_Use this form to file an issue for test failure:_
 * [Jenkins 
Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
 * [Gradle Build 
Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
 * [Test source 
code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]

Initial investigation:

Failing consistently since 10/7 5am until now ([test 
history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].


_After you've filled out the above details, please [assign the issue to an 
individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
 Assignee should [treat test failures as 
high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
 helping to fix the issue or find a more appropriate owner. See [Apache Beam 
Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642077#comment-16642077
 ] 

Scott Wegner commented on BEAM-5680:


The job correctly ran on 
[website1|https://builds.apache.org/computer/websites1] Jenkins node, which is 
the only thing that should be required to push to gitbox.

> [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication 
> failed'
> 
>
> Key: BEAM-5680
> URL: https://issues.apache.org/jira/browse/BEAM-5680
> Project: Beam
>  Issue Type: Bug
>  Components: test-failures, website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>  Labels: currently-failing
>
> _Use this form to file an issue for test failure:_
>  * [Jenkins 
> Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/]
>  * [Gradle Build 
> Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite]
>  * [Test source 
> code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221]
> Initial investigation:
> Failing consistently since 10/7 5am until now ([test 
> history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend].
> 
> _After you've filled out the above details, please [assign the issue to an 
> individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist].
>  Assignee should [treat test failures as 
> high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test],
>  helping to fix the issue or find a more appropriate owner. See [Apache Beam 
> Post-Commit 
> Policies|https://beam.apache.org/contribute/postcommits-policies]._



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642035#comment-16642035
 ] 

Scott Wegner commented on BEAM-5678:


The [html-proofer|https://github.com/gjtorikian/html-proofer#ignoring-content] 
extension that we use to validate URLs allows specifying a special 
[{{data-proofer-ignore}} 
|https://github.com/gjtorikian/html-proofer#ignoring-content] tag in the HTML 
link to disable validation. We could use the custom redirect page functionality 
to specify the tag.

This should be a sufficient work-around. Ideally we would like to validate that 
our redirects do work, but since the redirect pages we're excluding are all 
generated from the Jekyll plugin, the only reason they wouldn't work is due to 
a bug in the plugin itself.

> Jeckyll redirect_from incompatible with HTMLProofer url validation
> --
>
> Key: BEAM-5678
> URL: https://issues.apache.org/jira/browse/BEAM-5678
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Priority: Minor
>
> Renaming a URL and using the Jekyll {{redirect_from}} feature breaks 
> pre-commit validation.
> We hit this on 
> [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]:
> bq. The scenario for using a redirect is if you're moving {{pageA.md}} to 
> {{pageB.md}}; you would rename the file to {{pageB.md}} and add 
> {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} 
> with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} 
> doesn't exist on the livesite yet, so it will always fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642031#comment-16642031
 ] 

Scott Wegner commented on BEAM-5678:


[~alanmyrvold], [~udim] and I spent some time investigating this and didn't 
find easy work-arounds.

The [jekyll-redirect-from|https://github.com/jekyll/jekyll-redirect-from] 
plugin responsible for creating the redirect pages uses absolute paths for the 
"redirect to" URL, which is what causes issues. The plugin allows providing a 
custom page template, but only provides the absolute redirect-to URL as a 
variable.

> Jeckyll redirect_from incompatible with HTMLProofer url validation
> --
>
> Key: BEAM-5678
> URL: https://issues.apache.org/jira/browse/BEAM-5678
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Priority: Minor
>
> Renaming a URL and using the Jekyll {{redirect_from}} feature breaks 
> pre-commit validation.
> We hit this on 
> [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]:
> bq. The scenario for using a redirect is if you're moving {{pageA.md}} to 
> {{pageB.md}}; you would rename the file to {{pageB.md}} and add 
> {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} 
> with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} 
> doesn't exist on the livesite yet, so it will always fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation

2018-10-08 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5678:
--

 Summary: Jeckyll redirect_from incompatible with HTMLProofer url 
validation
 Key: BEAM-5678
 URL: https://issues.apache.org/jira/browse/BEAM-5678
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner


Renaming a URL and using the Jekyll {{redirect_from}} feature breaks pre-commit 
validation.

We hit this on 
[PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]:

bq. The scenario for using a redirect is if you're moving {{pageA.md}} to 
{{pageB.md}}; you would rename the file to {{pageB.md}} and add 
{{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} 
with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} 
doesn't exist on the livesite yet, so it will always fail.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642024#comment-16642024
 ] 

Scott Wegner commented on BEAM-5669:


re: whether fixing up history would break the website publishing. It appears 
not. The code for pulling down latest changes is 
[here|https://github.com/apache/infrastructure-puppet/blob/bae50f68a3215c3db80e8ac1bfe4bfdf45b72663/modules/gitwcsub/files/app/gitwcsub.py#L224],
 which is essentially:

{{git fetch origin/asf-site && git reset --hard origin/asf-site}}

Neither {{fetch}} or {{reset --hard}} will complain if history has been 
re-written.

> Empty commits pushed by PostCommit_Website_Publish jenkins job
> --
>
> Key: BEAM-5669
> URL: https://issues.apache.org/jira/browse/BEAM-5669
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: 2.8.0
>Reporter: Alan Myrvold
>Assignee: Alan Myrvold
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The isClean() call in 
> https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165
>  is returning false due to unstaged .gradle and build files.
> The asf-site branch needs a .gitignore, which can be the same as the master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642013#comment-16642013
 ] 

Scott Wegner commented on BEAM-5669:


The implications of these empty commits is extra commit spam for notifications, 
during fetch, and wasted CPU cycles for the Apache daemon publishing the 
website.

After we get this fixed, a secondary question is whether we should go back and 
cleanup the empty commits. It would be easy to do with a git command, but it 
would re-write the git history. I'm not sure if this would cause any issues 
with the Apache gitpubsub publishing agent.

> Empty commits pushed by PostCommit_Website_Publish jenkins job
> --
>
> Key: BEAM-5669
> URL: https://issues.apache.org/jira/browse/BEAM-5669
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: 2.8.0
>Reporter: Alan Myrvold
>Assignee: Alan Myrvold
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The isClean() call in 
> https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165
>  is returning false due to unstaged .gradle and build files.
> The asf-site branch needs a .gitignore, which can be the same as the master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Reopened] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job

2018-10-08 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner reopened BEAM-5669:


[~thw] [pointed 
out|https://issues.apache.org/jira/browse/BEAM-5671?focusedCommentId=16640524=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16640524]
 this is still broken. [~alanmyrvold] can you take a look?

> Empty commits pushed by PostCommit_Website_Publish jenkins job
> --
>
> Key: BEAM-5669
> URL: https://issues.apache.org/jira/browse/BEAM-5669
> Project: Beam
>  Issue Type: Bug
>  Components: build-system
>Affects Versions: 2.8.0
>Reporter: Alan Myrvold
>Assignee: Alan Myrvold
>Priority: Major
> Fix For: 2.8.0
>
>  Time Spent: 0.5h
>  Remaining Estimate: 0h
>
> The isClean() call in 
> https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165
>  is returning false due to unstaged .gradle and build files.
> The asf-site branch needs a .gitignore, which can be the same as the master



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Commented] (BEAM-5671) Website automation fit-and-finish

2018-10-08 Thread Scott Wegner (JIRA)


[ 
https://issues.apache.org/jira/browse/BEAM-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642006#comment-16642006
 ] 

Scott Wegner commented on BEAM-5671:


Thanks for the heads-up. [~alanmyrvold] tried to fix this with 
https://github.com/apache/beam/pull/6591, but I suppose the fix didn't work-- 
I'll reopen https://issues.apache.org/jira/browse/BEAM-5669

> Website automation fit-and-finish
> -
>
> Key: BEAM-5671
> URL: https://issues.apache.org/jira/browse/BEAM-5671
> Project: Beam
>  Issue Type: Wish
>  Components: website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>
> We are wrapping up the work for migrating the Beam Website 
> (https://s.apache.org/beam-site-automation); there are a few fit-and-finish 
> items that remain. If you notice things that should be improved, please add 
> them here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5672) Staged URL should be posted to GitHub PR

2018-10-05 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5672:
--

 Summary: Staged URL should be posted to GitHub PR
 Key: BEAM-5672
 URL: https://issues.apache.org/jira/browse/BEAM-5672
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner


Website PRs are automatically tested and staged via Jenkins to a location on 
GCS: https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Commit/

However, it's difficult to find the URL for the staged job; it's buried inside 
the Jenkins job, inside the Gradle scan, inside the log.

I believe there's an API for posting back a context message to the pull 
request. If possible, we should push the URL so it's easily viewable from the 
pull request.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5459) Publish javadocs/pydocs to an external source, i.e. buildbot

2018-10-05 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5459:
---
Issue Type: Task  (was: Sub-task)
Parent: (was: BEAM-4493)

> Publish javadocs/pydocs to an external source, i.e. buildbot
> 
>
> Key: BEAM-5459
> URL: https://issues.apache.org/jira/browse/BEAM-5459
> Project: Beam
>  Issue Type: Task
>  Components: website
>Reporter: Scott Wegner
>Priority: Minor
>
> Javadocs and Pydocs are generated for the website at every release, and the 
> generated content gets committed to the git repository in order to publish 
> via the githubpubsub publishing mechanism. Keeping all of this generated 
> content in git is cumbersome and seems unnecessary. Alternatives exist, for 
> example the Flink project uses a buildbot job to build and publish their 
> javadocs:
> https://ci.apache.org/projects/flink/flink-docs-release-1.5/
> The buildbot configuration is here (requires committer access):
> https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/flink.conf



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5459) Publish javadocs/pydocs to an external source, i.e. buildbot

2018-10-05 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5459:
---
Issue Type: Sub-task  (was: Task)
Parent: BEAM-5671

> Publish javadocs/pydocs to an external source, i.e. buildbot
> 
>
> Key: BEAM-5459
> URL: https://issues.apache.org/jira/browse/BEAM-5459
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Priority: Minor
>
> Javadocs and Pydocs are generated for the website at every release, and the 
> generated content gets committed to the git repository in order to publish 
> via the githubpubsub publishing mechanism. Keeping all of this generated 
> content in git is cumbersome and seems unnecessary. Alternatives exist, for 
> example the Flink project uses a buildbot job to build and publish their 
> javadocs:
> https://ci.apache.org/projects/flink/flink-docs-release-1.5/
> The buildbot configuration is here (requires committer access):
> https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/flink.conf



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-4497) Add pages for master Javadocs / Pydocs and incorporate into post-commit job

2018-10-05 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-4497:
---
Issue Type: Sub-task  (was: Task)
Parent: BEAM-5671

> Add pages for master Javadocs / Pydocs and incorporate into post-commit job
> ---
>
> Key: BEAM-4497
> URL: https://issues.apache.org/jira/browse/BEAM-4497
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: beam-site-automation-reliability
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-5671) Website automation fit-and-finish

2018-10-05 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-5671:
---
Issue Type: Wish  (was: Sub-task)
Parent: (was: BEAM-4493)

> Website automation fit-and-finish
> -
>
> Key: BEAM-5671
> URL: https://issues.apache.org/jira/browse/BEAM-5671
> Project: Beam
>  Issue Type: Wish
>  Components: website
>Reporter: Scott Wegner
>Assignee: Melissa Pashniak
>Priority: Major
>
> We are wrapping up the work for migrating the Beam Website 
> (https://s.apache.org/beam-site-automation); there are a few fit-and-finish 
> items that remain. If you notice things that should be improved, please add 
> them here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Updated] (BEAM-4497) Add pages for master Javadocs / Pydocs and incorporate into post-commit job

2018-10-05 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner updated BEAM-4497:
---
Issue Type: Task  (was: Sub-task)
Parent: (was: BEAM-4493)

> Add pages for master Javadocs / Pydocs and incorporate into post-commit job
> ---
>
> Key: BEAM-4497
> URL: https://issues.apache.org/jira/browse/BEAM-4497
> Project: Beam
>  Issue Type: Task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: beam-site-automation-reliability
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Created] (BEAM-5671) Website automation fit-and-finish

2018-10-05 Thread Scott Wegner (JIRA)
Scott Wegner created BEAM-5671:
--

 Summary: Website automation fit-and-finish
 Key: BEAM-5671
 URL: https://issues.apache.org/jira/browse/BEAM-5671
 Project: Beam
  Issue Type: Sub-task
  Components: website
Reporter: Scott Wegner
Assignee: Melissa Pashniak


We are wrapping up the work for migrating the Beam Website 
(https://s.apache.org/beam-site-automation); there are a few fit-and-finish 
items that remain. If you notice things that should be improved, please add 
them here.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-4499) Migrate Apache website publishing to use apache/beam asf-site branch

2018-10-05 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-4499.

   Resolution: Fixed
Fix Version/s: 2.8.0

> Migrate Apache website publishing to use apache/beam asf-site branch
> 
>
> Key: BEAM-4499
> URL: https://issues.apache.org/jira/browse/BEAM-4499
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Scott Wegner
>Assignee: Scott Wegner
>Priority: Major
>  Labels: beam-site-automation-reliability
> Fix For: 2.8.0
>
>  Time Spent: 2h
>  Remaining Estimate: 0h
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


[jira] [Resolved] (BEAM-5368) Re-enable apache/website build once it is stable

2018-10-05 Thread Scott Wegner (JIRA)


 [ 
https://issues.apache.org/jira/browse/BEAM-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Scott Wegner resolved BEAM-5368.

   Resolution: Fixed
Fix Version/s: Not applicable

> Re-enable apache/website build once it is stable
> 
>
> Key: BEAM-5368
> URL: https://issues.apache.org/jira/browse/BEAM-5368
> Project: Beam
>  Issue Type: Sub-task
>  Components: website
>Reporter: Batkhuyag Batsaikhan
>Assignee: Scott Wegner
>Priority: Major
> Fix For: Not applicable
>
>
> We are disabling beam check in 
> [https://github.com/apache/beam/pull/6370|https://github.com/apache/beam/pull/6370]
>  due to https://issues.apache.org/jira/browse/BEAM-5367.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)


  1   2   3   4   5   6   >