[jira] [Commented] (BEAM-5735) Contributor Guide Improvements
[ https://issues.apache.org/jira/browse/BEAM-5735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648557#comment-16648557 ] Scott Wegner commented on BEAM-5735: I made some small boilerplate changes while we were discussing these in person. It's not nearly ready to check-in, but in case it's useful: https://github.com/swegner/beam/commit/06aee6b3903020e04b7ce4000ba71be525e04721 > Contributor Guide Improvements > -- > > Key: BEAM-5735 > URL: https://issues.apache.org/jira/browse/BEAM-5735 > Project: Beam > Issue Type: Improvement > Components: website >Reporter: Scott Wegner >Assignee: Alan Myrvold >Priority: Major > > This is a wish-list for improvements to the Beam contributor guide. > Many thanks to [~rohdesam] for the feedback which helped shape this list. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5740) Refactor permissions section into bullet-points
[ https://issues.apache.org/jira/browse/BEAM-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-5740: -- Assignee: Scott Wegner > Refactor permissions section into bullet-points > --- > > Key: BEAM-5740 > URL: https://issues.apache.org/jira/browse/BEAM-5740 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > > The permissions section has good content, but it's not easily browseable if > you're looking for a specific thing (i.e. Slack permissions). We should > refactor it into bullet points. > For permissions that require reaching out via email/Slack, we should link to > some previous example. It lowers the barrier to entry if a new contributor > can copy/paste some existing template. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5743) Move "works in progress" out of getting started guide
[ https://issues.apache.org/jira/browse/BEAM-5743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-5743: -- Assignee: Scott Wegner > Move "works in progress" out of getting started guide > - > > Key: BEAM-5743 > URL: https://issues.apache.org/jira/browse/BEAM-5743 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > > These aren't relevant to a new contributor, and it makes the contributor > guide less focused. They should be moved to a separate page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5741) Move "Contact Us" to a top-level link
[ https://issues.apache.org/jira/browse/BEAM-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-5741: -- Assignee: (was: Melissa Pashniak) > Move "Contact Us" to a top-level link > - > > Key: BEAM-5741 > URL: https://issues.apache.org/jira/browse/BEAM-5741 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Priority: Major > > It should be very easy to figure out how to get in touch with community. > "Contact Us" should be a top-level link on the page. > The page can also be improved with: > * Some basic text on how to use subscribe / unsubscribe links > * Recommendations on how to use various communications channels (Slack for > quick questions, dev@ for longer conversations. And all decisions should make > it back to dev@) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5740) Refactor permissions section into bullet-points
[ https://issues.apache.org/jira/browse/BEAM-5740?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5740: --- Description: The permissions section has good content, but it's not easily browseable if you're looking for a specific thing (i.e. Slack permissions). We should refactor it into bullet points. For permissions that require reaching out via email/Slack, we should link to some previous example. It lowers the barrier to entry if a new contributor can copy/paste some existing template. was:The permissions section has good content, but it's not easily browseable if you're looking for a specific thing (i.e. Slack permissions). We should refactor it into bullet points. > Refactor permissions section into bullet-points > --- > > Key: BEAM-5740 > URL: https://issues.apache.org/jira/browse/BEAM-5740 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Priority: Major > > The permissions section has good content, but it's not easily browseable if > you're looking for a specific thing (i.e. Slack permissions). We should > refactor it into bullet points. > For permissions that require reaching out via email/Slack, we should link to > some previous example. It lowers the barrier to entry if a new contributor > can copy/paste some existing template. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5743) Move "works in progress" out of getting started guide
Scott Wegner created BEAM-5743: -- Summary: Move "works in progress" out of getting started guide Key: BEAM-5743 URL: https://issues.apache.org/jira/browse/BEAM-5743 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner These aren't relevant to a new contributor, and it makes the contributor guide less focused. They should be moved to a separate page. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5742) Add expectations for code reviews
Scott Wegner created BEAM-5742: -- Summary: Add expectations for code reviews Key: BEAM-5742 URL: https://issues.apache.org/jira/browse/BEAM-5742 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner There should be a dedicated section about how we to code reviews, and expectations for contributors / reviewers. Things like: * PR's should have linked JIRA * Code changes should also include tests * Small changes are easier to review * How to find a reviewer, when you can expect reviewer to engage * How automatic test runs work (PreCommits run by on each commit; build / test should be passing before asking for a reviewer) * How to re-run tests, and how/when to run Post-Commits -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5741) Move "Contact Us" to a top-level link
Scott Wegner created BEAM-5741: -- Summary: Move "Contact Us" to a top-level link Key: BEAM-5741 URL: https://issues.apache.org/jira/browse/BEAM-5741 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner Assignee: Melissa Pashniak It should be very easy to figure out how to get in touch with community. "Contact Us" should be a top-level link on the page. The page can also be improved with: * Some basic text on how to use subscribe / unsubscribe links * Recommendations on how to use various communications channels (Slack for quick questions, dev@ for longer conversations. And all decisions should make it back to dev@) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5740) Refactor permissions section into bullet-points
Scott Wegner created BEAM-5740: -- Summary: Refactor permissions section into bullet-points Key: BEAM-5740 URL: https://issues.apache.org/jira/browse/BEAM-5740 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner The permissions section has good content, but it's not easily browseable if you're looking for a specific thing (i.e. Slack permissions). We should refactor it into bullet points. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5739) Contributor Story: "Submitting your first PR"
Scott Wegner created BEAM-5739: -- Summary: Contributor Story: "Submitting your first PR" Key: BEAM-5739 URL: https://issues.apache.org/jira/browse/BEAM-5739 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner We should write the user story for "Submitting your first PR", with prescriptive steps on getting started. It should include: * Forking the repo and setting up the dev environment * How to build/test * Choosing an IDE * language / SDK-specific tips + website * "When will my changes go live?" -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5738) Refactor Contributor Guide into "Stories"
Scott Wegner created BEAM-5738: -- Summary: Refactor Contributor Guide into "Stories" Key: BEAM-5738 URL: https://issues.apache.org/jira/browse/BEAM-5738 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner The Contributor Guide has become a dumping ground for topics that are maybe relevant to contributors. It's long and unorganized and it doesn't really tell a contributor how to get started. We should refactor the contributor guide into "stories", like "How to submit your first PR", "How to contribute to the website", "How to propose a new feature", etc. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5737) Create a contributor FAQ page
Scott Wegner created BEAM-5737: -- Summary: Create a contributor FAQ page Key: BEAM-5737 URL: https://issues.apache.org/jira/browse/BEAM-5737 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner This should be an easy "dumping ground" place to add documentation, pretty much about anything. It should be low-barrier to add new content or update existing. High-quality or frequently-used topics can be promoted to top level pages. It probably makes sense to put this on the wiki. But it should be linked very prominently from the top of the Contributor Guide -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5736) List prerequisite knowledge with links to external docs
Scott Wegner created BEAM-5736: -- Summary: List prerequisite knowledge with links to external docs Key: BEAM-5736 URL: https://issues.apache.org/jira/browse/BEAM-5736 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner Assignee: Scott Wegner Our contributor guide makes some assumptions about prior knowledge. We should be explicit about what we expect contributors to already know, and give links to places where they can learn more if necessary. Examples: * Git & GitHub workflows * Beam (understand what it does, what it's for, how it fits into ecosystem) * JDK 8 installed * Windows / Mac / Linux dev environment (be explicit about what we support) -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5735) Contributor Guide Improvements
Scott Wegner created BEAM-5735: -- Summary: Contributor Guide Improvements Key: BEAM-5735 URL: https://issues.apache.org/jira/browse/BEAM-5735 Project: Beam Issue Type: Improvement Components: website Reporter: Scott Wegner Assignee: Alan Myrvold This is a wish-list for improvements to the Beam contributor guide. Many thanks to [~rohdesam] for the feedback which helped shape this list. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5672) Staged URL should be posted to GitHub PR
[ https://issues.apache.org/jira/browse/BEAM-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-5672: -- Assignee: Scott Wegner (was: Alan Myrvold) > Staged URL should be posted to GitHub PR > > > Key: BEAM-5672 > URL: https://issues.apache.org/jira/browse/BEAM-5672 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Website PRs are automatically tested and staged via Jenkins to a location on > GCS: https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Commit/ > However, it's difficult to find the URL for the staged job; it's buried > inside the Jenkins job, inside the Gradle scan, inside the log. > I believe there's an API for posting back a context message to the pull > request. If possible, we should push the URL so it's easily viewable from the > pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5672) Staged URL should be posted to GitHub PR
[ https://issues.apache.org/jira/browse/BEAM-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648082#comment-16648082 ] Scott Wegner commented on BEAM-5672: re: Add it to the pr template: We have an existing PR template in GitHub, but it's not specific to website changes. GitHub allows creating multiple PR templates for a repository, but it seems the way to select a template is via URL query parameters and is not triggered automatically i.e. based on changed file path. I don't think this is a viable solution. [PR 6669|https://github.com/apache/beam/pull/6669] pulls the website URL up to the top-level Jenkins job page; I think this is sufficient until we can think of nicer solutions. > Staged URL should be posted to GitHub PR > > > Key: BEAM-5672 > URL: https://issues.apache.org/jira/browse/BEAM-5672 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Alan Myrvold >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > Website PRs are automatically tested and staged via Jenkins to a location on > GCS: https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Commit/ > However, it's difficult to find the URL for the staged job; it's buried > inside the Jenkins job, inside the Gradle scan, inside the log. > I believe there's an API for posting back a context message to the pull > request. If possible, we should push the URL so it's easily viewable from the > pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5727) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No terminal state was returned"
[ https://issues.apache.org/jira/browse/BEAM-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648065#comment-16648065 ] Scott Wegner commented on BEAM-5727: [~rohdesam] could you help triage this? I don't see any additional logs captured here that would be helpful. Perhaps we can try to correlate it with some service logs; if not another idea would be to add more failure logging here so we have a better picture for next time. > [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No > terminal state was returned" > > > Key: BEAM-5727 > URL: https://issues.apache.org/jira/browse/BEAM-5727 > Project: Beam > Issue Type: Bug > Components: runner-dataflow, test-failures >Reporter: Scott Wegner >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w] > * [Test source > code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352] > * [Dataflow > job|https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-11_15_19_52-2781743142799123679?project=apache-beam-testing] > Initial investigation: > The Dataflow job succeeded, but it seems that the service response was > missing completion state / metrics: > Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob > waitUntilFinish > WARNING: No terminal state was returned. State value UNKNOWN > Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner > checkForPAssertSuccess > WARNING: Metrics not present for Dataflow job > 2018-10-11_15_19_52-2781743142799123679. > Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner > run > WARNING: Dataflow job 2018-10-11_15_19_52-2781743142799123679 did not output > a success or failure metric. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5727) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No terminal state was returned"
[ https://issues.apache.org/jira/browse/BEAM-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-5727: -- Assignee: Sam Rohde > [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No > terminal state was returned" > > > Key: BEAM-5727 > URL: https://issues.apache.org/jira/browse/BEAM-5727 > Project: Beam > Issue Type: Bug > Components: runner-dataflow, test-failures >Reporter: Scott Wegner >Assignee: Sam Rohde >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w] > * [Test source > code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352] > * [Dataflow > job|https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-11_15_19_52-2781743142799123679?project=apache-beam-testing] > Initial investigation: > The Dataflow job succeeded, but it seems that the service response was > missing completion state / metrics: > Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob > waitUntilFinish > WARNING: No terminal state was returned. State value UNKNOWN > Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner > checkForPAssertSuccess > WARNING: Metrics not present for Dataflow job > 2018-10-11_15_19_52-2781743142799123679. > Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner > run > WARNING: Dataflow job 2018-10-11_15_19_52-2781743142799123679 did not output > a success or failure metric. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5727) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No terminal state was returned"
[ https://issues.apache.org/jira/browse/BEAM-5727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16648063#comment-16648063 ] Scott Wegner commented on BEAM-5727: The failure here comes from the {{DataflowPipelineJob}} [waitUntilFinish()|https://github.com/apache/beam/blame/42a03a6bd2e6cfdffab02752a31f3139a08a8d94/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L309] loop. Th call to [getJobWithRetries()|https://github.com/apache/beam/blame/42a03a6bd2e6cfdffab02752a31f3139a08a8d94/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L497] seems to have returned without a valid job status. The code for this hasn't changed in a very long time. I suspect this is either a rare flake or some regression in the Dataflow service. > [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No > terminal state was returned" > > > Key: BEAM-5727 > URL: https://issues.apache.org/jira/browse/BEAM-5727 > Project: Beam > Issue Type: Bug > Components: runner-dataflow, test-failures >Reporter: Scott Wegner >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w] > * [Test source > code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352] > * [Dataflow > job|https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-11_15_19_52-2781743142799123679?project=apache-beam-testing] > Initial investigation: > The Dataflow job succeeded, but it seems that the service response was > missing completion state / metrics: > Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob > waitUntilFinish > WARNING: No terminal state was returned. State value UNKNOWN > Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner > checkForPAssertSuccess > WARNING: Metrics not present for Dataflow job > 2018-10-11_15_19_52-2781743142799123679. > Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner > run > WARNING: Dataflow job 2018-10-11_15_19_52-2781743142799123679 did not output > a success or failure metric. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5727) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No terminal state was returned"
Scott Wegner created BEAM-5727: -- Summary: [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] "No terminal state was returned" Key: BEAM-5727 URL: https://issues.apache.org/jira/browse/BEAM-5727 Project: Beam Issue Type: Bug Components: runner-dataflow, test-failures Reporter: Scott Wegner _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/] * [Gradle Build Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w] * [Test source code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352] * [Dataflow job|https://console.cloud.google.com/dataflow/jobsDetail/locations/us-central1/jobs/2018-10-11_15_19_52-2781743142799123679?project=apache-beam-testing] Initial investigation: The Dataflow job succeeded, but it seems that the service response was missing completion state / metrics: Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.DataflowPipelineJob waitUntilFinish WARNING: No terminal state was returned. State value UNKNOWN Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner checkForPAssertSuccess WARNING: Metrics not present for Dataflow job 2018-10-11_15_19_52-2781743142799123679. Oct 11, 2018 10:25:54 PM org.apache.beam.runners.dataflow.TestDataflowRunner run WARNING: Dataflow job 2018-10-11_15_19_52-2781743142799123679 did not output a success or failure metric. _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5726) [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] No terminal state was returned
Scott Wegner created BEAM-5726: -- Summary: [beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle] [testKvSwap] No terminal state was returned Key: BEAM-5726 URL: https://issues.apache.org/jira/browse/BEAM-5726 Project: Beam Issue Type: Bug Components: test-failures Reporter: Scott Wegner _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Java_ValidatesRunner_Dataflow_Gradle/1251/] * [Gradle Build Scan|https://scans.gradle.com/s/wzrkrntbnvshy/tests/ubfk4psvvdijy-nlpo2dsb25h2w] * [Test source code|https://github.com/apache/beam/blob/279a05604b83a54e8e5a79e13d8761f94841f326/runners/google-cloud-dataflow-java/src/main/java/org/apache/beam/runners/dataflow/DataflowPipelineJob.java#L352] Initial investigation: (Add any investigation notes so far) _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5672) Staged URL should be posted to GitHub PR
[ https://issues.apache.org/jira/browse/BEAM-5672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16647187#comment-16647187 ] Scott Wegner commented on BEAM-5672: The functionality I was thinking of is powered by the ghprb plugin via the [buildStatus|https://github.com/jenkinsci/ghprb-plugin#job-dsl-support] hook. However, this only supports a static string. It wouldn't be possible to set a status with a different URL per pull request. However, we could possible add it to the github PR template? > Staged URL should be posted to GitHub PR > > > Key: BEAM-5672 > URL: https://issues.apache.org/jira/browse/BEAM-5672 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Alan Myrvold >Priority: Major > > Website PRs are automatically tested and staged via Jenkins to a location on > GCS: https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Commit/ > However, it's difficult to find the URL for the staged job; it's buried > inside the Jenkins job, inside the Gradle scan, inside the log. > I believe there's an API for posting back a context message to the pull > request. If possible, we should push the URL so it's easily viewable from the > pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
[ https://issues.apache.org/jira/browse/BEAM-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reopened BEAM-5681: Assignee: Scott Wegner (was: Alan Myrvold) The fix from https://github.com/apache/beam/pull/6655 is incorrect; the PR ID binding happens at Seed Job runtime, rather than during the actual job execution. As a result, it was always using the PR context from the Seed Job run. > [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks > --- > > Key: BEAM-5681 > URL: https://issues.apache.org/jira/browse/BEAM-5681 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: currently-failing > Fix For: Not applicable > > Time Spent: 1h 50m > Remaining Estimate: 0h > > _Use this form to file an issue for test failure:_ > * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154] > * [Gradle Build > Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27] > * [Test source > code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140] > Initial investigation: > It seems every link is failing. Same error message: > {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}} > {{ * http:// is an invalid URL (line 76)}} > {{ }} > {{ src="//images/beam_logo_navbar.png">}} > {{ }} > The _Cron version of this job has been failing consistently since 10/5 > ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/]) > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5717) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination bucket gs://apache-beam-website-pull-requests does not exist or the write to the destination mu
[ https://issues.apache.org/jira/browse/BEAM-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5717. Resolution: Fixed Fix Version/s: Not applicable > [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination > bucket gs://apache-beam-website-pull-requests does not exist or the write to > the destination must be restarted > - > > Key: BEAM-5717 > URL: https://issues.apache.org/jira/browse/BEAM-5717 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: currently-failing > Fix For: Not applicable > > Time Spent: 3.5h > Remaining Estimate: 0h > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/] > * [Gradle Build > Scan|https://scans.gradle.com/s/3a7ujky4ekojw/console-log?task=:beam-website:stageWebsite#L4610] > * [Test source > code|https://github.com/apache/beam/blob/280277e7788b4c28680dd8ca02d54a55195b24ba/website/build.gradle#L271] > Initial investigation: > I haven't seen this issue before; it's not clear if this is a persistent > failure or a flake. I will investigate further. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5719) Misconfigured GCS staging jobs can nuke entire top-level bucket
[ https://issues.apache.org/jira/browse/BEAM-5719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5719. Resolution: Fixed Fix Version/s: Not applicable > Misconfigured GCS staging jobs can nuke entire top-level bucket > --- > > Key: BEAM-5719 > URL: https://issues.apache.org/jira/browse/BEAM-5719 > Project: Beam > Issue Type: Sub-task > Components: build-system, website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Fix For: Not applicable > > Time Spent: 0.5h > Remaining Estimate: 0h > > See: https://github.com/apache/beam/pull/6655#issuecomment-429037632 > A bug in the Website_Stage_GCS job was causing the Jenkins job to wipe out > the top-level GCS bucket used to stage and serve the website from. This ends > up blocking all future runs of the job and takes manual effort to recreate > and configure the bucket. > The linked PR should fix the immediate issue, but it's still the case that a > bug in the job/configuration could potentially wipe it out again. We should > refactor the job or modify the command so that there is no longer danger of > wiping it out. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5719) Misconfigured GCS staging jobs can nuke entire top-level bucket
Scott Wegner created BEAM-5719: -- Summary: Misconfigured GCS staging jobs can nuke entire top-level bucket Key: BEAM-5719 URL: https://issues.apache.org/jira/browse/BEAM-5719 Project: Beam Issue Type: Sub-task Components: build-system, website Reporter: Scott Wegner Assignee: Scott Wegner See: https://github.com/apache/beam/pull/6655#issuecomment-429037632 A bug in the Website_Stage_GCS job was causing the Jenkins job to wipe out the top-level GCS bucket used to stage and serve the website from. This ends up blocking all future runs of the job and takes manual effort to recreate and configure the bucket. The linked PR should fix the immediate issue, but it's still the case that a bug in the job/configuration could potentially wipe it out again. We should refactor the job or modify the command so that there is no longer danger of wiping it out. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job fixes
[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646845#comment-16646845 ] Scott Wegner commented on BEAM-5467: Thanks for the additional context. I'm not an expert on diagnosing memory issues, but here's what I can pull out of there: * The build scan shows [some stats on memory usage|https://scans.gradle.com/s/f2u3q2obrgaqu/performance/build#memory], and for this build I see "PS Eden Space" of 1.36/1.36 GB (99.5%). I would deduce that the JVM ran out of allotted memory causing the segfault. * The [infrastructure tab|https://scans.gradle.com/s/f2u3q2obrgaqu#infrastructure] shows the "Max JVM memory heap size" for the job: 3824 MB * In the [timeline|https://scans.gradle.com/s/f2u3q2obrgaqu/timeline] I can see that the task that failed was {{:beam-sdks-python:flinkCompatibilityMatrixBatch}}. Nothing was running concurrently as part of the build, so either this task ate up the entire heap space, or some previous task is leaking memory. My recommendation would be to work towards getting a local repro so that you can attach a memory profiler and validate potential fixes. The Jenkins job shows the full command-line used to launch the job, including JVM memory configuration: {{gradlew --info --continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g :beam-sdks-python:flinkCompatibilityMatrixBatch :beam-sdks-python:flinkCompatibilityMatrixStreaming}} > Python Flink ValidatesRunner job fixes > -- > > Key: BEAM-5467 > URL: https://issues.apache.org/jira/browse/BEAM-5467 > Project: Beam > Issue Type: Improvement > Components: runner-flink >Reporter: Thomas Weise >Assignee: Thomas Weise >Priority: Minor > Labels: portability-flink > Time Spent: 9h 40m > Remaining Estimate: 0h > > Add status to README > Rename script and job for consistency > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5467) Python Flink ValidatesRunner job fixes
[ https://issues.apache.org/jira/browse/BEAM-5467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646757#comment-16646757 ] Scott Wegner commented on BEAM-5467: [~thw] offhand this doesn't look familiar to me. Can you link to a Jenkin job run / Gradle build scan with more details? > Python Flink ValidatesRunner job fixes > -- > > Key: BEAM-5467 > URL: https://issues.apache.org/jira/browse/BEAM-5467 > Project: Beam > Issue Type: Improvement > Components: runner-flink >Reporter: Thomas Weise >Assignee: Thomas Weise >Priority: Minor > Labels: portability-flink > Time Spent: 9h 40m > Remaining Estimate: 0h > > Add status to README > Rename script and job for consistency > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job
[ https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5669. Resolution: Fixed Fix Version/s: (was: 2.8.0) Not applicable > Empty commits pushed by PostCommit_Website_Publish jenkins job > -- > > Key: BEAM-5669 > URL: https://issues.apache.org/jira/browse/BEAM-5669 > Project: Beam > Issue Type: Bug > Components: build-system >Affects Versions: 2.8.0 >Reporter: Alan Myrvold >Assignee: Scott Wegner >Priority: Major > Fix For: Not applicable > > Time Spent: 1h > Remaining Estimate: 0h > > The isClean() call in > https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165 > is returning false due to unstaged .gradle and build files. > The asf-site branch needs a .gitignore, which can be the same as the master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job
[ https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646738#comment-16646738 ] Scott Wegner commented on BEAM-5669: The above PR is now merged, and I'm no longer seeing empty commits. I've also removed previous empty commits from the branch using {{git rebase}}: https://github.com/apache/beam/commits/asf-site > Empty commits pushed by PostCommit_Website_Publish jenkins job > -- > > Key: BEAM-5669 > URL: https://issues.apache.org/jira/browse/BEAM-5669 > Project: Beam > Issue Type: Bug > Components: build-system >Affects Versions: 2.8.0 >Reporter: Alan Myrvold >Assignee: Scott Wegner >Priority: Major > Fix For: 2.8.0 > > Time Spent: 1h > Remaining Estimate: 0h > > The isClean() call in > https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165 > is returning false due to unstaged .gradle and build files. > The asf-site branch needs a .gitignore, which can be the same as the master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5717) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination bucket gs://apache-beam-website-pull-requests does not exist or the write to the destination m
[ https://issues.apache.org/jira/browse/BEAM-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646607#comment-16646607 ] Scott Wegner commented on BEAM-5717: It appears the root-cause of this is that from the Jenkins job we attempt to pass a {{-PgithubPullRequestId}} even when there is no associated PR (i.e. for the cron job). In the [Jenkins consoleFull|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/consoleFull] log, I see the commandline: {{$ /home/jenkins/jenkins-slave/workspace/beam_PreCommit_Website_Stage_GCS_Cron/src/gradlew --info --continue --max-workers=12 -Dorg.gradle.jvmargs=-Xms2g -Dorg.gradle.jvmargs=-Xmx4g -PgithubPullRequestId=${ghprbPullId} :beam-website:stageWebsite}} The easiest fix would probably be to remove the parameter from the Jenkins job if there is no PR. > [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination > bucket gs://apache-beam-website-pull-requests does not exist or the write to > the destination must be restarted > - > > Key: BEAM-5717 > URL: https://issues.apache.org/jira/browse/BEAM-5717 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/] > * [Gradle Build > Scan|https://scans.gradle.com/s/3a7ujky4ekojw/console-log?task=:beam-website:stageWebsite#L4610] > * [Test source > code|https://github.com/apache/beam/blob/280277e7788b4c28680dd8ca02d54a55195b24ba/website/build.gradle#L271] > Initial investigation: > I haven't seen this issue before; it's not clear if this is a persistent > failure or a flake. I will investigate further. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5717) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination bucket gs://apache-beam-website-pull-requests does not exist or the write to the destination m
[ https://issues.apache.org/jira/browse/BEAM-5717?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16646608#comment-16646608 ] Scott Wegner commented on BEAM-5717: /cc [~alanmyrvold] > [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination > bucket gs://apache-beam-website-pull-requests does not exist or the write to > the destination must be restarted > - > > Key: BEAM-5717 > URL: https://issues.apache.org/jira/browse/BEAM-5717 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/] > * [Gradle Build > Scan|https://scans.gradle.com/s/3a7ujky4ekojw/console-log?task=:beam-website:stageWebsite#L4610] > * [Test source > code|https://github.com/apache/beam/blob/280277e7788b4c28680dd8ca02d54a55195b24ba/website/build.gradle#L271] > Initial investigation: > I haven't seen this issue before; it's not clear if this is a persistent > failure or a flake. I will investigate further. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5717) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination bucket gs://apache-beam-website-pull-requests does not exist or the write to the destination mus
Scott Wegner created BEAM-5717: -- Summary: [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] 404 The destination bucket gs://apache-beam-website-pull-requests does not exist or the write to the destination must be restarted Key: BEAM-5717 URL: https://issues.apache.org/jira/browse/BEAM-5717 Project: Beam Issue Type: Bug Components: test-failures Reporter: Scott Wegner Assignee: Scott Wegner _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/28/] * [Gradle Build Scan|https://scans.gradle.com/s/3a7ujky4ekojw/console-log?task=:beam-website:stageWebsite#L4610] * [Test source code|https://github.com/apache/beam/blob/280277e7788b4c28680dd8ca02d54a55195b24ba/website/build.gradle#L271] Initial investigation: I haven't seen this issue before; it's not clear if this is a persistent failure or a flake. I will investigate further. _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation
[ https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5678. Resolution: Fixed Fix Version/s: 2.8.0 > Jeckyll redirect_from incompatible with HTMLProofer url validation > -- > > Key: BEAM-5678 > URL: https://issues.apache.org/jira/browse/BEAM-5678 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Minor > Fix For: 2.8.0 > > > Renaming a URL and using the Jekyll {{redirect_from}} feature breaks > pre-commit validation. > We hit this on > [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]: > bq. The scenario for using a redirect is if you're moving {{pageA.md}} to > {{pageB.md}}; you would rename the file to {{pageB.md}} and add > {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} > with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} > doesn't exist on the livesite yet, so it will always fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation
[ https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645734#comment-16645734 ] Scott Wegner commented on BEAM-5678: As of https://github.com/apache/beam/pull/6608, this no longer seems to be an issue. The locally-generated HTML used for testing no longer specifies as site {{url}} in the _config.yml, so the redirect plugin generates relative redirects. Huzzah! > Jeckyll redirect_from incompatible with HTMLProofer url validation > -- > > Key: BEAM-5678 > URL: https://issues.apache.org/jira/browse/BEAM-5678 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Minor > Fix For: 2.8.0 > > > Renaming a URL and using the Jekyll {{redirect_from}} feature breaks > pre-commit validation. > We hit this on > [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]: > bq. The scenario for using a redirect is if you're moving {{pageA.md}} to > {{pageB.md}}; you would rename the file to {{pageB.md}} and add > {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} > with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} > doesn't exist on the livesite yet, so it will always fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-4503) Drain/migrate in-progress PR's from apache/beam-site
[ https://issues.apache.org/jira/browse/BEAM-4503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645690#comment-16645690 ] Scott Wegner commented on BEAM-4503: Only 3 PR's remaining; I pinged each of them today. https://github.com/apache/beam-site/pulls > Drain/migrate in-progress PR's from apache/beam-site > > > Key: BEAM-4503 > URL: https://issues.apache.org/jira/browse/BEAM-4503 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: beam-site-automation-reliability > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation
[ https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-5678: -- Assignee: Scott Wegner > Jeckyll redirect_from incompatible with HTMLProofer url validation > -- > > Key: BEAM-5678 > URL: https://issues.apache.org/jira/browse/BEAM-5678 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Minor > > Renaming a URL and using the Jekyll {{redirect_from}} feature breaks > pre-commit validation. > We hit this on > [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]: > bq. The scenario for using a redirect is if you're moving {{pageA.md}} to > {{pageB.md}}; you would rename the file to {{pageB.md}} and add > {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} > with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} > doesn't exist on the livesite yet, so it will always fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-4497) Add pages for master Javadocs / Pydocs and incorporate into post-commit job
[ https://issues.apache.org/jira/browse/BEAM-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-4497: -- Assignee: (was: Scott Wegner) > Add pages for master Javadocs / Pydocs and incorporate into post-commit job > --- > > Key: BEAM-4497 > URL: https://issues.apache.org/jira/browse/BEAM-4497 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Priority: Major > Labels: beam-site-automation-reliability > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job
[ https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-5669: -- Assignee: Scott Wegner (was: Alan Myrvold) > Empty commits pushed by PostCommit_Website_Publish jenkins job > -- > > Key: BEAM-5669 > URL: https://issues.apache.org/jira/browse/BEAM-5669 > Project: Beam > Issue Type: Bug > Components: build-system >Affects Versions: 2.8.0 >Reporter: Alan Myrvold >Assignee: Scott Wegner >Priority: Major > Fix For: 2.8.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The isClean() call in > https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165 > is returning false due to unstaged .gradle and build files. > The asf-site branch needs a .gitignore, which can be the same as the master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job
[ https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reopened BEAM-5669: I believe this is still an issue. Here's an example empty commit from today: https://github.com/apache/beam/commit/5eb09607e52b621a3bed6a868a324465781d3e56 > Empty commits pushed by PostCommit_Website_Publish jenkins job > -- > > Key: BEAM-5669 > URL: https://issues.apache.org/jira/browse/BEAM-5669 > Project: Beam > Issue Type: Bug > Components: build-system >Affects Versions: 2.8.0 >Reporter: Alan Myrvold >Assignee: Alan Myrvold >Priority: Major > Fix For: 2.8.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The isClean() call in > https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165 > is returning false due to unstaged .gradle and build files. > The asf-site branch needs a .gitignore, which can be the same as the master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5688) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on githubPullRequestId assert
[ https://issues.apache.org/jira/browse/BEAM-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5688. Resolution: Fixed Fix Version/s: Not applicable This should be fixed with: https://github.com/apache/beam/pull/6608 > [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on > githubPullRequestId assert > -- > > Key: BEAM-5688 > URL: https://issues.apache.org/jira/browse/BEAM-5688 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Assignee: Alan Myrvold >Priority: Major > Labels: currently-failing > Fix For: Not applicable > > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/20/] > * [Gradle Build Scan|https://gradle.com/s/7mqwgjegf5hge] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L234] > Initial investigation: > This is a problem with how the website gradle scripts are implemented to > accept an githubPullRequestId. The Cron job will not have an associated PR, > so this currently fails. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
[ https://issues.apache.org/jira/browse/BEAM-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5681. Resolution: Fixed Fix Version/s: Not applicable This should be fixed with: https://github.com/apache/beam/pull/6608 > [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks > --- > > Key: BEAM-5681 > URL: https://issues.apache.org/jira/browse/BEAM-5681 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Alan Myrvold >Priority: Major > Labels: currently-failing > Fix For: Not applicable > > Time Spent: 1h 50m > Remaining Estimate: 0h > > _Use this form to file an issue for test failure:_ > * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154] > * [Gradle Build > Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27] > * [Test source > code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140] > Initial investigation: > It seems every link is failing. Same error message: > {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}} > {{ * http:// is an invalid URL (line 76)}} > {{ }} > {{ src="//images/beam_logo_navbar.png">}} > {{ }} > The _Cron version of this job has been failing consistently since 10/5 > ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/]) > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5704) Update release guide for beam-site changes
[ https://issues.apache.org/jira/browse/BEAM-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-5704. -- Resolution: Not A Problem Fix Version/s: (was: 2.8.0) Not applicable > Update release guide for beam-site changes > -- > > Key: BEAM-5704 > URL: https://issues.apache.org/jira/browse/BEAM-5704 > Project: Beam > Issue Type: Bug > Components: website >Reporter: Ahmet Altay >Assignee: Scott Wegner >Priority: Major > Fix For: Not applicable > > > [https://beam.apache.org/contribute/release-guide] has instructions related > to updating beam-site. They need to be updated to reflect the beam-site > deprecation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5704) Update release guide for beam-site changes
[ https://issues.apache.org/jira/browse/BEAM-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645412#comment-16645412 ] Scott Wegner commented on BEAM-5704: I spoke to [~altay] and we believe the docs are up-to-date. > Update release guide for beam-site changes > -- > > Key: BEAM-5704 > URL: https://issues.apache.org/jira/browse/BEAM-5704 > Project: Beam > Issue Type: Bug > Components: website >Reporter: Ahmet Altay >Assignee: Scott Wegner >Priority: Major > Fix For: Not applicable > > > [https://beam.apache.org/contribute/release-guide] has instructions related > to updating beam-site. They need to be updated to reflect the beam-site > deprecation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5704) Update release guide for beam-site changes
[ https://issues.apache.org/jira/browse/BEAM-5704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16645406#comment-16645406 ] Scott Wegner commented on BEAM-5704: This was tracked via BEAM-4501, [~udim] worked on this. Are there specific parts that still need updating? > Update release guide for beam-site changes > -- > > Key: BEAM-5704 > URL: https://issues.apache.org/jira/browse/BEAM-5704 > Project: Beam > Issue Type: Bug > Components: website >Reporter: Ahmet Altay >Assignee: Scott Wegner >Priority: Major > Fix For: 2.8.0 > > > [https://beam.apache.org/contribute/release-guide] has instructions related > to updating beam-site. They need to be updated to reflect the beam-site > deprecation. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to pip download flake
[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643939#comment-16643939 ] Scott Wegner commented on BEAM-5683: This failed again today: https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1303/ > [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to > pip download flake > -- > > Key: BEAM-5683 > URL: https://issues.apache.org/jira/browse/BEAM-5683 > Project: Beam > Issue Type: Bug > Components: sdk-py-harness, test-failures >Reporter: Scott Wegner >Assignee: Ankur Goenka >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/] > * [Gradle Build > Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests] > * [Test source > code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390] > Initial investigation: > Seems to be failing on pip download. > == > ERROR: test_multiple_empty_outputs > (apache_beam.transforms.ptransform_test.PTransformTest) > -- > Traceback (most recent call last): > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py", > line 277, in test_multiple_empty_outputs > pipeline.run() > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py", > line 104, in run > result = super(TestPipeline, self).run(test_runner_api) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 403, in run > self.to_runner_api(), self.runner, self._options).run(False) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 416, in run > return self.runner.run_pipeline(self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", > line 50, in run_pipeline > self.result = super(TestDataflowRunner, self).run_pipeline(pipeline) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", > line 389, in run_pipeline > self.dataflow_client.create_job(self.job), self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py", > line 184, in wrapper > return fun(*args, **kwargs) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 490, in create_job > self.create_job_description(job) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 519, in create_job_description > resources = self._stage_resour > ces(job.options) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 452, in _stage_resources > staging_location=google_cloud_options.staging_location) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 161, in stage_job_resources > requirements_cache_path) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 411, in _populate_requirements_cache > processes.check_call(cmd_args) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py", > line 46, in check_call > return subprocess.check_call(*args, **kwargs) > File "/usr/lib/python2.7/subprocess.py", line 541, in check_call > raise CalledProcessError(retcode, cmd) > CalledProcessError: Command > '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python', > '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', > 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', > ':all:']' returned non-zero exit status 1 > > _After you've
[jira] [Resolved] (BEAM-5294) [beam_Release_Gradle_NightlySnapshot] Failing due to website test.
[ https://issues.apache.org/jira/browse/BEAM-5294?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5294. Resolution: Fixed Fix Version/s: Not applicable > [beam_Release_Gradle_NightlySnapshot] Failing due to website test. > -- > > Key: BEAM-5294 > URL: https://issues.apache.org/jira/browse/BEAM-5294 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Boyuan Zhang >Assignee: Alan Myrvold >Priority: Major > Fix For: Not applicable > > > Build link: > [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/] > [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/185/] > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5522) beam_PostRelease_NightlySnapshot timed out
[ https://issues.apache.org/jira/browse/BEAM-5522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5522. Resolution: Cannot Reproduce Fix Version/s: Not applicable > beam_PostRelease_NightlySnapshot timed out > -- > > Key: BEAM-5522 > URL: https://issues.apache.org/jira/browse/BEAM-5522 > Project: Beam > Issue Type: Bug > Components: test-failures, testing >Reporter: Ahmet Altay >Assignee: Alan Myrvold >Priority: Major > Fix For: Not applicable > > > https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/383/console -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5407) [beam_PostCommit_Go_GradleBuild][testE2ETopWikiPages][RolledBack] Breaks post commit
[ https://issues.apache.org/jira/browse/BEAM-5407?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643833#comment-16643833 ] Scott Wegner commented on BEAM-5407: Is this resolved? Can this be closed? > [beam_PostCommit_Go_GradleBuild][testE2ETopWikiPages][RolledBack] Breaks post > commit > > > Key: BEAM-5407 > URL: https://issues.apache.org/jira/browse/BEAM-5407 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Batkhuyag Batsaikhan >Assignee: Pablo Estrada >Priority: Major > > Failing job url: > https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1482/testReport/ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5534) AutocompleteIT and WikiTopSessionsIT fail in google testing
[ https://issues.apache.org/jira/browse/BEAM-5534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-5534. -- Resolution: Fixed Fix Version/s: Not applicable > AutocompleteIT and WikiTopSessionsIT fail in google testing > --- > > Key: BEAM-5534 > URL: https://issues.apache.org/jira/browse/BEAM-5534 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Pablo Estrada >Assignee: Pablo Estrada >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Closed] (BEAM-5125) beam_PostCommit_Java_GradleBuild org.apache.beam.runners.flink PortableExecutionTest testExecution_1_ flaky
[ https://issues.apache.org/jira/browse/BEAM-5125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner closed BEAM-5125. -- Resolution: Cannot Reproduce Fix Version/s: Not applicable > beam_PostCommit_Java_GradleBuild org.apache.beam.runners.flink > PortableExecutionTest testExecution_1_ flaky > --- > > Key: BEAM-5125 > URL: https://issues.apache.org/jira/browse/BEAM-5125 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Mikhail Gryzykhin >Priority: Major > Fix For: Not applicable > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Test fails in both: post and precommit tests. Fails more often in pre-commits. > Pre-commit history: > [https://builds.apache.org/job/beam_PreCommit_Java_Phrase/180/testReport/junit/org.apache.beam.runners.flink/PortableExecutionTest/testExecution_1_/history/] > Post-commit history: > [https://builds.apache.org/job/beam_PostCommit_Java_GradleBuild/1223/testReport/junit/org.apache.beam.runners.flink/PortableExecutionTest/testExecution_1_/history/?start=75] > Sample job: > https://builds.apache.org/job/beam_PreCommit_Java_Phrase/180/testReport/junit/org.apache.beam.runners.flink/PortableExecutionTest/testExecution_1_/ > Log: > java.lang.AssertionError: job state expected: but was: at > org.junit.Assert.fail(Assert.java:88) at > org.junit.Assert.failNotEquals(Assert.java:834) at > org.junit.Assert.assertEquals(Assert.java:118) at > org.apache.beam.runners.flink.PortableExecutionTest.testExecution(PortableExecutionTest.java:177) > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5367) [beam_Release_Gradle_NightlySnapshot] is broken due to apache/beam website test failure
[ https://issues.apache.org/jira/browse/BEAM-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5367. Resolution: Fixed > [beam_Release_Gradle_NightlySnapshot] is broken due to apache/beam website > test failure > --- > > Key: BEAM-5367 > URL: https://issues.apache.org/jira/browse/BEAM-5367 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Batkhuyag Batsaikhan >Assignee: Batkhuyag Batsaikhan >Priority: Major > Fix For: 2.8.0 > > Time Spent: 1h > Remaining Estimate: 0h > > apache/website test is breaking nightly build. Since we don't have stable > apache/beam website, we should remove it from the build for now. Until we > have an actual plan to fully migrate from asf beam-site, we should disable > apache/website checks from Beam build. > Failing job url: [https://scans.gradle.com/s/uxb6mdigqj4n4/console-log#L23465] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5367) [beam_Release_Gradle_NightlySnapshot] is broken due to apache/beam website test failure
[ https://issues.apache.org/jira/browse/BEAM-5367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643830#comment-16643830 ] Scott Wegner commented on BEAM-5367: This is now fixed, latest runs have been successful: https://builds.apache.org/job/beam_PostRelease_NightlySnapshot/395/ > [beam_Release_Gradle_NightlySnapshot] is broken due to apache/beam website > test failure > --- > > Key: BEAM-5367 > URL: https://issues.apache.org/jira/browse/BEAM-5367 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Batkhuyag Batsaikhan >Assignee: Batkhuyag Batsaikhan >Priority: Major > Fix For: 2.8.0 > > Time Spent: 1h > Remaining Estimate: 0h > > apache/website test is breaking nightly build. Since we don't have stable > apache/beam website, we should remove it from the build for now. Until we > have an actual plan to fully migrate from asf beam-site, we should disable > apache/website checks from Beam build. > Failing job url: [https://scans.gradle.com/s/uxb6mdigqj4n4/console-log#L23465] -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5243) beam_Release_Gradle_NightlySnapshot InvocationError py27-cython/bin/python setup.py nosetests
[ https://issues.apache.org/jira/browse/BEAM-5243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5243: --- Labels: (was: currently-failing) > beam_Release_Gradle_NightlySnapshot InvocationError py27-cython/bin/python > setup.py nosetests > - > > Key: BEAM-5243 > URL: https://issues.apache.org/jira/browse/BEAM-5243 > Project: Beam > Issue Type: Bug > Components: sdk-py-core, test-failures >Reporter: Andrew Pilloud >Assignee: Ahmet Altay >Priority: Major > > It isn't clear to me what exactly failed, logs are full of stack traces. > [https://builds.apache.org/job/beam_Release_Gradle_NightlySnapshot/151/] > [https://builds.apache.org/job/beam_PostCommit_Python_Verify/5844/] > > *01:00:38* ERROR: InvocationError for command > '/home/jenkins/jenkins-slave/workspace/beam_Release_Gradle_NightlySnapshot/src/sdks/python/target/.tox/py27-cython/bin/python > setup.py nosetests' (exited with code -11)*01:00:38* > ___ summary > *01:00:38* ERROR: py27-cython: commands > failed*01:00:38* -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5665) [beam_PreCommit_Website_Commit] [:testWebsite] External link http://www.atrato.io/blog/2017/04/08/apache-apex-cli/ failed: response code 0 means something's wrong.
[ https://issues.apache.org/jira/browse/BEAM-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643826#comment-16643826 ] Scott Wegner commented on BEAM-5665: http://www.atrato.io/ still 404's. If this doesn't resolve by next week we should remove the dead link. > [beam_PreCommit_Website_Commit] [:testWebsite] External link > http://www.atrato.io/blog/2017/04/08/apache-apex-cli/ failed: response code 0 > means something's wrong. > > > Key: BEAM-5665 > URL: https://issues.apache.org/jira/browse/BEAM-5665 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PreCommit_Website_Commit/249/] > * [Gradle Build > Scan|https://scans.gradle.com/s/2sqxa7hlvdeum/console-log?task=:beam-website:testWebsite#L12] > * [Test source > code|https://github.com/apache/beam-site/blob/asf-site/Rakefile] > Initial investigation: > It seems http://www.atrato.io/ is down or no longer available. We should > disable the test for now and perhaps remove the dead link if it no longer > works. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5661) [beam_PostCommit_Py_ValCont] [:beam-sdks-python-container:docker] no such file or directory
[ https://issues.apache.org/jira/browse/BEAM-5661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5661: --- Labels: (was: currently-failing) > [beam_PostCommit_Py_ValCont] [:beam-sdks-python-container:docker] no such > file or directory > --- > > Key: BEAM-5661 > URL: https://issues.apache.org/jira/browse/BEAM-5661 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Assignee: Henning Rohde >Priority: Major > > _Use this form to file an issue for test failure:_ > * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Py_ValCont/902/] > * [Gradle Build > Scan|https://scans.gradle.com/s/pmjbkhxaeryx4/console-log?task=:beam-sdks-python-container:docker#L2] > * [Test source > code|https://github.com/apache/beam/blob/845f8d0abcc5a8d7f93457c27aff0feeb1a867d5/sdks/python/container/build.gradle#L61] > Initial investigation: > {{failed to get digest > sha256:4ee4ea2f0113e98b49d8e376ce847feb374ddf2b8ea775502459d8a1b8a3eaed: open > /var/lib/docker/image/aufs/imagedb/content/sha256/4ee4ea2f0113e98b49d8e376ce847feb374ddf2b8ea775502459d8a1b8a3eaed: > no such file or directory}} > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5665) [beam_PreCommit_Website_Commit] [:testWebsite] External link http://www.atrato.io/blog/2017/04/08/apache-apex-cli/ failed: response code 0 means something's wrong.
[ https://issues.apache.org/jira/browse/BEAM-5665?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5665: --- Labels: (was: currently-failing) > [beam_PreCommit_Website_Commit] [:testWebsite] External link > http://www.atrato.io/blog/2017/04/08/apache-apex-cli/ failed: response code 0 > means something's wrong. > > > Key: BEAM-5665 > URL: https://issues.apache.org/jira/browse/BEAM-5665 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PreCommit_Website_Commit/249/] > * [Gradle Build > Scan|https://scans.gradle.com/s/2sqxa7hlvdeum/console-log?task=:beam-website:testWebsite#L12] > * [Test source > code|https://github.com/apache/beam-site/blob/asf-site/Rakefile] > Initial investigation: > It seems http://www.atrato.io/ is down or no longer available. We should > disable the test for now and perhaps remove the dead link if it no longer > works. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5662) [beam_PostCommit_Website_Publish] [:testWebsite] External link http://wiki.apache.org/incubator/BeamProposal failed: got a time out
[ https://issues.apache.org/jira/browse/BEAM-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5662: --- Labels: (was: currently-failing) > [beam_PostCommit_Website_Publish] [:testWebsite] External link > http://wiki.apache.org/incubator/BeamProposal failed: got a time out > --- > > Key: BEAM-5662 > URL: https://issues.apache.org/jira/browse/BEAM-5662 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/94/] > * [Gradle Build > Scan|https://scans.gradle.com/s/h4mayefon7v7q/console-log?task=:beam-website:testWebsite#L12] > * [Test source > code|https://github.com/apache/beam/blob/845f8d0abcc5a8d7f93457c27aff0feeb1a867d5/website/Rakefile#L6] > Initial investigation: > The failed link is http://wiki.apache.org/incubator/BeamProposal > When I visit this link, it works for me. This is likely a flake. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5662) [beam_PostCommit_Website_Publish] [:testWebsite] External link http://wiki.apache.org/incubator/BeamProposal failed: got a time out
[ https://issues.apache.org/jira/browse/BEAM-5662?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643825#comment-16643825 ] Scott Wegner commented on BEAM-5662: I haven't seen this in a while, but I'll keep open for a bit just in case. > [beam_PostCommit_Website_Publish] [:testWebsite] External link > http://wiki.apache.org/incubator/BeamProposal failed: got a time out > --- > > Key: BEAM-5662 > URL: https://issues.apache.org/jira/browse/BEAM-5662 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/94/] > * [Gradle Build > Scan|https://scans.gradle.com/s/h4mayefon7v7q/console-log?task=:beam-website:testWebsite#L12] > * [Test source > code|https://github.com/apache/beam/blob/845f8d0abcc5a8d7f93457c27aff0feeb1a867d5/website/Rakefile#L6] > Initial investigation: > The failed link is http://wiki.apache.org/incubator/BeamProposal > When I visit this link, it works for me. This is likely a flake. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643824#comment-16643824 ] Scott Wegner commented on BEAM-5680: Latest run succeeded, this appears to be fixed: https://builds.apache.org/job/beam_PostCommit_Website_Publish/146/ > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: currently-failing > Fix For: Not applicable > > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]). > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5680. Resolution: Fixed Fix Version/s: Not applicable > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: currently-failing > Fix For: Not applicable > > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]). > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5688) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on githubPullRequestId assert
[ https://issues.apache.org/jira/browse/BEAM-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643817#comment-16643817 ] Scott Wegner commented on BEAM-5688: [~udim] fyi > [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on > githubPullRequestId assert > -- > > Key: BEAM-5688 > URL: https://issues.apache.org/jira/browse/BEAM-5688 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Assignee: Alan Myrvold >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/20/] > * [Gradle Build Scan|https://gradle.com/s/7mqwgjegf5hge] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L234] > Initial investigation: > This is a problem with how the website gradle scripts are implemented to > accept an githubPullRequestId. The Cron job will not have an associated PR, > so this currently fails. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5691) testWebsite failed owing to several url 404 not found
[ https://issues.apache.org/jira/browse/BEAM-5691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5691. Resolution: Duplicate Fix Version/s: Not applicable > testWebsite failed owing to several url 404 not found > - > > Key: BEAM-5691 > URL: https://issues.apache.org/jira/browse/BEAM-5691 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Boyuan Zhang >Priority: Major > Fix For: Not applicable > > > test log: > https://builds.apache.org/job/beam_PreCommit_Website_Commit/262/console -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to pip download flake
[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5683: --- Summary: [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to pip download flake (was: [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary) > [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Fails due to > pip download flake > -- > > Key: BEAM-5683 > URL: https://issues.apache.org/jira/browse/BEAM-5683 > Project: Beam > Issue Type: Bug > Components: sdk-py-harness, test-failures >Reporter: Scott Wegner >Assignee: Ankur Goenka >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/] > * [Gradle Build > Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests] > * [Test source > code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390] > Initial investigation: > Seems to be failing on pip download. > == > ERROR: test_multiple_empty_outputs > (apache_beam.transforms.ptransform_test.PTransformTest) > -- > Traceback (most recent call last): > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py", > line 277, in test_multiple_empty_outputs > pipeline.run() > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py", > line 104, in run > result = super(TestPipeline, self).run(test_runner_api) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 403, in run > self.to_runner_api(), self.runner, self._options).run(False) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 416, in run > return self.runner.run_pipeline(self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", > line 50, in run_pipeline > self.result = super(TestDataflowRunner, self).run_pipeline(pipeline) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", > line 389, in run_pipeline > self.dataflow_client.create_job(self.job), self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py", > line 184, in wrapper > return fun(*args, **kwargs) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 490, in create_job > self.create_job_description(job) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 519, in create_job_description > resources = self._stage_resour > ces(job.options) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 452, in _stage_resources > staging_location=google_cloud_options.staging_location) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 161, in stage_job_resources > requirements_cache_path) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 411, in _populate_requirements_cache > processes.check_call(cmd_args) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py", > line 46, in check_call > return subprocess.check_call(*args, **kwargs) > File "/usr/lib/python2.7/subprocess.py", line 541, in check_call > raise CalledProcessError(retcode, cmd) > CalledProcessError: Command > '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python', > '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', > 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', >
[jira] [Commented] (BEAM-5688) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on githubPullRequestId assert
[ https://issues.apache.org/jira/browse/BEAM-5688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643659#comment-16643659 ] Scott Wegner commented on BEAM-5688: I have https://github.com/apache/beam/pull/6608 out to fix this, but the current version doesn't work. > [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on > githubPullRequestId assert > -- > > Key: BEAM-5688 > URL: https://issues.apache.org/jira/browse/BEAM-5688 > Project: Beam > Issue Type: Bug > Components: test-failures >Reporter: Scott Wegner >Assignee: Alan Myrvold >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/20/] > * [Gradle Build Scan|https://gradle.com/s/7mqwgjegf5hge] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L234] > Initial investigation: > This is a problem with how the website gradle scripts are implemented to > accept an githubPullRequestId. The Cron job will not have an associated PR, > so this currently fails. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5688) [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on githubPullRequestId assert
Scott Wegner created BEAM-5688: -- Summary: [beam_PreCommit_Website_Stage_GCS_Cron] [stageWebsite] Fails on githubPullRequestId assert Key: BEAM-5688 URL: https://issues.apache.org/jira/browse/BEAM-5688 Project: Beam Issue Type: Bug Components: test-failures Reporter: Scott Wegner Assignee: Alan Myrvold _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/20/] * [Gradle Build Scan|https://gradle.com/s/7mqwgjegf5hge] * [Test source code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L234] Initial investigation: This is a problem with how the website gradle scripts are implemented to accept an githubPullRequestId. The Cron job will not have an associated PR, so this currently fails. _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5281) There are several deadlinks in beam-site, please removed.
[ https://issues.apache.org/jira/browse/BEAM-5281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5281. Resolution: Duplicate Fix Version/s: Not applicable Dupe of BEAM-5681; there's something broken in the pre-commit scripts. > There are several deadlinks in beam-site, please removed. > - > > Key: BEAM-5281 > URL: https://issues.apache.org/jira/browse/BEAM-5281 > Project: Beam > Issue Type: Bug > Components: website >Reporter: Boyuan Zhang >Assignee: Melissa Pashniak >Priority: Major > Fix For: Not applicable > > Time Spent: 40m > Remaining Estimate: 0h > > Deadlinks in beam-site cause nightly build failed: > https://scans.gradle.com/s/nzwfwj6iqlgrg/console-log?task=:beam-website:testWebsite#L13 -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16643606#comment-16643606 ] Scott Wegner commented on BEAM-5683: [~pabloem] / [~robertwb] can either of you help out? bq. Can we access pip subprocess logs? > [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary > -- > > Key: BEAM-5683 > URL: https://issues.apache.org/jira/browse/BEAM-5683 > Project: Beam > Issue Type: Bug > Components: sdk-py-harness, test-failures >Reporter: Scott Wegner >Assignee: Ankur Goenka >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/] > * [Gradle Build > Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests] > * [Test source > code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390] > Initial investigation: > Seems to be failing on pip download. > == > ERROR: test_multiple_empty_outputs > (apache_beam.transforms.ptransform_test.PTransformTest) > -- > Traceback (most recent call last): > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py", > line 277, in test_multiple_empty_outputs > pipeline.run() > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py", > line 104, in run > result = super(TestPipeline, self).run(test_runner_api) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 403, in run > self.to_runner_api(), self.runner, self._options).run(False) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 416, in run > return self.runner.run_pipeline(self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", > line 50, in run_pipeline > self.result = super(TestDataflowRunner, self).run_pipeline(pipeline) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", > line 389, in run_pipeline > self.dataflow_client.create_job(self.job), self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py", > line 184, in wrapper > return fun(*args, **kwargs) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 490, in create_job > self.create_job_description(job) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 519, in create_job_description > resources = self._stage_resour > ces(job.options) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 452, in _stage_resources > staging_location=google_cloud_options.staging_location) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 161, in stage_job_resources > requirements_cache_path) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 411, in _populate_requirements_cache > processes.check_call(cmd_args) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py", > line 46, in check_call > return subprocess.check_call(*args, **kwargs) > File "/usr/lib/python2.7/subprocess.py", line 541, in check_call > raise CalledProcessError(retcode, cmd) > CalledProcessError: Command > '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python', > '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', > 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', > ':all:']' returned non-zero exit status 1 > > _After you've filled out the above details, please
[jira] [Commented] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642309#comment-16642309 ] Scott Wegner commented on BEAM-5683: [~angoenka] I see you [previously worked on the [_populate_requirements_cache|https://github.com/apache/beam/pull/5251] code. Can you help investigate this flake or find a more appropriate owner? > [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary > -- > > Key: BEAM-5683 > URL: https://issues.apache.org/jira/browse/BEAM-5683 > Project: Beam > Issue Type: Bug > Components: sdk-py-harness, test-failures >Reporter: Scott Wegner >Assignee: Robert Bradshaw >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/] > * [Gradle Build > Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests] > * [Test source > code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390] > Initial investigation: > Seems to be failing on pip download. > == > ERROR: test_multiple_empty_outputs > (apache_beam.transforms.ptransform_test.PTransformTest) > -- > Traceback (most recent call last): > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py", > line 277, in test_multiple_empty_outputs > pipeline.run() > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py", > line 104, in run > result = super(TestPipeline, self).run(test_runner_api) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 403, in run > self.to_runner_api(), self.runner, self._options).run(False) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 416, in run > return self.runner.run_pipeline(self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", > line 50, in run_pipeline > self.result = super(TestDataflowRunner, self).run_pipeline(pipeline) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", > line 389, in run_pipeline > self.dataflow_client.create_job(self.job), self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py", > line 184, in wrapper > return fun(*args, **kwargs) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 490, in create_job > self.create_job_description(job) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 519, in create_job_description > resources = self._stage_resour > ces(job.options) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 452, in _stage_resources > staging_location=google_cloud_options.staging_location) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 161, in stage_job_resources > requirements_cache_path) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 411, in _populate_requirements_cache > processes.check_call(cmd_args) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py", > line 46, in check_call > return subprocess.check_call(*args, **kwargs) > File "/usr/lib/python2.7/subprocess.py", line 541, in check_call > raise CalledProcessError(retcode, cmd) > CalledProcessError: Command > '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python', > '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', > 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary',
[jira] [Assigned] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-5683: -- Assignee: Ankur Goenka (was: Robert Bradshaw) > [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary > -- > > Key: BEAM-5683 > URL: https://issues.apache.org/jira/browse/BEAM-5683 > Project: Beam > Issue Type: Bug > Components: sdk-py-harness, test-failures >Reporter: Scott Wegner >Assignee: Ankur Goenka >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/] > * [Gradle Build > Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests] > * [Test source > code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390] > Initial investigation: > Seems to be failing on pip download. > == > ERROR: test_multiple_empty_outputs > (apache_beam.transforms.ptransform_test.PTransformTest) > -- > Traceback (most recent call last): > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py", > line 277, in test_multiple_empty_outputs > pipeline.run() > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py", > line 104, in run > result = super(TestPipeline, self).run(test_runner_api) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 403, in run > self.to_runner_api(), self.runner, self._options).run(False) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", > line 416, in run > return self.runner.run_pipeline(self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", > line 50, in run_pipeline > self.result = super(TestDataflowRunner, self).run_pipeline(pipeline) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", > line 389, in run_pipeline > self.dataflow_client.create_job(self.job), self) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py", > line 184, in wrapper > return fun(*args, **kwargs) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 490, in create_job > self.create_job_description(job) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 519, in create_job_description > resources = self._stage_resour > ces(job.options) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", > line 452, in _stage_resources > staging_location=google_cloud_options.staging_location) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 161, in stage_job_resources > requirements_cache_path) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", > line 411, in _populate_requirements_cache > processes.check_call(cmd_args) > File > "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py", > line 46, in check_call > return subprocess.check_call(*args, **kwargs) > File "/usr/lib/python2.7/subprocess.py", line 541, in check_call > raise CalledProcessError(retcode, cmd) > CalledProcessError: Command > '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python', > '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', > 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', > ':all:']' returned non-zero exit status 1 > > _After you've filled out the above details, please [assign the issue to an >
[jira] [Updated] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
[ https://issues.apache.org/jira/browse/BEAM-5683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5683: --- Description: _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/] * [Gradle Build Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests] * [Test source code|https://github.com/apache/beam/blob/303a4275eb0a323761e1a4dec6a22fde9863acf8/sdks/python/apache_beam/runners/portability/stager.py#L390] Initial investigation: Seems to be failing on pip download. == ERROR: test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) -- Traceback (most recent call last): File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py", line 277, in test_multiple_empty_outputs pipeline.run() File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py", line 104, in run result = super(TestPipeline, self).run(test_runner_api) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", line 403, in run self.to_runner_api(), self.runner, self._options).run(False) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", line 416, in run return self.runner.run_pipeline(self) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", line 50, in run_pipeline self.result = super(TestDataflowRunner, self).run_pipeline(pipeline) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", line 389, in run_pipeline self.dataflow_client.create_job(self.job), self) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py", line 184, in wrapper return fun(*args, **kwargs) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 490, in create_job self.create_job_description(job) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 519, in create_job_description resources = self._stage_resour ces(job.options) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 452, in _stage_resources staging_location=google_cloud_options.staging_location) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", line 161, in stage_job_resources requirements_cache_path) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", line 411, in _populate_requirements_cache processes.check_call(cmd_args) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py", line 46, in check_call return subprocess.check_call(*args, **kwargs) File "/usr/lib/python2.7/subprocess.py", line 541, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python', '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']' returned non-zero exit status 1 _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ was: _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/] * [Gradle Build Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests] * [Test source code|TODO] Initial investigation: Seems to be failing on pip download.
[jira] [Created] (BEAM-5683) [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary
Scott Wegner created BEAM-5683: -- Summary: [beam_PostCommit_Py_VR_Dataflow] [test_multiple_empty_outputs] Failure summary Key: BEAM-5683 URL: https://issues.apache.org/jira/browse/BEAM-5683 Project: Beam Issue Type: Bug Components: sdk-py-harness, test-failures Reporter: Scott Wegner Assignee: Robert Bradshaw _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Py_VR_Dataflow/1289/] * [Gradle Build Scan|https://scans.gradle.com/s/hjmzvh4ylhs6y/console-log?task=:beam-sdks-python:validatesRunnerBatchTests] * [Test source code|TODO] Initial investigation: Seems to be failing on pip download. == ERROR: test_multiple_empty_outputs (apache_beam.transforms.ptransform_test.PTransformTest) -- Traceback (most recent call last): File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/transforms/ptransform_test.py", line 277, in test_multiple_empty_outputs pipeline.run() File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/testing/test_pipeline.py", line 104, in run result = super(TestPipeline, self).run(test_runner_api) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", line 403, in run self.to_runner_api(), self.runner, self._options).run(False) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/pipeline.py", line 416, in run return self.runner.run_pipeline(self) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/test_dataflow_runner.py", line 50, in run_pipeline self.result = super(TestDataflowRunner, self).run_pipeline(pipeline) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/dataflow_runner.py", line 389, in run_pipeline self.dataflow_client.create_job(self.job), self) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/retry.py", line 184, in wrapper return fun(*args, **kwargs) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 490, in create_job self.create_job_description(job) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 519, in create_job_description resources = self._stage_resour ces(job.options) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/dataflow/internal/apiclient.py", line 452, in _stage_resources staging_location=google_cloud_options.staging_location) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", line 161, in stage_job_resources requirements_cache_path) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/runners/portability/stager.py", line 411, in _populate_requirements_cache processes.check_call(cmd_args) File "/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/apache_beam/utils/processes.py", line 46, in check_call return subprocess.check_call(*args, **kwargs) File "/usr/lib/python2.7/subprocess.py", line 541, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '['/home/jenkins/jenkins-slave/workspace/beam_PostCommit_Py_VR_Dataflow/src/sdks/python/build/gradleenv/bin/python', '-m', 'pip', 'download', '--dest', '/tmp/dataflow-requirements-cache', '-r', 'postcommit_requirements.txt', '--exists-action', 'i', '--no-binary', ':all:']' returned non-zero exit status 1 _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
[ https://issues.apache.org/jira/browse/BEAM-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642132#comment-16642132 ] Scott Wegner commented on BEAM-5681: [beam_PreCommit_Website_Stage_GCS_Cron|https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Cron/] seems to be [failing for the same reason|https://scans.gradle.com/s/k2ggapk6a3hmm/console-log?task=:beam-website:stageWebsite], but with a better error message: {{Build file '/home/jenkins/jenkins-slave/workspace/beam_PreCommit_Website_Stage_GCS_Cron/src/website/build.gradle' line: 234}} {{assert githubPullRequestId.isInteger()}} {{ | |}} {{ ${ghprbPullId} false}} > [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks > --- > > Key: BEAM-5681 > URL: https://issues.apache.org/jira/browse/BEAM-5681 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Alan Myrvold >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154] > * [Gradle Build > Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27] > * [Test source > code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140] > Initial investigation: > It seems every link is failing. Same error message: > {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}} > {{ * http:// is an invalid URL (line 76)}} > {{ }} > {{ src="//images/beam_logo_navbar.png">}} > {{ }} > The _Cron version of this job has been failing consistently since 10/5 > ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/]) > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
[ https://issues.apache.org/jira/browse/BEAM-5681?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642127#comment-16642127 ] Scott Wegner commented on BEAM-5681: I wonder if it's due to the \-PgithubPullRequestId=${ghprbPullId} parameter to the job. The 'Cron' version of the job doesn't run as part of a pull-request, so this parameter will be empty. I'm able to repro this by running locally and explicitly setting the parameter to whitespace (\-PgithubPullRequestId=" "): [https://gradle.com/s/th6k66yjmmxj4] [~alanmyrvold] could you please investigate? > [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks > --- > > Key: BEAM-5681 > URL: https://issues.apache.org/jira/browse/BEAM-5681 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Alan Myrvold >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154] > * [Gradle Build > Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27] > * [Test source > code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140] > Initial investigation: > It seems every link is failing. Same error message: > {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}} > {{ * http:// is an invalid URL (line 76)}} > {{ }} > {{ src="//images/beam_logo_navbar.png">}} > {{ }} > The _Cron version of this job has been failing consistently since 10/5 > ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/]) > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5681) [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks
Scott Wegner created BEAM-5681: -- Summary: [beam_PreCommit_Website_Cron] [testWebsite] Fails on URL checks Key: BEAM-5681 URL: https://issues.apache.org/jira/browse/BEAM-5681 Project: Beam Issue Type: Bug Components: test-failures, website Reporter: Scott Wegner Assignee: Alan Myrvold _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PreCommit_Website_Cron/154] * [Gradle Build Scan|https://scans.gradle.com/s/k6ul7uyovyyce/console-log?task=:beam-website:testWebsite#L27] * [Test source code|https://github.com/apache/beam/blob/1c573ffba014f6969a689d0e6cdcace31358885d/website/build.gradle#L140] Initial investigation: It seems every link is failing. Same error message: {{- ./generated-content/beam/capability/2016/03/17/capability-matrix.html}} {{ * http:// is an invalid URL (line 76)}} {{ }} {{}} {{ }} The _Cron version of this job has been failing consistently since 10/5 ([history|https://builds.apache.org/job/beam_PreCommit_Website_Cron/]) _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642113#comment-16642113 ] Scott Wegner commented on BEAM-5680: I've filed an issue with INFRA. > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5680: --- Description: _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] * [Gradle Build Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] * [Test source code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] Initial investigation: Failing consistently since 10/7 5am until now ([test history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]). _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ was: _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] * [Gradle Build Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] * [Test source code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] Initial investigation: Failing consistently since 10/7 5am until now ([test history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]. _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]). > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Assigned] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reassigned BEAM-5680: -- Assignee: Scott Wegner (was: Melissa Pashniak) > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642098#comment-16642098 ] Scott Wegner commented on BEAM-5680: Another recent job failure, [Mesos-Websitebot|https://builds.apache.org/job/Mesos-Websitebot/lastSuccessfulBuild/], uses gitbox and has a similar failure messages: {{fatal: Authentication failed for 'https://gitbox.apache.org/repos/asf/mesos-site.git/'}} {{Build step 'Execute shell' marked build as failure}} I'm going to reach out to Apache INFRA. > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Melissa Pashniak >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642096#comment-16642096 ] Scott Wegner commented on BEAM-5680: I see other successful jobs run in the [git-website|https://builds.apache.org/label/git-websites/] label ([Solr-reference-guide-master|https://builds.apache.org/job/Solr-reference-guide-master/lastSuccessfulBuild/], [hbase_generate_website|https://builds.apache.org/job/hbase_generate_website/lastSuccessfulBuild/], [Solr-reference-guide-7.x|https://builds.apache.org/job/Solr-reference-guide-7.x/lastSuccessfulBuild/], [Incubator Site|https://builds.apache.org/job/Incubator%20Site/lastSuccessfulBuild/]), but none of them seem to be using gitbox. > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Melissa Pashniak >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642078#comment-16642078 ] Scott Wegner commented on BEAM-5680: Nothing posted to https://status.apache.org/ > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Melissa Pashniak >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642075#comment-16642075 ] Scott Wegner commented on BEAM-5680: /cc [~alanmyrvold] > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Melissa Pashniak >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
Scott Wegner created BEAM-5680: -- Summary: [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed' Key: BEAM-5680 URL: https://issues.apache.org/jira/browse/BEAM-5680 Project: Beam Issue Type: Bug Components: test-failures, website Reporter: Scott Wegner Assignee: Melissa Pashniak _Use this form to file an issue for test failure:_ * [Jenkins Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] * [Gradle Build Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] * [Test source code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] Initial investigation: Failing consistently since 10/7 5am until now ([test history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]. _After you've filled out the above details, please [assign the issue to an individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. Assignee should [treat test failures as high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], helping to fix the issue or find a more appropriate owner. See [Apache Beam Post-Commit Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5680) [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication failed'
[ https://issues.apache.org/jira/browse/BEAM-5680?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642077#comment-16642077 ] Scott Wegner commented on BEAM-5680: The job correctly ran on [website1|https://builds.apache.org/computer/websites1] Jenkins node, which is the only thing that should be required to push to gitbox. > [beam_PostCommit_Website_Publish] [:publishWebsite] git push 'Authentication > failed' > > > Key: BEAM-5680 > URL: https://issues.apache.org/jira/browse/BEAM-5680 > Project: Beam > Issue Type: Bug > Components: test-failures, website >Reporter: Scott Wegner >Assignee: Melissa Pashniak >Priority: Major > Labels: currently-failing > > _Use this form to file an issue for test failure:_ > * [Jenkins > Job|https://builds.apache.org/job/beam_PostCommit_Website_Publish/131/] > * [Gradle Build > Scan|https://scans.gradle.com/s/wkk4k3jp3l5ve/console-log?task=:beam-website:publishWebsite] > * [Test source > code|https://github.com/apache/beam/blob/a19183b05f0271f0a927aafcd778235335b7d269/website/build.gradle#L221] > Initial investigation: > Failing consistently since 10/7 5am until now ([test > history|https://builds.apache.org/job/beam_PostCommit_Website_Publish/buildTimeTrend]. > > _After you've filled out the above details, please [assign the issue to an > individual|https://beam.apache.org/contribute/postcommits-guides/index.html#find_specialist]. > Assignee should [treat test failures as > high-priority|https://beam.apache.org/contribute/postcommits-policies/#assigned-failing-test], > helping to fix the issue or find a more appropriate owner. See [Apache Beam > Post-Commit > Policies|https://beam.apache.org/contribute/postcommits-policies]._ -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation
[ https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642035#comment-16642035 ] Scott Wegner commented on BEAM-5678: The [html-proofer|https://github.com/gjtorikian/html-proofer#ignoring-content] extension that we use to validate URLs allows specifying a special [{{data-proofer-ignore}} |https://github.com/gjtorikian/html-proofer#ignoring-content] tag in the HTML link to disable validation. We could use the custom redirect page functionality to specify the tag. This should be a sufficient work-around. Ideally we would like to validate that our redirects do work, but since the redirect pages we're excluding are all generated from the Jekyll plugin, the only reason they wouldn't work is due to a bug in the plugin itself. > Jeckyll redirect_from incompatible with HTMLProofer url validation > -- > > Key: BEAM-5678 > URL: https://issues.apache.org/jira/browse/BEAM-5678 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Priority: Minor > > Renaming a URL and using the Jekyll {{redirect_from}} feature breaks > pre-commit validation. > We hit this on > [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]: > bq. The scenario for using a redirect is if you're moving {{pageA.md}} to > {{pageB.md}}; you would rename the file to {{pageB.md}} and add > {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} > with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} > doesn't exist on the livesite yet, so it will always fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation
[ https://issues.apache.org/jira/browse/BEAM-5678?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642031#comment-16642031 ] Scott Wegner commented on BEAM-5678: [~alanmyrvold], [~udim] and I spent some time investigating this and didn't find easy work-arounds. The [jekyll-redirect-from|https://github.com/jekyll/jekyll-redirect-from] plugin responsible for creating the redirect pages uses absolute paths for the "redirect to" URL, which is what causes issues. The plugin allows providing a custom page template, but only provides the absolute redirect-to URL as a variable. > Jeckyll redirect_from incompatible with HTMLProofer url validation > -- > > Key: BEAM-5678 > URL: https://issues.apache.org/jira/browse/BEAM-5678 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Priority: Minor > > Renaming a URL and using the Jekyll {{redirect_from}} feature breaks > pre-commit validation. > We hit this on > [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]: > bq. The scenario for using a redirect is if you're moving {{pageA.md}} to > {{pageB.md}}; you would rename the file to {{pageB.md}} and add > {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} > with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} > doesn't exist on the livesite yet, so it will always fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5678) Jeckyll redirect_from incompatible with HTMLProofer url validation
Scott Wegner created BEAM-5678: -- Summary: Jeckyll redirect_from incompatible with HTMLProofer url validation Key: BEAM-5678 URL: https://issues.apache.org/jira/browse/BEAM-5678 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner Renaming a URL and using the Jekyll {{redirect_from}} feature breaks pre-commit validation. We hit this on [PR580|https://github.com/apache/beam-site/pull/580#issuecomment-427516952]: bq. The scenario for using a redirect is if you're moving {{pageA.md}} to {{pageB.md}}; you would rename the file to {{pageB.md}} and add {{redirect_from: pageA}}. This will cause jekyll to generate a {{pageA.html}} with a redirect to {{https://beam.apache.org/pageB.html}}. But {{pageB.html}} doesn't exist on the livesite yet, so it will always fail. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job
[ https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642024#comment-16642024 ] Scott Wegner commented on BEAM-5669: re: whether fixing up history would break the website publishing. It appears not. The code for pulling down latest changes is [here|https://github.com/apache/infrastructure-puppet/blob/bae50f68a3215c3db80e8ac1bfe4bfdf45b72663/modules/gitwcsub/files/app/gitwcsub.py#L224], which is essentially: {{git fetch origin/asf-site && git reset --hard origin/asf-site}} Neither {{fetch}} or {{reset --hard}} will complain if history has been re-written. > Empty commits pushed by PostCommit_Website_Publish jenkins job > -- > > Key: BEAM-5669 > URL: https://issues.apache.org/jira/browse/BEAM-5669 > Project: Beam > Issue Type: Bug > Components: build-system >Affects Versions: 2.8.0 >Reporter: Alan Myrvold >Assignee: Alan Myrvold >Priority: Major > Fix For: 2.8.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The isClean() call in > https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165 > is returning false due to unstaged .gradle and build files. > The asf-site branch needs a .gitignore, which can be the same as the master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job
[ https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642013#comment-16642013 ] Scott Wegner commented on BEAM-5669: The implications of these empty commits is extra commit spam for notifications, during fetch, and wasted CPU cycles for the Apache daemon publishing the website. After we get this fixed, a secondary question is whether we should go back and cleanup the empty commits. It would be easy to do with a git command, but it would re-write the git history. I'm not sure if this would cause any issues with the Apache gitpubsub publishing agent. > Empty commits pushed by PostCommit_Website_Publish jenkins job > -- > > Key: BEAM-5669 > URL: https://issues.apache.org/jira/browse/BEAM-5669 > Project: Beam > Issue Type: Bug > Components: build-system >Affects Versions: 2.8.0 >Reporter: Alan Myrvold >Assignee: Alan Myrvold >Priority: Major > Fix For: 2.8.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The isClean() call in > https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165 > is returning false due to unstaged .gradle and build files. > The asf-site branch needs a .gitignore, which can be the same as the master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Reopened] (BEAM-5669) Empty commits pushed by PostCommit_Website_Publish jenkins job
[ https://issues.apache.org/jira/browse/BEAM-5669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner reopened BEAM-5669: [~thw] [pointed out|https://issues.apache.org/jira/browse/BEAM-5671?focusedCommentId=16640524=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-16640524] this is still broken. [~alanmyrvold] can you take a look? > Empty commits pushed by PostCommit_Website_Publish jenkins job > -- > > Key: BEAM-5669 > URL: https://issues.apache.org/jira/browse/BEAM-5669 > Project: Beam > Issue Type: Bug > Components: build-system >Affects Versions: 2.8.0 >Reporter: Alan Myrvold >Assignee: Alan Myrvold >Priority: Major > Fix For: 2.8.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > The isClean() call in > https://github.com/apache/beam/blob/67562393de09e1e8e24c4a83ca5274f57c8379bb/website/build.gradle#L165 > is returning false due to unstaged .gradle and build files. > The asf-site branch needs a .gitignore, which can be the same as the master -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Commented] (BEAM-5671) Website automation fit-and-finish
[ https://issues.apache.org/jira/browse/BEAM-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16642006#comment-16642006 ] Scott Wegner commented on BEAM-5671: Thanks for the heads-up. [~alanmyrvold] tried to fix this with https://github.com/apache/beam/pull/6591, but I suppose the fix didn't work-- I'll reopen https://issues.apache.org/jira/browse/BEAM-5669 > Website automation fit-and-finish > - > > Key: BEAM-5671 > URL: https://issues.apache.org/jira/browse/BEAM-5671 > Project: Beam > Issue Type: Wish > Components: website >Reporter: Scott Wegner >Assignee: Melissa Pashniak >Priority: Major > > We are wrapping up the work for migrating the Beam Website > (https://s.apache.org/beam-site-automation); there are a few fit-and-finish > items that remain. If you notice things that should be improved, please add > them here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5672) Staged URL should be posted to GitHub PR
Scott Wegner created BEAM-5672: -- Summary: Staged URL should be posted to GitHub PR Key: BEAM-5672 URL: https://issues.apache.org/jira/browse/BEAM-5672 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner Website PRs are automatically tested and staged via Jenkins to a location on GCS: https://builds.apache.org/job/beam_PreCommit_Website_Stage_GCS_Commit/ However, it's difficult to find the URL for the staged job; it's buried inside the Jenkins job, inside the Gradle scan, inside the log. I believe there's an API for posting back a context message to the pull request. If possible, we should push the URL so it's easily viewable from the pull request. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5459) Publish javadocs/pydocs to an external source, i.e. buildbot
[ https://issues.apache.org/jira/browse/BEAM-5459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5459: --- Issue Type: Task (was: Sub-task) Parent: (was: BEAM-4493) > Publish javadocs/pydocs to an external source, i.e. buildbot > > > Key: BEAM-5459 > URL: https://issues.apache.org/jira/browse/BEAM-5459 > Project: Beam > Issue Type: Task > Components: website >Reporter: Scott Wegner >Priority: Minor > > Javadocs and Pydocs are generated for the website at every release, and the > generated content gets committed to the git repository in order to publish > via the githubpubsub publishing mechanism. Keeping all of this generated > content in git is cumbersome and seems unnecessary. Alternatives exist, for > example the Flink project uses a buildbot job to build and publish their > javadocs: > https://ci.apache.org/projects/flink/flink-docs-release-1.5/ > The buildbot configuration is here (requires committer access): > https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/flink.conf -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5459) Publish javadocs/pydocs to an external source, i.e. buildbot
[ https://issues.apache.org/jira/browse/BEAM-5459?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5459: --- Issue Type: Sub-task (was: Task) Parent: BEAM-5671 > Publish javadocs/pydocs to an external source, i.e. buildbot > > > Key: BEAM-5459 > URL: https://issues.apache.org/jira/browse/BEAM-5459 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Priority: Minor > > Javadocs and Pydocs are generated for the website at every release, and the > generated content gets committed to the git repository in order to publish > via the githubpubsub publishing mechanism. Keeping all of this generated > content in git is cumbersome and seems unnecessary. Alternatives exist, for > example the Flink project uses a buildbot job to build and publish their > javadocs: > https://ci.apache.org/projects/flink/flink-docs-release-1.5/ > The buildbot configuration is here (requires committer access): > https://svn.apache.org/repos/infra/infrastructure/buildbot/aegis/buildmaster/master1/projects/flink.conf -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-4497) Add pages for master Javadocs / Pydocs and incorporate into post-commit job
[ https://issues.apache.org/jira/browse/BEAM-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-4497: --- Issue Type: Sub-task (was: Task) Parent: BEAM-5671 > Add pages for master Javadocs / Pydocs and incorporate into post-commit job > --- > > Key: BEAM-4497 > URL: https://issues.apache.org/jira/browse/BEAM-4497 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: beam-site-automation-reliability > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-5671) Website automation fit-and-finish
[ https://issues.apache.org/jira/browse/BEAM-5671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-5671: --- Issue Type: Wish (was: Sub-task) Parent: (was: BEAM-4493) > Website automation fit-and-finish > - > > Key: BEAM-5671 > URL: https://issues.apache.org/jira/browse/BEAM-5671 > Project: Beam > Issue Type: Wish > Components: website >Reporter: Scott Wegner >Assignee: Melissa Pashniak >Priority: Major > > We are wrapping up the work for migrating the Beam Website > (https://s.apache.org/beam-site-automation); there are a few fit-and-finish > items that remain. If you notice things that should be improved, please add > them here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Updated] (BEAM-4497) Add pages for master Javadocs / Pydocs and incorporate into post-commit job
[ https://issues.apache.org/jira/browse/BEAM-4497?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner updated BEAM-4497: --- Issue Type: Task (was: Sub-task) Parent: (was: BEAM-4493) > Add pages for master Javadocs / Pydocs and incorporate into post-commit job > --- > > Key: BEAM-4497 > URL: https://issues.apache.org/jira/browse/BEAM-4497 > Project: Beam > Issue Type: Task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: beam-site-automation-reliability > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Created] (BEAM-5671) Website automation fit-and-finish
Scott Wegner created BEAM-5671: -- Summary: Website automation fit-and-finish Key: BEAM-5671 URL: https://issues.apache.org/jira/browse/BEAM-5671 Project: Beam Issue Type: Sub-task Components: website Reporter: Scott Wegner Assignee: Melissa Pashniak We are wrapping up the work for migrating the Beam Website (https://s.apache.org/beam-site-automation); there are a few fit-and-finish items that remain. If you notice things that should be improved, please add them here. -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-4499) Migrate Apache website publishing to use apache/beam asf-site branch
[ https://issues.apache.org/jira/browse/BEAM-4499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-4499. Resolution: Fixed Fix Version/s: 2.8.0 > Migrate Apache website publishing to use apache/beam asf-site branch > > > Key: BEAM-4499 > URL: https://issues.apache.org/jira/browse/BEAM-4499 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Scott Wegner >Assignee: Scott Wegner >Priority: Major > Labels: beam-site-automation-reliability > Fix For: 2.8.0 > > Time Spent: 2h > Remaining Estimate: 0h > -- This message was sent by Atlassian JIRA (v7.6.3#76005)
[jira] [Resolved] (BEAM-5368) Re-enable apache/website build once it is stable
[ https://issues.apache.org/jira/browse/BEAM-5368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Scott Wegner resolved BEAM-5368. Resolution: Fixed Fix Version/s: Not applicable > Re-enable apache/website build once it is stable > > > Key: BEAM-5368 > URL: https://issues.apache.org/jira/browse/BEAM-5368 > Project: Beam > Issue Type: Sub-task > Components: website >Reporter: Batkhuyag Batsaikhan >Assignee: Scott Wegner >Priority: Major > Fix For: Not applicable > > > We are disabling beam check in > [https://github.com/apache/beam/pull/6370|https://github.com/apache/beam/pull/6370] > due to https://issues.apache.org/jira/browse/BEAM-5367. -- This message was sent by Atlassian JIRA (v7.6.3#76005)