FYI, the docker was restarted on beam15. On Tue, Oct 23, 2018 at 7:08 AM Thomas Weise <t...@apache.org> wrote:
> For the latter (createProcessWorker): > https://github.com/apache/beam/pull/6793 > > > On Tue, Oct 23, 2018 at 6:47 AM Thomas Weise <t...@apache.org> wrote: > >> Thanks for taking a look Yifan. Yes, it appears this was an intermittent >> issue. >> >> For beam_PostCommit_Python_VR_Flink we are left with: >> >> * beam15 docker errors >> * segmentation faults >> * "Execution failed for task ':beam-sdks-python:createProcessWorker'" - >> which should not even execute since we are using Docker >> >> >> On Mon, Oct 22, 2018 at 10:50 PM Yifan Zou <yifan...@google.com> wrote: >> >>> I'm not able to reproduce that error in Beam6 (#459 >>> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/459/>, >>> #460 >>> <https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/460/>), >>> it probably due to some outage of Debian [1]. The image was successfully >>> built, but the test failed in other reasons. >>> And indeed, the beam_PostCommit_Python_VR_Flink is very flaky. >>> >>> Yifan >>> >>> [1] https://github.com/docker-library/python/issues/241 >>> >>> On Mon, Oct 22, 2018 at 5:39 PM Thomas Weise <t...@apache.org> wrote: >>> >>>> Looks like we have more container build related errors. >>>> >>>> This is from beam6 - >>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink_PR/44/ >>>> >>>> Reading package lists... >>>> [91mW: The repository 'http://deb.debian.org/debian stretch Release' >>>> does not have a Release file. >>>> >>>> W: The repository 'http://deb.debian.org/debian stretch-updates Release' >>>> does not have a Release file. >>>> E: Failed to fetch >>>> http://deb.debian.org/debian/dists/stretch/main/binary-amd64/Packages 404 >>>> Not Found >>>> E: Failed to fetch >>>> http://deb.debian.org/debian/dists/stretch-updates/main/binary-amd64/Packages >>>> 404 Not Found >>>> E: Some index files failed to download. They have been ignored, or old >>>> ones used instead. >>>> >>>> >>>> On Mon, Oct 22, 2018 at 2:54 PM Ankur Goenka <goe...@google.com> wrote: >>>> >>>>> Thanks Yifan! >>>>> >>>>> On Mon, Oct 22, 2018 at 2:53 PM Yifan Zou <yifan...@google.com> wrote: >>>>> >>>>>> So, looks like none of us have the permissions. I filed INFRA-17167 >>>>>> <https://issues.apache.org/jira/browse/INFRA-17167> to the Infra >>>>>> team to restart the docker on the beam15. >>>>>> >>>>>> Thanks. >>>>>> Yifan >>>>>> >>>>>> On Mon, Oct 22, 2018 at 9:20 AM Scott Wegner <sc...@apache.org> >>>>>> wrote: >>>>>> >>>>>>> I've seen the docker issue pop-up on website pre-commits as well: >>>>>>> https://issues.apache.org/jira/browse/BEAM-5783. There were also on >>>>>>> beam15. >>>>>>> >>>>>>> When I searched around the internet I found lots of instances of the >>>>>>> same error; it seems to be some unreliability in the guts of Docker [1]. >>>>>>> Perhaps restarting the VM or docker daemon could help. Does anybody have >>>>>>> permissions to log on and try it? >>>>>>> >>>>>>> [1] https://github.com/moby/moby/issues/31849#issuecomment-320236354 >>>>>>> >>>>>>> On Sun, Oct 21, 2018 at 7:13 PM Thomas Weise <t...@apache.org> wrote: >>>>>>> >>>>>>>> There are two issues with >>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/ >>>>>>>> currently: >>>>>>>> >>>>>>>> 1) The mentioned issue with docker on beam15 - Jason, can you >>>>>>>> possibly advise how to deal with it? >>>>>>>> >>>>>>>> 2) Frequent failure due to "Segmentation fault (core dumped)", as >>>>>>>> exhibited by >>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/449/consoleText >>>>>>>> >>>>>>>> The Gradle scan is here: >>>>>>>> >>>>>>>> >>>>>>>> https://scans.gradle.com/s/ebhxs4l65cow4/failure?openFailures=WzBd&openStackTraces=WzEse31d#top=0 >>>>>>>> >>>>>>>> There are multiple of those in sequence on beam13 >>>>>>>> >>>>>>>> Some more comments: https://issues.apache.org/jira/browse/BEAM-5467 >>>>>>>> >>>>>>>> Any help to further investigate or fix would be appreciated! >>>>>>>> >>>>>>>> Thanks, >>>>>>>> Thomas >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, Oct 19, 2018 at 4:51 PM Yifan Zou <yifan...@google.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> I got "Failed to restart docker.service: Interactive >>>>>>>>> authentication required" while trying to restart the docker on >>>>>>>>> beam15. >>>>>>>>> Does anyone have the permission to do that? Or, we need to ask >>>>>>>>> Apache Infra for help. >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> Yifan >>>>>>>>> >>>>>>>>> On Fri, Oct 19, 2018 at 2:51 PM Ankur Goenka <goe...@google.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi, >>>>>>>>>> >>>>>>>>>> Can we restart docker as it seems to have fixed the issue for >>>>>>>>>> others https://github.com/moby/moby/issues/31849 ? >>>>>>>>>> >>>>>>>>>> Thanks, >>>>>>>>>> Ankur >>>>>>>>>> >>>>>>>>>> On Fri, Oct 19, 2018 at 1:11 PM Yifan Zou <yifan...@google.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>>> Hi, >>>>>>>>>>> >>>>>>>>>>> The docker has been installed on all Jenkins VMs. The image >>>>>>>>>>> build process was interrupted by a grpc connection issue. >>>>>>>>>>> >>>>>>>>>>> *11:02:12* Starting process 'command 'docker''. Working directory: >>>>>>>>>>> /home/jenkins/jenkins-slave/workspace/beam_PostCommit_Python_VR_Flink/src/sdks/python/container/build/docker >>>>>>>>>>> Command: docker build --no-cache -t >>>>>>>>>>> jenkins-docker-apache.bintray.io/beam/python:latest .*11:02:12* >>>>>>>>>>> Successfully started process 'command 'docker''*11:02:12* Sending >>>>>>>>>>> build context to Docker daemon 17.65MB >>>>>>>>>>> *11:02:12* Step 1/9 : FROM python:2-stretch*11:02:12* ---> >>>>>>>>>>> 3c43a5d4034a*11:02:12* Step 2/9 : MAINTAINER "Apache Beam >>>>>>>>>>> <dev@beam.apache.org>"*11:02:12* ---> Running in >>>>>>>>>>> f86bad9aef9c*11:02:12* ---> 610a5dec907e*11:02:12* Removing >>>>>>>>>>> intermediate container f86bad9aef9c*11:02:12* Step 3/9 : RUN >>>>>>>>>>> apt-get update && apt-get install -y libsnappy-dev >>>>>>>>>>> libyaml-dev && rm -rf /var/lib/apt/lists/**11:02:12* >>>>>>>>>>> ---> Running in 5e9b67be03f9*11:02:12* grpc: the connection is >>>>>>>>>>> unavailable >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> - Yifan >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Fri, Oct 19, 2018 at 12:45 PM Ankur Goenka <goe...@google.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>>> Hi, >>>>>>>>>>>> >>>>>>>>>>>> Flink Validates Runner test cases are failing on Beam 15 >>>>>>>>>>>> because docker is not installed. >>>>>>>>>>>> Failing tasks >>>>>>>>>>>> https://builds.apache.org/job/beam_PostCommit_Python_VR_Flink/buildTimeTrend >>>>>>>>>>>> Can we install docker on all the machines as the Portable >>>>>>>>>>>> Validates Runner tests need it. >>>>>>>>>>>> >>>>>>>>>>>> Thanks, >>>>>>>>>>>> Ankur >>>>>>>>>>>> >>>>>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> Got feedback? tinyurl.com/swegner-feedback >>>>>>> >>>>>>
