My thinking was that anything created in the file system (like the python
environment) should already be properly isolated within each job's
workspace, but docker image tags are shared.
Brian
On Thu, Mar 26, 2020 at 1:44 PM Kyle Weaver <kcwea...@google.com> wrote:
> Is it possible the Python environment the Docker image built from was
> contaminated to begin with?
>
> On Thu, Mar 26, 2020 at 11:06 AM Udi Meiri <eh...@google.com> wrote:
>
>> The Python HDFS IT uses the jenkins BUILD_TAG to create unique names:
>> PROJECT_NAME=$(echo hdfs_IT-${BUILD_TAG:-non-jenkins})
>>
>> The BUILD_TAG is unique and easily traced back to the Jenkins job that
>> made it.
>> It might need some sanitizing though if it contains any invalid
>> characters.
>>
>> On Tue, Mar 24, 2020 at 1:50 PM Hannah Jiang <hannahji...@google.com>
>> wrote:
>>
>>> This can be done by 1). passing "-Pdocker-tag=xxx" to the test and 2).
>>> make sure to specify the custom tag when using docker images.
>>> For example, *:sdks:python:test-suites:portable:py35:preCommitPy35
>>> -Pdocker-tag=20200324 *will create an image with a tag 20200324.
>>> *--environment_config=path/to/container/image* pipeline option can be
>>> used for Python pipeline to pass custom docker images.
>>>
>>>
>>>
>>> On Tue, Mar 24, 2020 at 11:42 AM Brian Hulette <bhule...@google.com>
>>> wrote:
>>>
>>>> Failing run:
>>>> https://builds.apache.org/job/beam_PostCommit_XVR_Flink_PR/65/
>>>> Passing run:
>>>> https://builds.apache.org/job/beam_PostCommit_XVR_Flink_PR/66/
>>>>
>>>> On Tue, Mar 24, 2020 at 11:33 AM Hannah Jiang <hannahji...@google.com>
>>>> wrote:
>>>>
>>>>> Hi Brian
>>>>>
>>>>> I think that's possible if we use the default tag for the Jenkins
>>>>> tests. To prevent this, we can use a customized tag, for example,
>>>>> timestamp, for each build.
>>>>> Can you please point me to the failing tests? I will check more
>>>>> details.
>>>>>
>>>>> Thanks,
>>>>> Hannah
>>>>>
>>>>>
>>>>> On Tue, Mar 24, 2020 at 10:11 AM Brian Hulette <bhule...@google.com>
>>>>> wrote:
>>>>>
>>>>>> I ran into a test failure on the XVR tests in [1] which looked like
>>>>>> the test was executing with a python docker container that did _not_
>>>>>> include the python changes in my PR. The test ran successfully after a
>>>>>> second run.
>>>>>>
>>>>>> It seems likely that the initial failure occurred because some other
>>>>>> job was running concurrently on the same jenkins worker and overwrote
>>>>>> the `
>>>>>> apache/beam_python2.7_sdk:2.21.0.dev` image that my run had
>>>>>> generated. Is this possible? If so, is there something we should do to
>>>>>> isolate these images?
>>>>>>
>>>>>> [1] https://github.com/apache/beam/pull/10055
>>>>>>
>>>>>
