[
https://issues.apache.org/jira/browse/BEAM-3990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Udi Meiri closed BEAM-3990.
---------------------------
Resolution: Fixed
Fix Version/s: Not applicable
> Dataflow jobs fail with "KeyError: 'location'" when uploading to GCS
> --------------------------------------------------------------------
>
> Key: BEAM-3990
> URL: https://issues.apache.org/jira/browse/BEAM-3990
> Project: Beam
> Issue Type: Bug
> Components: sdk-py-core
> Reporter: Chamikara Jayalath
> Priority: Major
> Fix For: Not applicable
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Some Dataflow jobs are failing due to following error (in worker logs).
>
> Error in _start_upload while inserting file
> gs://cloud-ml-benchmark-output-us-central/df1-cloudml-benchmark-criteo-small-python-033010274088282-presubmit3/033010274088282/temp/df1-cloudml-benchmark-criteo-small-python-033010274088282-presubmit3.1522430898.446147/dax-tmp-2018-03-30_10_28_40-14595186994726940229-S241-1-dc87ef69274882bf/tmp-dc87ef6927488c5a-shard--try-308ae8b3268d12b2-endshard.avro:
> Traceback (most recent call last): File
> "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/gcsio.py", line
> 559, in _start_upload self._client.objects.Insert(self._insert_request,
> upload=self._upload) File
> "/usr/local/lib/python2.7/dist-packages/apache_beam/io/gcp/internal/clients/storage/storage_v1_client.py",
> line 971, in Insert download=download) File
> "/usr/local/lib/python2.7/dist-packages/apitools/base/py/base_api.py", line
> 706, in _RunMethod http_request, client=self.client) File
> "/usr/local/lib/python2.7/dist-packages/apitools/base/py/transfer.py", line
> 860, in InitializeUpload url =
> [http_response.info|https://www.google.com/url?q=http://http_response.info&sa=D&usg=AFQjCNGvYHYJBb_G4YNo3VvGoqX2Gq-6Yw]['location']
> KeyError: 'location'
>
> This seems to be due to [https://github.com/apache/beam/pull/4891.] Possibly
> storage.StorageV1() cannot be shared across multiple requests without
> additional fixes.
--
This message was sent by Atlassian JIRA
(v7.6.14#76016)
