[
https://issues.apache.org/jira/browse/BEAM-6049?focusedWorklogId=199457&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-199457
]
ASF GitHub Bot logged work on BEAM-6049:
----------------------------------------
Author: ASF GitHub Bot
Created on: 15/Feb/19 21:55
Start Date: 15/Feb/19 21:55
Worklog Time Spent: 10m
Work Description: bbhoss commented on issue #7047: [BEAM-6049] Add option
to load job to GCS in Dataflow Runner
URL: https://github.com/apache/beam/pull/7047#issuecomment-464216336
Is this functional? We just upgraded to 2.9 and are hitting the size
limitation on one of the bigger jobs. I have specified the following options on
the command line:
```
--experiments=upload_graph --tempLocation=gs://mybucket/foo
--stagingLocation=gs://mybucket/foo
```
I have verified the graph is uploaded to the directory, but dataflow
immediately fails due to
`Failed to retrieve the job graph. Please see the job logs for more details.`
The logs that I can see don't have any details (Logs in the Dataflow UI and
Stackdriver are blank other than this message)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 199457)
Time Spent: 2h 50m (was: 2h 40m)
> Support Job / Graph upload to GCS in Dataflow Runner
> ----------------------------------------------------
>
> Key: BEAM-6049
> URL: https://issues.apache.org/jira/browse/BEAM-6049
> Project: Beam
> Issue Type: Improvement
> Components: runner-dataflow
> Reporter: Andrea Foegler
> Assignee: Andrea Foegler
> Priority: Major
> Labels: triaged
> Time Spent: 2h 50m
> Remaining Estimate: 0h
>
> Add uploadGraph flag to support uploading the job / graph to GCS instead of
> embedding in the request. This change allows the API to support much larger
> graphs than the embedded format of CreateJob.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
