GitHub user ubunatic opened a pull request:
https://github.com/apache/beam/pull/2509
[BEAM-1909] BigQuery read transform fails for DirectRunner when querying
non-US regions
I partially fixed the issue by getting the `location` of the source
`Dataset`. Then I use this location as location of the created temp dataset.
The added parameters are optional and should not break anything.
**Note:** The solution works for me when using the `DirectRunner` if I
create a `BigQuerySource` with `table=<some-table>`. It does not work for a
`BigQuerySource` with `query=<some-query>`.
The corresponding Jira issue should not yet be closed.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/ubunatic/beam temp-table-region
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/beam/pull/2509.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2509
----
commit 79537846431fb679cde85755e4bbdd868cff3eee
Author: Uwe Jugel <[email protected]>
Date: 2017-04-10T12:24:44Z
better log message for bigquery temp tables
commit 652663887c3448046ede70d1f430940e1cfdd9ce
Author: Uwe Jugel <[email protected]>
Date: 2017-04-10T12:30:32Z
fixed spelling, improved wording
commit 4378aceeb9b3ee244452510fb5b729954cf52fea
Author: Uwe Jugel <[email protected]>
Date: 2017-04-10T15:14:48Z
use log formatting instead of string formatting
commit b818777335398fe93d26ffc118ddfc1609f7ddc0
Author: Uwe Jugel <[email protected]>
Date: 2017-04-11T15:10:26Z
Merge remote-tracking branch 'origin/master' into temp-table-log-message
commit 0075addfeab1777feb2e4aa7217936dd882bc52e
Author: Uwe Jugel <[email protected]>
Date: 2017-04-12T12:54:42Z
ignore cythonized files
commit 9f966e60bb29b65588a7aed5267d53e375853887
Author: Uwe Jugel <[email protected]>
Date: 2017-04-12T12:56:50Z
add temp dataset location for non-query BigQuerySource
commit 7d9f5ee0c1705073ae240805d972ad0c6ca07de4
Author: Uwe Jugel <[email protected]>
Date: 2017-04-12T13:01:50Z
Merge branch 'master' into temp-table-region
commit d773fbd44d9f6d8101aca7701702bae2563ee4a5
Author: Uwe Jugel <[email protected]>
Date: 2017-04-12T13:03:42Z
fixed location comment
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
