[
https://issues.apache.org/jira/browse/BEAM-1909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16020138#comment-16020138
]
Chamikara Jayalath commented on BEAM-1909:
------------------------------------------
Hi Uwe,
The issue here is not creating a table/dataset in the correct region for
executing queries for non-US regions. We can do this if we know the region.
You can determine the region using a dry-run of the query. When you dry-run the
query you should get statistics about the query including the set of tables
referenced. This information can be used to determine the region (pick one of
the tables and get the dataset).
An example response to a dry-run query is given below.
<Job
configuration: <JobConfiguration
dryRun: True
query: <JobConfigurationQuery
allowLargeResults: True
destinationTable: <TableReference
datasetId: u'temp_dataset_a33db6e872704a238568ba77f6b55e80'
projectId: <my project>
tableId: u'temp_table_a33db6e872704a238568ba77f6b55e80'>
flattenResults: True
query: u'SELECT * FROM [clouddataflow-readonly:samples.weather_stations];'
useLegacySql: True
userDefinedFunctionResources: []>>
etag: u'"a19Ag3pFAfODmeUgLyZLAGQkEbw/9Uc9pvfbc0cqTPPIgjuvzASt7UY"'
jobReference: <JobReference
projectId: <my-project>>
kind: u'bigquery#job'
statistics: <JobStatistics
creationTime: 1495484353438
query: <JobStatistics2
cacheHit: False
queryPlan: []
referencedTables: [<TableReference
datasetId: u'samples'
projectId: u'clouddataflow-readonly'
tableId: u'weather_stations'>]
totalBytesBilled: 0
totalBytesProcessed: 151184>
totalBytesProcessed: 151184>
status: <JobStatus
errors: []
state: u'DONE'>
user_email: <my-email>'>
- Cham
> BigQuery read transform fails for DirectRunner when querying non-US regions
> ---------------------------------------------------------------------------
>
> Key: BEAM-1909
> URL: https://issues.apache.org/jira/browse/BEAM-1909
> Project: Beam
> Issue Type: Bug
> Components: sdk-py
> Reporter: Chamikara Jayalath
>
> See:
> http://stackoverflow.com/questions/42135002/google-dataflow-cannot-read-and-write-in-different-locations-python-sdk-v0-5-5/42144748?noredirect=1#comment73621983_42144748
> This should be fixed by creating the temp dataset and table in the correct
> region.
> cc: [~sb2nov]
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
