[
https://issues.apache.org/jira/browse/BEAM-3406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16314008#comment-16314008
]
Alan Myrvold commented on BEAM-3406:
------------------------------------
Created the table and loaded the two records in the attached file.
Used the superset schema from
https://github.com/GoogleCloudPlatform/PerfKitExplorer/blob/master/data/samples_mart/results_table_schema.json
bq load --source_format=NEWLINE_DELIMITED_JSON
apache-beam-testing:beam_performance.filebasedioit_pkb_results textiot.txt
schema.txt
This painpoint of needing to create the table seems to be
https://github.com/GoogleCloudPlatform/PerfKitBenchmarker/issues/229
> FIeBasedIO IT fails due to "BigQuery error"
> -------------------------------------------
>
> Key: BEAM-3406
> URL: https://issues.apache.org/jira/browse/BEAM-3406
> Project: Beam
> Issue Type: Bug
> Components: testing
> Reporter: Chamikara Jayalath
> Assignee: Łukasz Gajowy
> Attachments: schema, textioit
>
>
> For example,
> https://builds.apache.org/view/A-D/view/Beam/job/beam_PerformanceTests_FileBasedIO_IT/54/console
> beam_performance.filebasedioit_pkb_results
> /tmp/perfkitbenchmarker/runs/e2e3d02d/perfkit-bq-pub3ntgN3.json. Got return
> code (1).
> STDOUT: BigQuery error in load operation: No schema specified on job or table.
> STDERR:
> /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73:
> UserWarning: You have requested explicit scopes to be used with a GCE
> service account.
> Using this argument will have no effect on the actual scopes for tokens
> requested. These scopes are set at VM instance creation time and
> can't be overridden in the request.
> warnings.warn(_SCOPES_WARNING)
> Lukasz and Darek, can you take a look ?
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
