See <https://builds.apache.org/job/beam_PerformanceTests_Spark/1583/display/redirect?page=changes>
Changes: [tgroh] Add a ModelCoders class for Coder Utilities [tgroh] Use ModelCoders in LengthPrefixUnknownCoders [tgroh] Format UrnUtils [XuMingmin] [BEAM-3983] [SQL] Tables interface supports BigQuery (#4991) [kedin] [SQL] Infer Row type from input elements ------------------------------------------ [...truncated 98.15 KB...] 'apache-beam-testing:bqjob_r5369b60426b8b778_00000162bc7ef824_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r5369b60426b8b778_00000162bc7ef824_1 ... (0s) Current status: RUNNING Waiting on bqjob_r5369b60426b8b778_00000162bc7ef824_1 ... (0s) Current status: DONE 2018-04-13 00:54:22,222 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:54:39,385 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:54:41,607 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r16a7583cb12c1244_00000162bc7f437b_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r16a7583cb12c1244_00000162bc7f437b_1 ... (0s) Current status: RUNNING Waiting on bqjob_r16a7583cb12c1244_00000162bc7f437b_1 ... (0s) Current status: DONE 2018-04-13 00:54:41,608 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:55:01,051 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:55:03,180 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r93ec73a6c9f4946_00000162bc7f9802_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r93ec73a6c9f4946_00000162bc7f9802_1 ... (0s) Current status: RUNNING Waiting on bqjob_r93ec73a6c9f4946_00000162bc7f9802_1 ... (0s) Current status: DONE 2018-04-13 00:55:03,180 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:55:22,426 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:55:24,672 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r7dd877f0ebe664b6_00000162bc7febb6_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r7dd877f0ebe664b6_00000162bc7febb6_1 ... (0s) Current status: RUNNING Waiting on bqjob_r7dd877f0ebe664b6_00000162bc7febb6_1 ... (0s) Current status: DONE 2018-04-13 00:55:24,672 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:55:41,961 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:55:44,325 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r7e5c914957f4c300_00000162bc803832_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r7e5c914957f4c300_00000162bc803832_1 ... (0s) Current status: RUNNING Waiting on bqjob_r7e5c914957f4c300_00000162bc803832_1 ... (0s) Current status: DONE 2018-04-13 00:55:44,326 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:56:07,038 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:56:09,319 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r7bad0f3dad66642d_00000162bc8099e8_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r7bad0f3dad66642d_00000162bc8099e8_1 ... (0s) Current status: RUNNING Waiting on bqjob_r7bad0f3dad66642d_00000162bc8099e8_1 ... (0s) Current status: DONE 2018-04-13 00:56:09,320 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:56:34,015 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:56:36,232 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r6b4b3e0ad0562ca0_00000162bc81033a_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r6b4b3e0ad0562ca0_00000162bc81033a_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6b4b3e0ad0562ca0_00000162bc81033a_1 ... (0s) Current status: DONE 2018-04-13 00:56:36,232 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:56:52,741 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:56:55,392 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r786af02b98703ea2_00000162bc814cb3_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r786af02b98703ea2_00000162bc814cb3_1 ... (0s) Current status: RUNNING Waiting on bqjob_r786af02b98703ea2_00000162bc814cb3_1 ... (0s) Current status: DONE 2018-04-13 00:56:55,393 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:57:10,647 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:57:13,865 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r70f1c57cafd9201b_00000162bc819259_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r70f1c57cafd9201b_00000162bc819259_1 ... (0s) Current status: RUNNING Waiting on bqjob_r70f1c57cafd9201b_00000162bc819259_1 ... (1s) Current status: RUNNING Waiting on bqjob_r70f1c57cafd9201b_00000162bc819259_1 ... (1s) Current status: DONE 2018-04-13 00:57:13,866 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:57:31,417 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:57:33,741 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r6c2cdb434b1be0ff_00000162bc81e395_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r6c2cdb434b1be0ff_00000162bc81e395_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6c2cdb434b1be0ff_00000162bc81e395_1 ... (0s) Current status: DONE 2018-04-13 00:57:33,741 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:58:00,233 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:58:02,494 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r143103edc38d1953_00000162bc825431_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r143103edc38d1953_00000162bc825431_1 ... (0s) Current status: RUNNING Waiting on bqjob_r143103edc38d1953_00000162bc825431_1 ... (0s) Current status: DONE 2018-04-13 00:58:02,494 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:58:25,557 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:58:28,690 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r5b006bd06a107c12_00000162bc82b6f7_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r5b006bd06a107c12_00000162bc82b6f7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r5b006bd06a107c12_00000162bc82b6f7_1 ... (0s) Current status: DONE 2018-04-13 00:58:28,690 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:58:49,474 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:58:51,742 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r41736a6049be829_00000162bc831461_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r41736a6049be829_00000162bc831461_1 ... (0s) Current status: RUNNING Waiting on bqjob_r41736a6049be829_00000162bc831461_1 ... (0s) Current status: DONE 2018-04-13 00:58:51,742 f9036bd4 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-13 00:59:16,622 f9036bd4 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json> 2018-04-13 00:59:18,790 f9036bd4 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/f9036bd4/perfkit-bq-publaF6Ll.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r280596104bfd665c_00000162bc837e72_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r280596104bfd665c_00000162bc837e72_1 ... (0s) Current status: RUNNING Waiting on bqjob_r280596104bfd665c_00000162bc837e72_1 ... (0s) Current status: DONE Traceback (most recent call last): File "PerfKitBenchmarker/pkb.py", line 21, in <module> sys.exit(Main()) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 951, in Main return RunBenchmarks() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 878, in RunBenchmarks collector.PublishSamples() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 927, in PublishSamples publisher.PublishSamples(self.samples) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 538, in PublishSamples vm_util.IssueRetryableCommand(load_cmd) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 249, in WrappedFunction return f(*args, **kwargs) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 387, in IssueRetryableCommand 'Command returned a non-zero exit code.\n') perfkitbenchmarker.errors.CalledProcessException: Command returned a non-zero exit code. Build step 'Execute shell' marked build as failure