See <https://builds.apache.org/job/beam_PerformanceTests_Spark/1615/display/redirect?page=changes>
Changes: [djiofack007] modified result set to be forward only and read only [djiofack007] made fetchsize configurable [djiofack007] modified result set to be forward only and read only [djiofack007] made fetchsize configurable [djiofack007] added documentation [bbassingthwaite] Adding a go datastore io reader [kirpichov] Reshuffle filenames before returning them from WriteFilesResult [djiofack007] modified docs [bbassingthwaite] Add apache header and fix imports / gofmt [bbassingthwaite] - Add package doc [swegner] Disable Jacoco plugin unless explicitly requested. [djiofack007] fixed checkstyle [thw] [BEAM-2887, BEAM-3886] Have the Python SDK use the state api service [swegner] Add parameter overrides to run examples with personal GCS resources. [github] Minor style updates for consistency. ------------------------------------------ [...truncated 90.61 KB...] 'apache-beam-testing:bqjob_r526ea41c45a8661c_00000162e5a58d7c_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r526ea41c45a8661c_00000162e5a58d7c_1 ... (0s) Current status: RUNNING Waiting on bqjob_r526ea41c45a8661c_00000162e5a58d7c_1 ... (0s) Current status: DONE 2018-04-21 00:40:56,868 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:41:19,272 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:41:21,465 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r17ceec1293e87da7_00000162e5a5ee01_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r17ceec1293e87da7_00000162e5a5ee01_1 ... (0s) Current status: RUNNING Waiting on bqjob_r17ceec1293e87da7_00000162e5a5ee01_1 ... (0s) Current status: DONE 2018-04-21 00:41:21,466 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:41:45,163 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:41:48,333 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r6318391394da2eca_00000162e5a656e7_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r6318391394da2eca_00000162e5a656e7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6318391394da2eca_00000162e5a656e7_1 ... (0s) Current status: DONE 2018-04-21 00:41:48,334 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:42:13,125 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:42:15,662 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r3ad66434d530a625_00000162e5a6c15c_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r3ad66434d530a625_00000162e5a6c15c_1 ... (0s) Current status: RUNNING Waiting on bqjob_r3ad66434d530a625_00000162e5a6c15c_1 ... (0s) Current status: DONE 2018-04-21 00:42:15,662 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:42:43,422 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:42:46,630 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r5638fd32237f9f01_00000162e5a7368f_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r5638fd32237f9f01_00000162e5a7368f_1 ... (0s) Current status: RUNNING Waiting on bqjob_r5638fd32237f9f01_00000162e5a7368f_1 ... (1s) Current status: RUNNING Waiting on bqjob_r5638fd32237f9f01_00000162e5a7368f_1 ... (1s) Current status: DONE 2018-04-21 00:42:46,631 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:43:02,468 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:43:04,857 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r3323362a26cda661_00000162e5a7810b_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r3323362a26cda661_00000162e5a7810b_1 ... (0s) Current status: RUNNING Waiting on bqjob_r3323362a26cda661_00000162e5a7810b_1 ... (0s) Current status: DONE 2018-04-21 00:43:04,857 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:43:28,136 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:43:30,557 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r38e0731db04dc8f7_00000162e5a7e5ce_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r38e0731db04dc8f7_00000162e5a7e5ce_1 ... (0s) Current status: RUNNING Waiting on bqjob_r38e0731db04dc8f7_00000162e5a7e5ce_1 ... (0s) Current status: DONE 2018-04-21 00:43:30,558 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:43:59,250 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:44:01,538 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r6c1bf65baab26046_00000162e5a85ec0_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r6c1bf65baab26046_00000162e5a85ec0_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6c1bf65baab26046_00000162e5a85ec0_1 ... (0s) Current status: DONE 2018-04-21 00:44:01,538 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:44:25,016 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:44:27,326 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r51565bafcd6ac046_00000162e5a8c41a_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r51565bafcd6ac046_00000162e5a8c41a_1 ... (0s) Current status: RUNNING Waiting on bqjob_r51565bafcd6ac046_00000162e5a8c41a_1 ... (0s) Current status: DONE 2018-04-21 00:44:27,327 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:44:50,092 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:44:52,756 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r58d1073e77113060_00000162e5a92625_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r58d1073e77113060_00000162e5a92625_1 ... (0s) Current status: RUNNING Waiting on bqjob_r58d1073e77113060_00000162e5a92625_1 ... (0s) Current status: DONE 2018-04-21 00:44:52,756 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:45:16,020 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:45:19,137 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_rf63136aa228e107_00000162e5a98e0c_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_rf63136aa228e107_00000162e5a98e0c_1 ... (0s) Current status: RUNNING Waiting on bqjob_rf63136aa228e107_00000162e5a98e0c_1 ... (0s) Current status: DONE 2018-04-21 00:45:19,137 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:45:35,692 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:45:38,373 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r14e6ced21a98af1_00000162e5a9d8d1_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r14e6ced21a98af1_00000162e5a9d8d1_1 ... (0s) Current status: RUNNING Waiting on bqjob_r14e6ced21a98af1_00000162e5a9d8d1_1 ... (0s) Current status: DONE 2018-04-21 00:45:38,373 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:46:02,459 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:46:04,780 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r44387aadbe306a6e_00000162e5aa40bd_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r44387aadbe306a6e_00000162e5aa40bd_1 ... (0s) Current status: RUNNING Waiting on bqjob_r44387aadbe306a6e_00000162e5aa40bd_1 ... (0s) Current status: DONE 2018-04-21 00:46:04,781 8f36af82 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-21 00:46:34,424 8f36af82 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json> 2018-04-21 00:46:36,917 8f36af82 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/8f36af82/perfkit-bq-pubqaLSZG.json}> ReturnCode:1 STDOUT: BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r62138494b55a1b96_00000162e5aabe3c_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Upload complete.Waiting on bqjob_r62138494b55a1b96_00000162e5aabe3c_1 ... (0s) Current status: RUNNING Waiting on bqjob_r62138494b55a1b96_00000162e5aabe3c_1 ... (0s) Current status: DONE Traceback (most recent call last): File "PerfKitBenchmarker/pkb.py", line 21, in <module> sys.exit(Main()) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 977, in Main return RunBenchmarks() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py",> line 904, in RunBenchmarks collector.PublishSamples() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 927, in PublishSamples publisher.PublishSamples(self.samples) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py",> line 538, in PublishSamples vm_util.IssueRetryableCommand(load_cmd) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 249, in WrappedFunction return f(*args, **kwargs) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py",> line 387, in IssueRetryableCommand 'Command returned a non-zero exit code.\n') perfkitbenchmarker.errors.CalledProcessException: Command returned a non-zero exit code. Build step 'Execute shell' marked build as failure