See <https://builds.apache.org/job/beam_PerformanceTests_Spark/1412/display/redirect?page=changes>
Changes: [echauchot] [BEAM-3681] Make S3FileSystem copy atomic for smaller than 5GB objects [echauchot] [BEAM-3681] Add a comment for the extra check of objectSize in ------------------------------------------ [...truncated 93.44 KB...] 'apache-beam-testing:bqjob_r5b4dc83be3dc75f1_00000161dda49007_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:20:07,260 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:20:27,814 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:20:29,924 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.10s, CPU:0.27s, MaxMemory:25256kb STDOUT: Upload complete. Waiting on bqjob_r21a6f85a9b361a2d_00000161dda4ea33_1 ... (0s) Current status: RUNNING Waiting on bqjob_r21a6f85a9b361a2d_00000161dda4ea33_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r21a6f85a9b361a2d_00000161dda4ea33_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:20:29,924 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:20:52,406 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:20:54,533 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.12s, CPU:0.24s, MaxMemory:25336kb STDOUT: Upload complete. Waiting on bqjob_r69aaf5b1dc2ac00b_00000161dda54a3f_1 ... (0s) Current status: RUNNING Waiting on bqjob_r69aaf5b1dc2ac00b_00000161dda54a3f_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r69aaf5b1dc2ac00b_00000161dda54a3f_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:20:54,534 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:21:11,694 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:21:13,928 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.22s, CPU:0.24s, MaxMemory:25508kb STDOUT: Upload complete. Waiting on bqjob_r4554ba69fafe47fa_00000161dda595a1_1 ... (0s) Current status: RUNNING Waiting on bqjob_r4554ba69fafe47fa_00000161dda595a1_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r4554ba69fafe47fa_00000161dda595a1_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:21:13,929 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:21:30,502 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:21:32,914 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.40s, CPU:0.25s, MaxMemory:25320kb STDOUT: Upload complete. Waiting on bqjob_r26d2d50906201224_00000161dda5df12_1 ... (0s) Current status: RUNNING Waiting on bqjob_r26d2d50906201224_00000161dda5df12_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r26d2d50906201224_00000161dda5df12_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:21:32,915 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:21:50,209 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:21:52,358 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.14s, CPU:0.24s, MaxMemory:25320kb STDOUT: Upload complete. Waiting on bqjob_r151ad39332ee2fbc_00000161dda62c0e_1 ... (0s) Current status: RUNNING Waiting on bqjob_r151ad39332ee2fbc_00000161dda62c0e_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r151ad39332ee2fbc_00000161dda62c0e_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:21:52,358 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:22:08,629 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:22:11,058 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.42s, CPU:0.27s, MaxMemory:25452kb STDOUT: Upload complete. Waiting on bqjob_r779c7347e7e7787_00000161dda6740f_1 ... (0s) Current status: RUNNING Waiting on bqjob_r779c7347e7e7787_00000161dda6740f_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r779c7347e7e7787_00000161dda6740f_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:22:11,058 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:22:37,757 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:22:39,871 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.10s, CPU:0.24s, MaxMemory:25400kb STDOUT: Upload complete. Waiting on bqjob_r7d8915d1786158e6_00000161dda6e5cc_1 ... (0s) Current status: RUNNING Waiting on bqjob_r7d8915d1786158e6_00000161dda6e5cc_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r7d8915d1786158e6_00000161dda6e5cc_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:22:39,871 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:22:58,212 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:23:00,340 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.12s, CPU:0.28s, MaxMemory:25196kb STDOUT: Upload complete. Waiting on bqjob_r37f1854007ab13a3_00000161dda735b7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r37f1854007ab13a3_00000161dda735b7_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r37f1854007ab13a3_00000161dda735b7_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:23:00,341 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:23:19,078 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:23:21,456 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.37s, CPU:0.24s, MaxMemory:25512kb STDOUT: Upload complete. Waiting on bqjob_r178c8a60d733c5a1_00000161dda7872a_1 ... (0s) Current status: RUNNING Waiting on bqjob_r178c8a60d733c5a1_00000161dda7872a_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r178c8a60d733c5a1_00000161dda7872a_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:23:21,456 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:23:37,662 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:23:39,816 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.14s, CPU:0.27s, MaxMemory:25256kb STDOUT: Upload complete. Waiting on bqjob_r1986047c62dbd5c4_00000161dda7cfcf_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1986047c62dbd5c4_00000161dda7cfcf_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r1986047c62dbd5c4_00000161dda7cfcf_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:23:39,817 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:24:02,670 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:24:04,711 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.03s, CPU:0.24s, MaxMemory:25320kb STDOUT: Upload complete. Waiting on bqjob_r201d16179c8dee19_00000161dda83188_1 ... (0s) Current status: RUNNING Waiting on bqjob_r201d16179c8dee19_00000161dda83188_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r201d16179c8dee19_00000161dda83188_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:24:04,712 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:24:32,596 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:24:34,744 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:02.14s, CPU:0.24s, MaxMemory:25552kb STDOUT: Upload complete. Waiting on bqjob_r148728c55e711a2c_00000161dda8a658_1 ... (0s) Current status: RUNNING Waiting on bqjob_r148728c55e711a2c_00000161dda8a658_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r148728c55e711a2c_00000161dda8a658_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-02-28 18:24:34,744 890b6841 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-02-28 18:24:51,686 890b6841 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json> 2018-02-28 18:24:58,729 890b6841 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/890b6841/perfkit-bq-pubpngeof.json}> ReturnCode:1, WallTime:0:07.03s, CPU:0.24s, MaxMemory:25564kb STDOUT: Upload complete. Waiting on bqjob_r177db1278a7644d4_00000161dda8f0e8_1 ... (0s) Current status: RUNNING Waiting on bqjob_r177db1278a7644d4_00000161dda8f0e8_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r177db1278a7644d4_00000161dda8f0e8_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Traceback (most recent call last): File "PerfKitBenchmarker/pkb.py", line 21, in <module> sys.exit(Main()) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 926, in Main return RunBenchmarks() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 853, in RunBenchmarks collector.PublishSamples() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py";,> line 886, in PublishSamples publisher.PublishSamples(self.samples) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py";,> line 538, in PublishSamples vm_util.IssueRetryableCommand(load_cmd) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py";,> line 249, in WrappedFunction return f(*args, **kwargs) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py";,> line 387, in IssueRetryableCommand 'Command returned a non-zero exit code.\n') perfkitbenchmarker.errors.CalledProcessException: Command returned a non-zero exit code. Build step 'Execute shell' marked build as failure
