See <https://builds.apache.org/job/beam_PerformanceTests_Spark/1602/display/redirect?page=changes>
Changes: [swegner] Exclude UsesImpluse from Apex postcommit tests [swegner] Exclude UsesImpluse from Spark postcommit tests [swegner] Increase Dataflow max parallelism. [axelmagn] Rename EnvironmentManager to EnvironmentFactory [alan.myrvold] Fix faling spark quickstart [swegner] fixup! Improve comment ------------------------------------------ [...truncated 96.04 KB...] 'apache-beam-testing:bqjob_raec8eb7ffad53e1_00000162d4edfc29_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:46:31,574 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:46:55,963 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:46:58,445 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r7d3125299a58db1a_00000162d4ee6609_1 ... (0s) Current status: RUNNING Waiting on bqjob_r7d3125299a58db1a_00000162d4ee6609_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r7d3125299a58db1a_00000162d4ee6609_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:46:58,446 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:47:27,805 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:47:30,426 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r5bf54b67d4910767_00000162d4eee2ee_1 ... (0s) Current status: RUNNING Waiting on bqjob_r5bf54b67d4910767_00000162d4eee2ee_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r5bf54b67d4910767_00000162d4eee2ee_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:47:30,427 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:47:49,045 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:47:51,614 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r3ac0b03b226d7acc_00000162d4ef35aa_1 ... (0s) Current status: RUNNING Waiting on bqjob_r3ac0b03b226d7acc_00000162d4ef35aa_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r3ac0b03b226d7acc_00000162d4ef35aa_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:47:51,614 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:48:09,951 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:48:12,814 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r44003e77aa753215_00000162d4ef878f_1 ... (0s) Current status: RUNNING Waiting on bqjob_r44003e77aa753215_00000162d4ef878f_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r44003e77aa753215_00000162d4ef878f_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:48:12,814 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:48:38,113 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:48:40,767 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_rab4eaf224107cb0_00000162d4eff532_1 ... (0s) Current status: RUNNING Waiting on bqjob_rab4eaf224107cb0_00000162d4eff532_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_rab4eaf224107cb0_00000162d4eff532_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:48:40,767 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:49:01,893 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:49:04,362 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r5ae2621c1563cf90_00000162d4f05241_1 ... (0s) Current status: RUNNING Waiting on bqjob_r5ae2621c1563cf90_00000162d4f05241_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r5ae2621c1563cf90_00000162d4f05241_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:49:04,362 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:49:23,388 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:49:26,043 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r6efdd4455aa6d858_00000162d4f0a5fb_1 ... (0s) Current status: RUNNING Waiting on bqjob_r6efdd4455aa6d858_00000162d4f0a5fb_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r6efdd4455aa6d858_00000162d4f0a5fb_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:49:26,044 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:49:52,540 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:49:55,002 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r1e94145302b17a38_00000162d4f117dc_1 ... (0s) Current status: RUNNING Waiting on bqjob_r1e94145302b17a38_00000162d4f117dc_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r1e94145302b17a38_00000162d4f117dc_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:49:55,003 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:50:22,205 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:50:24,848 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r5873bd01f6022dc8_00000162d4f18bc5_1 ... (0s) Current status: RUNNING Waiting on bqjob_r5873bd01f6022dc8_00000162d4f18bc5_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r5873bd01f6022dc8_00000162d4f18bc5_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:50:24,848 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:50:41,045 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:50:43,547 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r71db11213adf8e9_00000162d4f1d577_1 ... (0s) Current status: RUNNING Waiting on bqjob_r71db11213adf8e9_00000162d4f1d577_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r71db11213adf8e9_00000162d4f1d577_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:50:43,547 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:51:11,830 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:51:14,298 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_rc4754a13c1240d6_00000162d4f24da7_1 ... (0s) Current status: RUNNING Waiting on bqjob_rc4754a13c1240d6_00000162d4f24da7_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_rc4754a13c1240d6_00000162d4f24da7_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:51:14,298 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:51:40,389 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:51:42,985 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r7f8506000e8fe1b6_00000162d4f2bd29_1 ... (0s) Current status: RUNNING Waiting on bqjob_r7f8506000e8fe1b6_00000162d4f2bd29_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r7f8506000e8fe1b6_00000162d4f2bd29_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) 2018-04-17 18:51:42,986 25e77575 MainThread INFO Retrying exception running IssueRetryableCommand: Command returned a non-zero exit code. 2018-04-17 18:52:00,557 25e77575 MainThread INFO Running: bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json> 2018-04-17 18:52:02,821 25e77575 MainThread INFO Ran: {bq load --autodetect --source_format=NEWLINE_DELIMITED_JSON beam_performance.pkb_results <https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/runs/25e77575/perfkit-bq-pubzhGJte.json}> ReturnCode:1 STDOUT: Upload complete. Waiting on bqjob_r3f5f3bada8691da9_00000162d4f30be7_1 ... (0s) Current status: RUNNING Waiting on bqjob_r3f5f3bada8691da9_00000162d4f30be7_1 ... (0s) Current status: DONE BigQuery error in load operation: Error processing job 'apache-beam-testing:bqjob_r3f5f3bada8691da9_00000162d4f30be7_1': Invalid schema update. Field timestamp has changed type from TIMESTAMP to FLOAT STDERR: /usr/lib/google-cloud-sdk/platform/bq/third_party/oauth2client/contrib/gce.py:73: UserWarning: You have requested explicit scopes to be used with a GCE service account. Using this argument will have no effect on the actual scopes for tokens requested. These scopes are set at VM instance creation time and can't be overridden in the request. warnings.warn(_SCOPES_WARNING) Traceback (most recent call last): File "PerfKitBenchmarker/pkb.py", line 21, in <module> sys.exit(Main()) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 975, in Main return RunBenchmarks() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/pkb.py";,> line 902, in RunBenchmarks collector.PublishSamples() File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py";,> line 927, in PublishSamples publisher.PublishSamples(self.samples) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/publisher.py";,> line 538, in PublishSamples vm_util.IssueRetryableCommand(load_cmd) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py";,> line 249, in WrappedFunction return f(*args, **kwargs) File "<https://builds.apache.org/job/beam_PerformanceTests_Spark/ws/PerfKitBenchmarker/perfkitbenchmarker/vm_util.py";,> line 387, in IssueRetryableCommand 'Command returned a non-zero exit code.\n') perfkitbenchmarker.errors.CalledProcessException: Command returned a non-zero exit code. Build step 'Execute shell' marked build as failure
