[
https://issues.apache.org/jira/browse/BEAM-12918?focusedWorklogId=768495&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-768495
]
ASF GitHub Bot logged work on BEAM-12918:
-----------------------------------------
Author: ASF GitHub Bot
Created on: 10/May/22 13:16
Start Date: 10/May/22 13:16
Worklog Time Spent: 10m
Work Description: echauchot commented on code in PR #15679:
URL: https://github.com/apache/beam/pull/15679#discussion_r869220682
##########
.test-infra/jenkins/job_PostCommit_Java_Tpcds_Spark.groovy:
##########
@@ -0,0 +1,188 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements. See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership. The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+import CommonJobProperties as commonJobProperties
+import NoPhraseTriggeringPostCommitBuilder
+import PhraseTriggeringPostCommitBuilder
+import InfluxDBCredentialsHelper
+
+// This job runs the Tpcds benchmark suite against the Spark runner.
+NoPhraseTriggeringPostCommitBuilder.postCommitJob('beam_PostCommit_Java_Tpcds_Spark',
+ 'Spark Runner Tpcds Tests', this) {
+ description('Runs the Tpcds suite on the Spark runner.')
+
+ // Set common parameters.
+ commonJobProperties.setTopLevelMainJobProperties(delegate, 'master',
240, true, 'beam-perf')
+ InfluxDBCredentialsHelper.useCredentials(delegate)
+
+ // Gradle goals for this job.
+ steps {
+ shell('echo "*** RUN TPCDS IN BATCH MODE USING SPARK 2 RUNNER ***"')
+ gradle {
+ rootBuildScriptDir(commonJobProperties.checkoutDir)
+ tasks(':sdks:java:testing:tpcds:run')
+ commonJobProperties.setGradleSwitches(delegate)
+ switches('-Ptpcds.runner=":runners:spark:2"' +
+ ' -Ptpcds.args="' +
+ [
+ '--dataSize=1GB',
+ '--sourceType=PARQUET',
+ '--dataDirectory=gs://beam-tpcds/datasets/parquet/partitioned',
+ '--resultsDirectory=gs://beam-tpcds/results/',
+ '--tpcParallel=1',
+ '--runner=SparkRunner',
+ '--queries=3'
Review Comment:
:+1:
Issue Time Tracking
-------------------
Worklog Id: (was: 768495)
Time Spent: 7h (was: 6h 50m)
> TPC-DS: add Jenkins jobs
> ------------------------
>
> Key: BEAM-12918
> URL: https://issues.apache.org/jira/browse/BEAM-12918
> Project: Beam
> Issue Type: Test
> Components: testing-tpcds
> Reporter: Alexey Romanenko
> Assignee: Alexey Romanenko
> Priority: P2
> Time Spent: 7h
> Remaining Estimate: 0h
>
--
This message was sent by Atlassian Jira
(v8.20.7#820007)
