[GitHub] [spark] AmplabJenkins removed a comment on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader
AmplabJenkins removed a comment on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader URL: https://github.com/apache/spark/pull/28022#issuecomment-603948326 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25075/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader
AmplabJenkins removed a comment on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader URL: https://github.com/apache/spark/pull/28022#issuecomment-603948314 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader
AmplabJenkins commented on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader URL: https://github.com/apache/spark/pull/28022#issuecomment-603948326 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25075/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader
AmplabJenkins commented on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader URL: https://github.com/apache/spark/pull/28022#issuecomment-603948314 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
AmplabJenkins removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603959177 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120345/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
jiangxb1987 commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-603980359 It's fine if you want to focus on the k8s behaviour here, I can submit another PR to fix the Standalone backend after this is merged. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
jiangxb1987 commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-603980696 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates to/from Julian calendar in write/read for ORC datasource
AmplabJenkins removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates to/from Julian calendar in write/read for ORC datasource URL: https://github.com/apache/spark/pull/28016#issuecomment-603980152 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25076/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
AmplabJenkins commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-603980191 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25077/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28018: [SPARK-30758][SQL][TESTS][FOLLOWUP] Fix bug tests imported bracketed comments
AmplabJenkins removed a comment on issue #28018: [SPARK-30758][SQL][TESTS][FOLLOWUP] Fix bug tests imported bracketed comments URL: https://github.com/apache/spark/pull/28018#issuecomment-603979738 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates to/from Julian calendar in write/read for ORC datasource
AmplabJenkins commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates to/from Julian calendar in write/read for ORC datasource URL: https://github.com/apache/spark/pull/28016#issuecomment-603980152 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25076/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28018: [SPARK-30758][SQL][TESTS][FOLLOWUP] Fix bug tests imported bracketed comments
AmplabJenkins removed a comment on issue #28018: [SPARK-30758][SQL][TESTS][FOLLOWUP] Fix bug tests imported bracketed comments URL: https://github.com/apache/spark/pull/28018#issuecomment-603979746 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120341/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
AmplabJenkins commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-603980184 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates to/from Julian calendar in write/read for ORC datasource
AmplabJenkins removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates to/from Julian calendar in write/read for ORC datasource URL: https://github.com/apache/spark/pull/28016#issuecomment-603980109 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] tgravescs commented on a change in pull request #27943: [SPARK-31179] Fast fail the connection while last connection failed in fast fail time window
tgravescs commented on a change in pull request #27943: [SPARK-31179] Fast fail the connection while last connection failed in fast fail time window URL: https://github.com/apache/spark/pull/27943#discussion_r398060555 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClientFactory.java ## @@ -86,6 +87,7 @@ private EventLoopGroup workerGroup; private final PooledByteBufAllocator pooledAllocator; private final NettyMemoryMetrics metrics; + private final double fastFailTimeWindow; Review comment: we don't really need this as double, just case to an int. ioRetryWaitTimeMs returns int This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28020: [MINOR][BUILD] Pin the avro version in SBT
AmplabJenkins removed a comment on issue #28020: [MINOR][BUILD] Pin the avro version in SBT URL: https://github.com/apache/spark/pull/28020#issuecomment-604019020 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120361/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
MaxGekk commented on a change in pull request #28001: [SPARK-31237][SQL][TESTS]
Replace 3-letter time zones by zone offsets
URL: https://github.com/apache/spark/pull/28001#discussion_r398090030
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchemaSuite.scala
##
@@ -22,13 +22,15 @@ import java.util.Locale
import org.apache.spark.SparkFunSuite
import org.apache.spark.sql.catalyst.plans.SQLHelper
+import org.apache.spark.sql.catalyst.util.DateTimeTestUtils.UTC
import org.apache.spark.sql.internal.SQLConf
import org.apache.spark.sql.types._
class CSVInferSchemaSuite extends SparkFunSuite with SQLHelper {
test("String fields types are inferred correctly from null types") {
-val options = new CSVOptions(Map("timestampFormat" -> "-MM-dd
HH:mm:ss"), false, "GMT")
Review comment:
I replaced `GMT` by `UTC`, and `UTC.getId` by `UTC`
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28020: [MINOR][BUILD] Pin the avro version in SBT
AmplabJenkins removed a comment on issue #28020: [MINOR][BUILD] Pin the avro version in SBT URL: https://github.com/apache/spark/pull/28020#issuecomment-604019010 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28020: [MINOR][BUILD] Pin the avro version in SBT
AmplabJenkins commented on issue #28020: [MINOR][BUILD] Pin the avro version in SBT URL: https://github.com/apache/spark/pull/28020#issuecomment-604019010 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] viirya commented on a change in pull request #28015: [SPARK-31244][K8S][TEST] Use Minio instead of Ceph in K8S DepsTestsSuite
viirya commented on a change in pull request #28015: [SPARK-31244][K8S][TEST]
Use Minio instead of Ceph in K8S DepsTestsSuite
URL: https://github.com/apache/spark/pull/28015#discussion_r398089426
##
File path:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/DepsTestsSuite.scala
##
@@ -59,13 +58,14 @@ private[spark] trait DepsTestsSuite { k8sSuite:
KubernetesSuite =>
).asJava
new ContainerBuilder()
- .withImage("ceph/daemon:v4.0.3-stable-4.0-nautilus-centos-7-x86_64")
+ .withImage("minio/minio:latest")
.withImagePullPolicy("Always")
.withName(cName)
+ .withArgs("server", "/data")
Review comment:
Is this an argument passed to minio server?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
SparkQA removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-603979607 **[Test build #120368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120368/testReport)** for PR 27994 at commit [`54a7b9f`](https://github.com/apache/spark/commit/54a7b9ff30b00152983a19e81965338fd993989e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
SparkQA commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-604025982 **[Test build #120368 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120368/testReport)** for PR 27994 at commit [`54a7b9f`](https://github.com/apache/spark/commit/54a7b9ff30b00152983a19e81965338fd993989e). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28009: [SPARK-31235][YARN] Separates different categories of applications
SparkQA commented on issue #28009: [SPARK-31235][YARN] Separates different categories of applications URL: https://github.com/apache/spark/pull/28009#issuecomment-604054952 **[Test build #120371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120371/testReport)** for PR 28009 at commit [`ef705dc`](https://github.com/apache/spark/commit/ef705dcc937c07c60dc5ba29a59913b81e05ce23). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] tgravescs commented on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader
tgravescs commented on issue #28022: [SPARK-31253][SQL] add metrics to shuffle reader URL: https://github.com/apache/spark/pull/28022#issuecomment-604054877 is this only for adaptive scheduler mode? I've seen a bunch of go by that were specific to it but didn't mention it in the title or description, so can we tag or at least mention that in the description. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #28015: [SPARK-31244][K8S][TEST] Use Minio instead of Ceph in K8S DepsTestsSuite
dongjoon-hyun commented on issue #28015: [SPARK-31244][K8S][TEST] Use Minio instead of Ceph in K8S DepsTestsSuite URL: https://github.com/apache/spark/pull/28015#issuecomment-603956657 Hi, @HyukjinKwon and @viirya . Could you review this please when you have some time? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
AmplabJenkins removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-603980184 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
AmplabJenkins removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-603980191 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25077/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #28016: [SPARK-31238][SQL] Rebase dates to/from Julian calendar in write/read for ORC datasource
MaxGekk commented on issue #28016: [SPARK-31238][SQL] Rebase dates to/from Julian calendar in write/read for ORC datasource URL: https://github.com/apache/spark/pull/28016#issuecomment-604002887 @cloud-fan @dongjoon-hyun @bersprockets Please, review this PR. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
AmplabJenkins removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets URL: https://github.com/apache/spark/pull/28001#issuecomment-604021448 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28021: [SPARK-31161][WEBUI][3.0] Refactor the on-click timeline action in streagming-page.js
SparkQA removed a comment on issue #28021: [SPARK-31161][WEBUI][3.0] Refactor the on-click timeline action in streagming-page.js URL: https://github.com/apache/spark/pull/28021#issuecomment-603908886 **[Test build #120362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120362/testReport)** for PR 28021 at commit [`4062cf4`](https://github.com/apache/spark/commit/4062cf419fcbb92e8b46300c4e0a15f711b9fa8c). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
AmplabJenkins removed a comment on issue #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets URL: https://github.com/apache/spark/pull/28001#issuecomment-604021455 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25079/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28017: [MINOR][DOCS] Fix some links for python api doc
AmplabJenkins removed a comment on issue #28017: [MINOR][DOCS] Fix some links for python api doc URL: https://github.com/apache/spark/pull/28017#issuecomment-603943373 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120359/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28017: [MINOR][DOCS] Fix some links for python api doc
AmplabJenkins removed a comment on issue #28017: [MINOR][DOCS] Fix some links for python api doc URL: https://github.com/apache/spark/pull/28017#issuecomment-603943362 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
AmplabJenkins commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603959162 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
AmplabJenkins removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603959162 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
AmplabJenkins commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603959177 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120345/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
SparkQA commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603958162 **[Test build #120345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120345/testReport)** for PR 28016 at commit [`47d8588`](https://github.com/apache/spark/commit/47d8588510d453040d316abe560061f2e4242ea6). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
SparkQA removed a comment on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603795274 **[Test build #120345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120345/testReport)** for PR 28016 at commit [`47d8588`](https://github.com/apache/spark/commit/47d8588510d453040d316abe560061f2e4242ea6). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603977426 I think the failure doesn't related to my changes: https://github.com/apache/spark/pull/28016#issuecomment-603958162 . I will update the PR desciption shortly, so the PR will be ready for review. /cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603976469 > So, does this happen only in vectorized reader @dongjoon-hyun Correct, the regular reader uses `DateTimeUtils.fromJavaDate`: https://github.com/apache/spark/blob/300ec1a74cb14867c22e616e657566d510426331/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcDeserializer.scala#L111 And the PRs #27807 and #27980 introduced date rebasing in `fromJavaDate()`. Also, I fixed ORC writer, and added a round trip test for both vectorized and non-vectorized readers. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] huaxingao commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
huaxingao commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-603976529 retest this please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27969: [SPARK-31170][SQL][test-hive1.2] Spark SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir
AmplabJenkins commented on issue #27969: [SPARK-31170][SQL][test-hive1.2] Spark SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir URL: https://github.com/apache/spark/pull/27969#issuecomment-603984808 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27969: [SPARK-31170][SQL][test-hive1.2] Spark SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir
AmplabJenkins removed a comment on issue #27969: [SPARK-31170][SQL][test-hive1.2] Spark SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir URL: https://github.com/apache/spark/pull/27969#issuecomment-603984808 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27969: [SPARK-31170][SQL][test-hive1.2] Spark SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir
AmplabJenkins commented on issue #27969: [SPARK-31170][SQL][test-hive1.2] Spark SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir URL: https://github.com/apache/spark/pull/27969#issuecomment-603984825 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120351/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
SparkQA commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604004318 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/25078/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
SparkQA removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-603823133 **[Test build #120354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120354/testReport)** for PR 27019 at commit [`d303b2f`](https://github.com/apache/spark/commit/d303b2fe2bc1ddc0b1e98c2028fb92322a0bb976). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-604020363 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-604020363 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins removed a comment on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-604020371 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120354/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
AmplabJenkins commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-604020371 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120354/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #28015: [SPARK-31244][K8S][TEST] Use Minio instead of Ceph in K8S DepsTestsSuite
dongjoon-hyun closed pull request #28015: [SPARK-31244][K8S][TEST] Use Minio instead of Ceph in K8S DepsTestsSuite URL: https://github.com/apache/spark/pull/28015 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #28015: [SPARK-31244][K8S][TEST] Use Minio instead of Ceph in K8S DepsTestsSuite
dongjoon-hyun commented on issue #28015: [SPARK-31244][K8S][TEST] Use Minio instead of Ceph in K8S DepsTestsSuite URL: https://github.com/apache/spark/pull/28015#issuecomment-604044491 Thank you so much, @viirya . Merged to master/3.0. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-604044578 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on a change in pull request #28015: [SPARK-31244][K8S][TEST] Use Minio instead of Ceph in K8S DepsTestsSuite
dongjoon-hyun commented on a change in pull request #28015:
[SPARK-31244][K8S][TEST] Use Minio instead of Ceph in K8S DepsTestsSuite
URL: https://github.com/apache/spark/pull/28015#discussion_r398119557
##
File path:
resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/DepsTestsSuite.scala
##
@@ -59,13 +58,14 @@ private[spark] trait DepsTestsSuite { k8sSuite:
KubernetesSuite =>
).asJava
new ContainerBuilder()
- .withImage("ceph/daemon:v4.0.3-stable-4.0-nautilus-centos-7-x86_64")
+ .withImage("minio/minio:latest")
.withImagePullPolicy("Always")
.withName(cName)
+ .withArgs("server", "/data")
Review comment:
Yes, right! `minio` docker image needs these argument.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-604044578 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-604044592 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120360/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
SparkQA removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603862942 **[Test build #120360 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120360/testReport)** for PR 27991 at commit [`1bf68f0`](https://github.com/apache/spark/commit/1bf68f03acce3cb46380ecbadd66693cbff39195). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-604044592 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120360/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] gengliangwang commented on issue #28021: [SPARK-31161][WEBUI][3.0] Refactor the on-click timeline action in streagming-page.js
gengliangwang commented on issue #28021: [SPARK-31161][WEBUI][3.0] Refactor the on-click timeline action in streagming-page.js URL: https://github.com/apache/spark/pull/28021#issuecomment-604052966 Thanks, merging to 3.0 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] tgravescs commented on a change in pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
tgravescs commented on a change in pull request #28009: [SPARK-31235][YARN]
Separates different categories of applications
URL: https://github.com/apache/spark/pull/28009#discussion_r398128487
##
File path: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
##
@@ -332,6 +332,36 @@ class SparkSubmitSuite
sys.props("SPARK_SUBMIT") should be ("true")
}
+ test("handles spark.yarn.applicationType on yarn mode") {
+val clArgs = Seq(
+ "--deploy-mode", "client",
+ "--master", "yarn",
+ "--executor-memory", "5g",
+ "--executor-cores", "5",
+ "--class", "org.SomeClass",
+ "--jars", "one.jar,two.jar,three.jar",
+ "--driver-memory", "4g",
+ "--queue", "thequeue",
+ "--files", "file1.txt,file2.txt",
+ "--archives", "archive1.txt,archive2.txt",
+ "--num-executors", "6",
+ "--name", "trill",
+ "--conf", "spark.yarn.applicationType=SPARK-SQL",
Review comment:
I agree, what happens if empty string, any characters not allowed in it? Add
tests and documentation to cover.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
AmplabJenkins removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604051616 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120369/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] tgravescs commented on issue #28009: [SPARK-31235][YARN] Separates different categories of applications
tgravescs commented on issue #28009: [SPARK-31235][YARN] Separates different categories of applications URL: https://github.com/apache/spark/pull/28009#issuecomment-604052317 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] tgravescs commented on a change in pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
tgravescs commented on a change in pull request #28009: [SPARK-31235][YARN]
Separates different categories of applications
URL: https://github.com/apache/spark/pull/28009#discussion_r398127404
##
File path:
resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/config.scala
##
@@ -91,6 +91,12 @@ package object config {
.booleanConf
.createWithDefault(false)
+ private[spark] val APPLICATION_TYPE =
ConfigBuilder("spark.yarn.applicationType")
+.doc("Type of this application," +
Review comment:
it allows user to specify a more specific type for the application , such
as
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28009: [SPARK-31235][YARN] Separates different categories of applications
AmplabJenkins removed a comment on issue #28009: [SPARK-31235][YARN] Separates different categories of applications URL: https://github.com/apache/spark/pull/28009#issuecomment-603594661 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28005#issuecomment-603962426 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test
AmplabJenkins removed a comment on issue #28005: [SPARK-31231][BUILD][FOLLOW-UP] Set the upper bound (before 46.1.0) for setuptools in pip package test URL: https://github.com/apache/spark/pull/28005#issuecomment-603962440 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120358/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
SparkQA commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-603969481 **[Test build #120364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120364/testReport)** for PR 27994 at commit [`54a7b9f`](https://github.com/apache/spark/commit/54a7b9ff30b00152983a19e81965338fd993989e). * This patch **fails PySpark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
SparkQA removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-603927887 **[Test build #120364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120364/testReport)** for PR 27994 at commit [`54a7b9f`](https://github.com/apache/spark/commit/54a7b9ff30b00152983a19e81965338fd993989e). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader
MaxGekk commented on issue #28016: [WIP][SPARK-31238][SQL] Rebase dates in ORC Vectorized Reader URL: https://github.com/apache/spark/pull/28016#issuecomment-603977578 jenkins, retest this, please This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27969: [SPARK-31170][SQL][test-hive1.2] Spark SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir
AmplabJenkins removed a comment on issue #27969: [SPARK-31170][SQL][test-hive1.2] Spark SQL Cli should respect hive-site.xml and spark.sql.warehouse.dir URL: https://github.com/apache/spark/pull/27969#issuecomment-603984825 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120351/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28003: [SPARK-31234][SQL] ResetCommand should reset config to sc.conf only
AmplabJenkins commented on issue #28003: [SPARK-31234][SQL] ResetCommand should reset config to sc.conf only URL: https://github.com/apache/spark/pull/28003#issuecomment-604000276 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28003: [SPARK-31234][SQL] ResetCommand should reset config to sc.conf only
AmplabJenkins commented on issue #28003: [SPARK-31234][SQL] ResetCommand should reset config to sc.conf only URL: https://github.com/apache/spark/pull/28003#issuecomment-604000290 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120349/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28003: [SPARK-31234][SQL] ResetCommand should reset config to sc.conf only
AmplabJenkins removed a comment on issue #28003: [SPARK-31234][SQL] ResetCommand should reset config to sc.conf only URL: https://github.com/apache/spark/pull/28003#issuecomment-604000276 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603999812 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #28003: [SPARK-31234][SQL] ResetCommand should reset config to sc.conf only
AmplabJenkins removed a comment on issue #28003: [SPARK-31234][SQL] ResetCommand should reset config to sc.conf only URL: https://github.com/apache/spark/pull/28003#issuecomment-604000290 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120349/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins removed a comment on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603999817 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120356/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec
SparkQA commented on issue #27019: [SPARK-30027][SQL] Support codegen for aggregate filters in HashAggregateExec URL: https://github.com/apache/spark/pull/27019#issuecomment-604018920 **[Test build #120354 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120354/testReport)** for PR 27019 at commit [`d303b2f`](https://github.com/apache/spark/commit/d303b2fe2bc1ddc0b1e98c2028fb92322a0bb976). * This patch passes all tests. * This patch merges cleanly. * This patch adds the following public classes _(experimental)_: * `trait GeneratePredicateHelper extends PredicateHelper ` * `case class FilterExec(condition: Expression, child: SparkPlan)` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #28020: [MINOR][BUILD] Pin the avro version in SBT
AmplabJenkins commented on issue #28020: [MINOR][BUILD] Pin the avro version in SBT URL: https://github.com/apache/spark/pull/28020#issuecomment-604019020 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120361/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
AmplabJenkins commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-604026948 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
AmplabJenkins commented on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-604026954 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120368/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
AmplabJenkins removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-604026948 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data
AmplabJenkins removed a comment on issue #27994: [SPARK-31223][ML] Set seed in np.random to regenerate test data URL: https://github.com/apache/spark/pull/27994#issuecomment-604026954 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120368/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun closed pull request #27902: [SPARK-31147][SQL] Forbid CHAR type in non-Hive-Serde tables
dongjoon-hyun closed pull request #27902: [SPARK-31147][SQL] Forbid CHAR type in non-Hive-Serde tables URL: https://github.com/apache/spark/pull/27902 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #28018: [SPARK-30758][SQL][TESTS][FOLLOWUP] Fix bug tests imported bracketed comments
SparkQA commented on issue #28018: [SPARK-30758][SQL][TESTS][FOLLOWUP] Fix bug tests imported bracketed comments URL: https://github.com/apache/spark/pull/28018#issuecomment-603972438 **[Test build #120341 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120341/testReport)** for PR 28018 at commit [`cec4670`](https://github.com/apache/spark/commit/cec4670d5f6c8c3fe1c06d71b6545d4729daea7d). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #28018: [SPARK-30758][SQL][TESTS][FOLLOWUP] Fix bug tests imported bracketed comments
SparkQA removed a comment on issue #28018: [SPARK-30758][SQL][TESTS][FOLLOWUP] Fix bug tests imported bracketed comments URL: https://github.com/apache/spark/pull/28018#issuecomment-603779199 **[Test build #120341 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120341/testReport)** for PR 28018 at commit [`cec4670`](https://github.com/apache/spark/commit/cec4670d5f6c8c3fe1c06d71b6545d4729daea7d). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603999812 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type
AmplabJenkins commented on issue #27991: [SPARK-31227][SQL] Non-nullable null type in complex types should not coerce to nullable type URL: https://github.com/apache/spark/pull/27991#issuecomment-603999817 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120356/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] jiangxb1987 commented on a change in pull request #28009: [SPARK-31235][YARN] Separates different categories of applications
jiangxb1987 commented on a change in pull request #28009: [SPARK-31235][YARN]
Separates different categories of applications
URL: https://github.com/apache/spark/pull/28009#discussion_r398083703
##
File path: core/src/test/scala/org/apache/spark/deploy/SparkSubmitSuite.scala
##
@@ -332,6 +332,36 @@ class SparkSubmitSuite
sys.props("SPARK_SUBMIT") should be ("true")
}
+ test("handles spark.yarn.applicationType on yarn mode") {
+val clArgs = Seq(
+ "--deploy-mode", "client",
+ "--master", "yarn",
+ "--executor-memory", "5g",
+ "--executor-cores", "5",
+ "--class", "org.SomeClass",
+ "--jars", "one.jar,two.jar,three.jar",
+ "--driver-memory", "4g",
+ "--queue", "thequeue",
+ "--files", "file1.txt,file2.txt",
+ "--archives", "archive1.txt,archive2.txt",
+ "--num-executors", "6",
+ "--name", "trill",
+ "--conf", "spark.yarn.applicationType=SPARK-SQL",
Review comment:
should you also test the case when `spark.yarn.applicationType` is not set?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
AmplabJenkins commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604014608 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
SparkQA commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604014579 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/25078/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
AmplabJenkins commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604014618 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25078/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
AmplabJenkins removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604014608 Merged build finished. Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
AmplabJenkins removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604014618 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/25078/ Test PASSed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] MaxGekk commented on a change in pull request #28001: [SPARK-31237][SQL][TESTS] Replace 3-letter time zones by zone offsets
MaxGekk commented on a change in pull request #28001: [SPARK-31237][SQL][TESTS]
Replace 3-letter time zones by zone offsets
URL: https://github.com/apache/spark/pull/28001#discussion_r398095971
##
File path:
sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeTestUtils.scala
##
@@ -22,20 +22,31 @@ import java.util.TimeZone
import java.util.concurrent.TimeUnit
import org.apache.spark.sql.catalyst.util.DateTimeConstants._
+import org.apache.spark.sql.catalyst.util.DateTimeUtils.getZoneId
/**
* Helper functions for testing date and time functionality.
*/
object DateTimeTestUtils {
+ val CEST = getZoneId("+02:00")
+ val CET = getZoneId("+01:00")
+ val JST = getZoneId("+09:00")
Review comment:
I haven't found alternative definition of `JST` in
https://www.timeanddate.com/time/zones/
The problem of `JST` is it maps to `Asia/Tokyo`, see
https://github.com/frohoff/jdk8u-jdk/blob/master/src/share/classes/java/time/ZoneId.java#L241
And `Asia/Tokyo` had daylight saving time transitions before 8-Sep-1951.
Since `JST` is `+09:00` by definition, the changed line reflects this.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
AmplabJenkins commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604051616 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/120369/ Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
AmplabJenkins removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604051604 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
AmplabJenkins commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604051604 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
SparkQA commented on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-604051121 **[Test build #120369 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120369/testReport)** for PR 27695 at commit [`2b3ad5b`](https://github.com/apache/spark/commit/2b3ad5bff2db4aa1f0c49503c3bffbb230cb). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers
SparkQA removed a comment on issue #27695: [SPARK-30949][K8S][CORE] decouple requests and parallelism on kubernetes drivers URL: https://github.com/apache/spark/pull/27695#issuecomment-603983643 **[Test build #120369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/120369/testReport)** for PR 27695 at commit [`2b3ad5b`](https://github.com/apache/spark/commit/2b3ad5bff2db4aa1f0c49503c3bffbb230cb). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] dongjoon-hyun commented on issue #27902: [SPARK-31147][SQL] Forbid CHAR type in non-Hive-Serde tables
dongjoon-hyun commented on issue #27902: [SPARK-31147][SQL] Forbid CHAR type in non-Hive-Serde tables URL: https://github.com/apache/spark/pull/27902#issuecomment-603942303 Hi, @cloud-fan . There was a conflict on branch-3.0. Could you make a backporting PR, please? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
