[GitHub] [spark] SparkQA removed a comment on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
SparkQA removed a comment on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980847213 **[Test build #145680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145680/testReport)** for PR 33896 at commit

[GitHub] [spark] SparkQA commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
SparkQA commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980979620 **[Test build #145680 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145680/testReport)** for PR 33896 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980982180 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145680/ -- This

[GitHub] [spark] HyukjinKwon commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
HyukjinKwon commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981070123 The demo looks good. cc @falaki @felixcheung @shivaram FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] HyukjinKwon commented on pull request #34725: [SPARK-37473][CORE] BypassMergeSortShuffleWriter may loss data when disk is missing however catagory is present

2021-11-28 Thread GitBox
HyukjinKwon commented on pull request #34725: URL: https://github.com/apache/spark/pull/34725#issuecomment-981070349 Mind filling the PR description for: - Does this PR introduce any user-facing change? - How was this patch tested? -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981102390 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145682/

[GitHub] [spark] gengliangwang commented on a change in pull request #34729: [SPARK-37475][SQL] Adds scale parameeter to floor and ceil functions

2021-11-28 Thread GitBox
gengliangwang commented on a change in pull request #34729: URL: https://github.com/apache/spark/pull/34729#discussion_r757919629 ## File path: sql/core/src/test/resources/sql-tests/results/operators.sql.out ## @@ -357,87 +357,87 @@ struct -- !query select ceiling(0) --

[GitHub] [spark] rshkv commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-28 Thread GitBox
rshkv commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-981126049 I noticed that `IntegralExtensionOpsTest.test_invert` fails on Pandas 1.0.0 and succeeds on 1.0.1 (error below). So Maybe safer to recommend that version. Otherwise everything

[GitHub] [spark] agrawaldevesh commented on a change in pull request #34659: [SPARK-34863][SQL] Support complex types for Parquet vectorized reader

2021-11-28 Thread GitBox
agrawaldevesh commented on a change in pull request #34659: URL: https://github.com/apache/spark/pull/34659#discussion_r757856767 ## File path: sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedRleValuesReader.java ## @@ -227,30 +230,340 @@

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980982180 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145680/

[GitHub] [spark] mridulm commented on a change in pull request #34720: [SPARK-37469][WebUI] unified shuffle read block time to shuffle read fetch wait time in StagePage

2021-11-28 Thread GitBox
mridulm commented on a change in pull request #34720: URL: https://github.com/apache/spark/pull/34720#discussion_r757869064 ## File path: core/src/main/scala/org/apache/spark/status/storeTypes.scala ## @@ -129,7 +129,7 @@ private[spark] object TaskIndexNames { final val

[GitHub] [spark] HyukjinKwon commented on pull request #34722: [SPARK-37319][K8S][FOLLOWUP] Set JAVA_HOME for Java 17 installed by apt-get

2021-11-28 Thread GitBox
HyukjinKwon commented on pull request #34722: URL: https://github.com/apache/spark/pull/34722#issuecomment-981070908 cc @dongjoon-hyun and @holdenk FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981071794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981083692 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50151/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981083692 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50151/

[GitHub] [spark] srowen commented on a change in pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
srowen commented on a change in pull request #34728: URL: https://github.com/apache/spark/pull/34728#discussion_r757905025 ## File path: R/pkg/vignettes/sparkr-vignettes.Rmd ## @@ -76,10 +76,10 @@ if (override_eval) { We use default settings in which it runs in local mode. It

[GitHub] [spark] SparkQA removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981093365 **[Test build #145682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145682/testReport)** for PR 34728 at commit

[GitHub] [spark] zero323 commented on a change in pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
zero323 commented on a change in pull request #34728: URL: https://github.com/apache/spark/pull/34728#discussion_r757915138 ## File path: R/pkg/vignettes/sparkr-vignettes.Rmd ## @@ -76,10 +76,10 @@ if (override_eval) { We use default settings in which it runs in local mode.

[GitHub] [spark] AmplabJenkins commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-98478 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50152/ --

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981117094 **[Test build #145683 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145683/testReport)** for PR 34728 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981120228 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145683/

[GitHub] [spark] AmplabJenkins commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981074950 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145681/ -- This

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981099222 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] zero323 commented on a change in pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
zero323 commented on a change in pull request #34728: URL: https://github.com/apache/spark/pull/34728#discussion_r757913845 ## File path: R/pkg/vignettes/sparkr-vignettes.Rmd ## @@ -76,10 +76,10 @@ if (override_eval) { We use default settings in which it runs in local mode.

[GitHub] [spark] rshkv edited a comment on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-28 Thread GitBox
rshkv edited a comment on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-981126049 I noticed that `IntegralExtensionOpsTest.test_invert` fails on Pandas 1.0.0 and succeeds on 1.0.1 (error below). So maybe safer to recommend that version. Otherwise

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981126910 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50153/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980974357 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50150/

[GitHub] [spark] SparkQA removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981066875 **[Test build #145681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145681/testReport)** for PR 34728 at commit

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981093365 **[Test build #145682 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145682/testReport)** for PR 34728 at commit

[GitHub] [spark] zero323 commented on a change in pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
zero323 commented on a change in pull request #34728: URL: https://github.com/apache/spark/pull/34728#discussion_r757915824 ## File path: R/pkg/_pkgdown.yml ## @@ -0,0 +1,293 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] [spark] zero323 commented on a change in pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
zero323 commented on a change in pull request #34728: URL: https://github.com/apache/spark/pull/34728#discussion_r757915824 ## File path: R/pkg/_pkgdown.yml ## @@ -0,0 +1,293 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] [spark] AmplabJenkins commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-980974357 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50150/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981074950 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145681/

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981118183 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50153/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981120228 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145683/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981128487 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50153/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981128487 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50153/

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981066875 **[Test build #145681 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145681/testReport)** for PR 34728 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #34715: [SPARK-37445][BUILD] Rename the maven hadoop profile to hadoop-3 and hadoop-2

2021-11-28 Thread GitBox
HyukjinKwon commented on pull request #34715: URL: https://github.com/apache/spark/pull/34715#issuecomment-981070745 Thanks for doing this, @AngersZh . @sunchao should be one of those who're able to take a closer look since he already took a look around here. -- This is an automated

[GitHub] [spark] holdenk commented on pull request #34722: [SPARK-37319][K8S][FOLLOWUP] Set JAVA_HOME for Java 17 installed by apt-get

2021-11-28 Thread GitBox
holdenk commented on pull request #34722: URL: https://github.com/apache/spark/pull/34722#issuecomment-981071868 LGTM yay java17 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981081158 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50151/ -- This is an automated message from the

[GitHub] [spark] srowen commented on pull request #34710: [SPARK-37461][YARN] YARN-CLIENT mode client.appId is always null

2021-11-28 Thread GitBox
srowen commented on pull request #34710: URL: https://github.com/apache/spark/pull/34710#issuecomment-981099060 Merged to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] srowen closed pull request #34710: [SPARK-37461][YARN] YARN-CLIENT mode client.appId is always null

2021-11-28 Thread GitBox
srowen closed pull request #34710: URL: https://github.com/apache/spark/pull/34710 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981106313 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50152/ -- This is an automated message from the

[GitHub] [spark] gengliangwang commented on a change in pull request #34729: [SPARK-37475][SQL] Adds scale parameeter to floor and ceil functions

2021-11-28 Thread GitBox
gengliangwang commented on a change in pull request #34729: URL: https://github.com/apache/spark/pull/34729#discussion_r757920021 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala ## @@ -453,40 +439,26 @@ case class

[GitHub] [spark] AmplabJenkins commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981102390 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145682/ -- This

[GitHub] [spark] zero323 commented on a change in pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
zero323 commented on a change in pull request #34728: URL: https://github.com/apache/spark/pull/34728#discussion_r757915337 ## File path: R/pkg/_pkgdown.yml ## @@ -0,0 +1,293 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-98854 **[Test build #145683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145683/testReport)** for PR 34728 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-98478 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50152/

[GitHub] [spark] SparkQA removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-98854 **[Test build #145683 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145683/testReport)** for PR 34728 at commit

[GitHub] [spark] imback82 commented on a change in pull request #34726: [SPARK-33875][SQL][FOLLOWUP] Handle the char/varchar column for `Describe column` command

2021-11-28 Thread GitBox
imback82 commented on a change in pull request #34726: URL: https://github.com/apache/spark/pull/34726#discussion_r757939914 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/DataSourceV2SQLSuite.scala ## @@ -158,7 +158,8 @@ class DataSourceV2SQLSuite

[GitHub] [spark] SparkQA commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
SparkQA commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-981146654 **[Test build #145685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145685/testReport)** for PR 33896 at commit

[GitHub] [spark] sadikovi commented on pull request #34729: [SPARK-37475][SQL] Adds scale parameeter to floor and ceil functions

2021-11-28 Thread GitBox
sadikovi commented on pull request #34729: URL: https://github.com/apache/spark/pull/34729#issuecomment-981150668 @sathiyapk Can you update the PR description? There is a typo, should be `[SPARK-37475][SQL] Add scale parameter to floor and ceil functions` -- This is an automated message

[GitHub] [spark] SparkQA removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981145445 **[Test build #145684 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145684/testReport)** for PR 34728 at commit

[GitHub] [spark] sadikovi commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

2021-11-28 Thread GitBox
sadikovi commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r757952876 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -465,8 +470,19 @@ object DateTimeUtils { }

[GitHub] [spark] sadikovi commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

2021-11-28 Thread GitBox
sadikovi commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r757958467 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVOptions.scala ## @@ -164,6 +164,10 @@ class CSVOptions(

[GitHub] [spark] itholic commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-28 Thread GitBox
itholic commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-981182023 > I noticed that `IntegralExtensionOpsTest.test_invert` fails on Pandas 1.0.0 and succeeds on 1.0.1 (error below). So maybe safer to recommend that version. Otherwise

[GitHub] [spark] github-actions[bot] closed pull request #33726: [SPARK-36493] Skip Retrieving keytab with SparkFiles.get if keytab found in the CWD of Yarn Container

2021-11-28 Thread GitBox
github-actions[bot] closed pull request #33726: URL: https://github.com/apache/spark/pull/33726 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] github-actions[bot] closed pull request #33288: [SPARK-36080][SQL] Broadcast join the outer join stream side

2021-11-28 Thread GitBox
github-actions[bot] closed pull request #33288: URL: https://github.com/apache/spark/pull/33288 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] xinrong-databricks commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-28 Thread GitBox
xinrong-databricks commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-981212899 Thanks @Yikun, how do you think about bumping to `1.0.1`? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981153540 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145684/ -- This

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

2021-11-28 Thread GitBox
SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-981162646 **[Test build #145686 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145686/testReport)** for PR 34596 at commit

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

2021-11-28 Thread GitBox
SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-981169237 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50156/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

2021-11-28 Thread GitBox
SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-981175781 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50156/ -- This is an automated message from the

[GitHub] [spark] github-actions[bot] closed pull request #33746: [SPARK-36514][SQL] Support to use the meta conf in HiveMetastoreClient

2021-11-28 Thread GitBox
github-actions[bot] closed pull request #33746: URL: https://github.com/apache/spark/pull/33746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] Yikun commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-28 Thread GitBox
Yikun commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-981220341 Sure, thanks for your suggestion, I'd like to update. and I added a simple test to install pandas v1.0.1 and run test on https://github.com/apache/spark/pull/34730 , wait for the

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34559: [SPARK-37291][PYTHON][SQL] PySpark init SparkSession should copy conf to sharedState

2021-11-28 Thread GitBox
HyukjinKwon commented on a change in pull request #34559: URL: https://github.com/apache/spark/pull/34559#discussion_r758001741 ## File path: python/pyspark/sql/session.py ## @@ -301,6 +306,9 @@ def __init__(self, sparkContext: SparkContext, jsparkSession: Optional[JavaObjec

[GitHub] [spark] seayoun commented on pull request #34725: [SPARK-37473][CORE] BypassMergeSortShuffleWriter may loss data when disk is missing however catagory is present

2021-11-28 Thread GitBox
seayoun commented on pull request #34725: URL: https://github.com/apache/spark/pull/34725#issuecomment-981238230 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34725: [SPARK-37473][CORE] BypassMergeSortShuffleWriter may loss data when disk is missing however catagory is present

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34725: URL: https://github.com/apache/spark/pull/34725#issuecomment-980516157 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] seayoun removed a comment on pull request #34725: [SPARK-37473][CORE] BypassMergeSortShuffleWriter may loss data when disk is missing however catagory is present

2021-11-28 Thread GitBox
seayoun removed a comment on pull request #34725: URL: https://github.com/apache/spark/pull/34725#issuecomment-981238230 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AngersZhuuuu removed a comment on pull request #34721: [SPARK-37471][SQL] spark-sql support nested bracketed comment

2021-11-28 Thread GitBox
AngersZh removed a comment on pull request #34721: URL: https://github.com/apache/spark/pull/34721#issuecomment-979788835 ping @maropu @wangyum cc @turboFei -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] AngersZhuuuu commented on pull request #34721: [SPARK-37471][SQL] spark-sql support nested bracketed comment

2021-11-28 Thread GitBox
AngersZh commented on pull request #34721: URL: https://github.com/apache/spark/pull/34721#issuecomment-981238046 ping @cloud-fan @wangyum -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] sathiyapk commented on a change in pull request #34729: [SPARK-37475][SQL] Add scale parameter to floor and ceil functions

2021-11-28 Thread GitBox
sathiyapk commented on a change in pull request #34729: URL: https://github.com/apache/spark/pull/34729#discussion_r757960252 ## File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/float8.sql.out ## @@ -319,37 +319,37 @@ struct -- !query select ceil(f1) as

[GitHub] [spark] SparkQA commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
SparkQA commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-981169136 **[Test build #145685 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145685/testReport)** for PR 33896 at commit

[GitHub] [spark] srowen commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.0

2021-11-28 Thread GitBox
srowen commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-981182681 Yeah just require 1.0.1 for this reason -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34509: [SPARK-34521][PYTHON][SQL] Fix spark.createDataFrame when using pandas with StringDtype

2021-11-28 Thread GitBox
HyukjinKwon commented on a change in pull request #34509: URL: https://github.com/apache/spark/pull/34509#discussion_r757974181 ## File path: python/pyspark/sql/pandas/serializers.py ## @@ -169,6 +169,8 @@ def create_array(s, t): elif

[GitHub] [spark] AmplabJenkins commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-981184413 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50156/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-981184413 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50156/

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #34559: [SPARK-37291][PYTHON][SQL] PySpark init SparkSession should copy conf to sharedState

2021-11-28 Thread GitBox
AngersZh commented on a change in pull request #34559: URL: https://github.com/apache/spark/pull/34559#discussion_r758003899 ## File path: python/pyspark/sql/session.py ## @@ -301,6 +306,9 @@ def __init__(self, sparkContext: SparkContext, jsparkSession: Optional[JavaObjec

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981145445 **[Test build #145684 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145684/testReport)** for PR 34728 at commit

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981150344 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
SparkQA commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-981152508 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50155/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981153540 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145684/

[GitHub] [spark] SparkQA commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
SparkQA commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981157432 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50154/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
SparkQA commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-981160867 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50155/ -- This is an automated message from the

[GitHub] [spark] sadikovi commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

2021-11-28 Thread GitBox
sadikovi commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-981161919 @gengliangwang @cloud-fan I updated the code, could you do the penultimate review? Thanks. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] sathiyapk commented on a change in pull request #34729: [SPARK-37475][SQL] Add scale parameter to floor and ceil functions

2021-11-28 Thread GitBox
sathiyapk commented on a change in pull request #34729: URL: https://github.com/apache/spark/pull/34729#discussion_r757959074 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala ## @@ -453,40 +439,26 @@ case class

[GitHub] [spark] sathiyapk commented on pull request #34729: [SPARK-37475][SQL] Add scale parameter to floor and ceil functions

2021-11-28 Thread GitBox
sathiyapk commented on pull request #34729: URL: https://github.com/apache/spark/pull/34729#issuecomment-981165573 ``` 19:40:16.643 ERROR org.apache.spark.sql.SQLQueryTestSuite: Error using configs: [info] - describe-table-column.sql *** FAILED *** (1 second, 385 milliseconds)

[GitHub] [spark] SparkQA removed a comment on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
SparkQA removed a comment on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-981146654 **[Test build #145685 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145685/testReport)** for PR 33896 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-981171551 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145685/ -- This

[GitHub] [spark] Yikun opened a new pull request #34730: [DNM] Trigger test for 'pandas==1.0.1'

2021-11-28 Thread GitBox
Yikun opened a new pull request #34730: URL: https://github.com/apache/spark/pull/34730 Here is a test for bump pandas to 1.0.1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981162288 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50154/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33896: [SPARK-33701][SHUFFLE] Adaptive shuffle merge finalization for push-based shuffle

2021-11-28 Thread GitBox
AmplabJenkins removed a comment on pull request #33896: URL: https://github.com/apache/spark/pull/33896#issuecomment-981162289 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50155/

[GitHub] [spark] itholic commented on pull request #34509: [SPARK-34521][PYTHON][SQL] Fix spark.createDataFrame when using pandas with StringDtype

2021-11-28 Thread GitBox
itholic commented on pull request #34509: URL: https://github.com/apache/spark/pull/34509#issuecomment-981183195 It's LGTM but let's wait until other members to verify this change since I don't have permission to merge  cc @HyukjinKwon @ueshin Could you take a look at this one when

[GitHub] [spark] SparkQA commented on pull request #34730: [DNM] Trigger test for 'pandas==1.0.1'

2021-11-28 Thread GitBox
SparkQA commented on pull request #34730: URL: https://github.com/apache/spark/pull/34730#issuecomment-981232709 **[Test build #145687 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145687/testReport)** for PR 34730 at commit

[GitHub] [spark] SparkQA commented on pull request #34726: [SPARK-33875][SQL][FOLLOWUP] Handle the char/varchar column for `Describe column` command

2021-11-28 Thread GitBox
SparkQA commented on pull request #34726: URL: https://github.com/apache/spark/pull/34726#issuecomment-981232745 **[Test build #145688 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145688/testReport)** for PR 34726 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34559: [SPARK-37291][PYTHON][SQL] PySpark init SparkSession should copy conf to sharedState

2021-11-28 Thread GitBox
HyukjinKwon commented on a change in pull request #34559: URL: https://github.com/apache/spark/pull/34559#discussion_r758001741 ## File path: python/pyspark/sql/session.py ## @@ -301,6 +306,9 @@ def __init__(self, sparkContext: SparkContext, jsparkSession: Optional[JavaObjec

[GitHub] [spark] imback82 commented on pull request #34726: [SPARK-33875][SQL][FOLLOWUP] Handle the char/varchar column for `Describe column` command

2021-11-28 Thread GitBox
imback82 commented on pull request #34726: URL: https://github.com/apache/spark/pull/34726#issuecomment-981138243 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] sadikovi edited a comment on pull request #34729: [SPARK-37475][SQL] Adds scale parameeter to floor and ceil functions

2021-11-28 Thread GitBox
sadikovi edited a comment on pull request #34729: URL: https://github.com/apache/spark/pull/34729#issuecomment-981150668 @sathiyapk Can you update the PR description? There is a couple of typos, should be `[SPARK-37475][SQL] Add scale parameter to floor and ceil functions` -- This is an

[GitHub] [spark] AmplabJenkins commented on pull request #34728: [WIP][SPARK-37474][R][DOCS] Migrate SparkR docs to pkgdown

2021-11-28 Thread GitBox
AmplabJenkins commented on pull request #34728: URL: https://github.com/apache/spark/pull/34728#issuecomment-981162288 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50154/ --

  1   2   3   >