[GitHub] [spark] MaxGekk commented on a change in pull request #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource

MaxGekk commented on a change in pull request #27366: URL: https://github.com/apache/spark/pull/27366#discussion_r451582334 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonFilters.scala ## @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache

[GitHub] [spark] SparkQA commented on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

SparkQA commented on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-655549681 **[Test build #125372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125372/testReport)** for PR 29034 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #29040: [SPARK-32230][R][INFRA] Use Hadoop 2.7 profile in AppVeyor SparkR build

HyukjinKwon commented on pull request #29040: URL: https://github.com/apache/spark/pull/29040#issuecomment-655549870 I filed a JIRA for that at SPARK-32231 This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins commented on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

AmplabJenkins commented on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-655549139 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

AmplabJenkins removed a comment on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-655549139 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] MaxGekk commented on a change in pull request #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource

MaxGekk commented on a change in pull request #27366: URL: https://github.com/apache/spark/pull/27366#discussion_r451581234 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/StructFilters.scala ## @@ -0,0 +1,161 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] SparkQA commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655548572 **[Test build #125371 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125371/testReport)** for PR 29039 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #29040: [SPARK-32230][R][INFRA] Use Hadoop 2.7 profile in AppVeyor SparkR build

HyukjinKwon commented on pull request #29040: URL: https://github.com/apache/spark/pull/29040#issuecomment-655548695 cc @dongjoon-hyun. BTW, I will use Hadoop 3 profile in AppVeyor later in a separate PR. This is an

[GitHub] [spark] MaxGekk commented on a change in pull request #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource

MaxGekk commented on a change in pull request #27366: URL: https://github.com/apache/spark/pull/27366#discussion_r451578687 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/StructFilters.scala ## @@ -0,0 +1,161 @@ +/* + * Licensed to the Apache Software

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

AmplabJenkins removed a comment on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-655545966 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] maropu commented on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

maropu commented on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-655547083 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

AmplabJenkins removed a comment on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-655545955 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

AmplabJenkins commented on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-655545955 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins removed a comment on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-655544926 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

AmplabJenkins removed a comment on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-655544789 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655544813 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

SparkQA commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-655545021 **[Test build #125369 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125369/testReport)** for PR 28993 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #29017: [WIP][SPARK-XXX][TESTS] Disable SBT unidoc generation testing in Jenkins

HeartSaVioR commented on pull request #29017: URL: https://github.com/apache/spark/pull/29017#issuecomment-655544712 I hope we do the migration iteratively - we all know it became a huge pain to see the green light of the build (rarely possible according to the Jenkins PRB build trends),

[GitHub] [spark] AmplabJenkins commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-655544926 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

AmplabJenkins removed a comment on pull request #28998: URL: https://github.com/apache/spark/pull/28998#issuecomment-655543957 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655544813 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

SparkQA removed a comment on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-655426277 **[Test build #125345 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125345/testReport)** for PR 29034 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

AmplabJenkins commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-655544789 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29034: [SPARK-32219][SQL] Add SHOW CACHED TABLES Command

SparkQA commented on pull request #29034: URL: https://github.com/apache/spark/pull/29034#issuecomment-65552 **[Test build #125345 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125345/testReport)** for PR 29034 at commit

[GitHub] [spark] SparkQA commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

SparkQA commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-655544417 **[Test build #125370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125370/testReport)** for PR 28683 at commit

[GitHub] [spark] HeartSaVioR commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

HeartSaVioR commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655544209 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

AmplabJenkins removed a comment on pull request #28998: URL: https://github.com/apache/spark/pull/28998#issuecomment-655543947 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

AmplabJenkins commented on pull request #28998: URL: https://github.com/apache/spark/pull/28998#issuecomment-655543947 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] cloud-fan commented on pull request #28993: [SPARK-32168][SQL] Fix hidden partitioning correctness bug in SQL overwrite

cloud-fan commented on pull request #28993: URL: https://github.com/apache/spark/pull/28993#issuecomment-655542118 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] cloud-fan commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

cloud-fan commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-655541763 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins removed a comment on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-655540149 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655540609 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29041: [SPARK-20680][SQL][FOLLOW-UP] Revert NullType.simpleString from 'unknown' to 'null'

AmplabJenkins removed a comment on pull request #29041: URL: https://github.com/apache/spark/pull/29041#issuecomment-655540433 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins removed a comment on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-655540135 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655540609 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655540008 **[Test build #125368 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125368/testReport)** for PR 29039 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29041: [SPARK-20680][SQL][FOLLOW-UP] Revert NullType.simpleString from 'unknown' to 'null'

AmplabJenkins commented on pull request #29041: URL: https://github.com/apache/spark/pull/29041#issuecomment-655540433 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

AmplabJenkins commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-655540135 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA removed a comment on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

SparkQA removed a comment on pull request #28998: URL: https://github.com/apache/spark/pull/28998#issuecomment-655344891 **[Test build #125327 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125327/testReport)** for PR 28998 at commit

[GitHub] [spark] SparkQA commented on pull request #29041: [SPARK-20680][SQL][FOLLOW-UP] Revert NullType.simpleString from 'unknown' to 'null'

SparkQA commented on pull request #29041: URL: https://github.com/apache/spark/pull/29041#issuecomment-655540157 **[Test build #125367 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125367/testReport)** for PR 29041 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #28859: [SPARK-32024][WEBUI] Update ApplicationStoreInfo.size during HistoryServerDiskManager initializing

HyukjinKwon commented on pull request #28859: URL: https://github.com/apache/spark/pull/28859#issuecomment-655539393 Honestly, I do merge too time to time e.g.) when there are some reasons or when the Jenkins build itself is completely orthogonal such as fixing AppVeyor CI issue .. but

[GitHub] [spark] SparkQA commented on pull request #28998: [SPARK-32173][SQL] Deduplicate code in FromUTCTimestamp and ToUTCTimestamp

SparkQA commented on pull request #28998: URL: https://github.com/apache/spark/pull/28998#issuecomment-655539732 **[Test build #125327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125327/testReport)** for PR 28998 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

SparkQA removed a comment on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-655472327 **[Test build #125356 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125356/testReport)** for PR 28683 at commit

[GitHub] [spark] SparkQA commented on pull request #28683: [SPARK-31875][SQL] Provide a option to disable user supplied Hints

SparkQA commented on pull request #28683: URL: https://github.com/apache/spark/pull/28683#issuecomment-655537963 **[Test build #125356 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125356/testReport)** for PR 28683 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655536916 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655536902 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655536902 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon opened a new pull request #29041: [SPARK-20680][SQL][FOLLOW-UP] Revert NullType.simpleString from 'unknown' to 'null'

HyukjinKwon opened a new pull request #29041: URL: https://github.com/apache/spark/pull/29041 ### What changes were proposed in this pull request? This PR proposes to partially reverts the simple string in `NullType` at https://github.com/apache/spark/pull/28833:

[GitHub] [spark] SparkQA removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655500419 **[Test build #125363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125363/testReport)** for PR 29039 at commit

[GitHub] [spark] SparkQA commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655535127 **[Test build #125363 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125363/testReport)** for PR 29039 at commit

[GitHub] [spark] tgravescs commented on a change in pull request #28972: [SPARK-30794][CORE] Stage Level scheduling: Add ability to set off heap memory

tgravescs commented on a change in pull request #28972: URL: https://github.com/apache/spark/pull/28972#discussion_r451547527 ## File path: core/src/test/scala/org/apache/spark/resource/ResourceProfileSuite.scala ## @@ -256,4 +266,29 @@ class ResourceProfileSuite extends

[GitHub] [spark] HeartSaVioR commented on pull request #28859: [SPARK-32024][WEBUI] Update ApplicationStoreInfo.size during HistoryServerDiskManager initializing

HeartSaVioR commented on pull request #28859: URL: https://github.com/apache/spark/pull/28859#issuecomment-655530835 Oh OK. Thanks for noticing. I hope we'd be a bit flexible and pragmatic, but I also sincerely respect the rule and policy. I'll follow it in next time.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins removed a comment on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-655522163 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-655522163 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #28859: [SPARK-32024][WEBUI] Update ApplicationStoreInfo.size during HistoryServerDiskManager initializing

HyukjinKwon commented on pull request #28859: URL: https://github.com/apache/spark/pull/28859#issuecomment-655522336 Hey @HeartSaVioR, let's don't merge PRs when tests are being failed. This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

SparkQA commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-655521983 **[Test build #125366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125366/testReport)** for PR 29019 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655521039 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655521030 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] MaxGekk commented on a change in pull request #27366: [SPARK-30648][SQL] Support filters pushdown in JSON datasource

MaxGekk commented on a change in pull request #27366: URL: https://github.com/apache/spark/pull/27366#discussion_r451545802 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JsonFilters.scala ## @@ -0,0 +1,133 @@ +/* + * Licensed to the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655521030 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655520302 **[Test build #125365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125365/testReport)** for PR 29039 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655504940 **[Test build #125365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125365/testReport)** for PR 29039 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

HyukjinKwon commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-655517986 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins removed a comment on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-655516134 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-655516113 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

AmplabJenkins removed a comment on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-655516113 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA removed a comment on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

SparkQA removed a comment on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-655472334 **[Test build #125354 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125354/testReport)** for PR 29019 at commit

[GitHub] [spark] SparkQA commented on pull request #29019: [2.4][SPARK-32167][SQL] Fix GetArrayStructFields to respect inner field's nullability together

SparkQA commented on pull request #29019: URL: https://github.com/apache/spark/pull/29019#issuecomment-655512819 **[Test build #125354 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125354/testReport)** for PR 29019 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29002: [SPARK-32175][CORE] Fix the order between initialization for ExecutorPlugin and starting heartbeat thread

AmplabJenkins removed a comment on pull request #29002: URL: https://github.com/apache/spark/pull/29002#issuecomment-655510817 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29002: [SPARK-32175][CORE] Fix the order between initialization for ExecutorPlugin and starting heartbeat thread

AmplabJenkins removed a comment on pull request #29002: URL: https://github.com/apache/spark/pull/29002#issuecomment-655510803 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #29002: [SPARK-32175][CORE] Fix the order between initialization for ExecutorPlugin and starting heartbeat thread

AmplabJenkins commented on pull request #29002: URL: https://github.com/apache/spark/pull/29002#issuecomment-655510803 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

HeartSaVioR commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655508577 retest this, please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] Ngone51 commented on a change in pull request #29028: [SPARK-32212][CORE]RDD.takeOrdered can choose to merge intermediate r…

Ngone51 commented on a change in pull request #29028: URL: https://github.com/apache/spark/pull/29028#discussion_r451528174 ## File path: core/src/main/scala/org/apache/spark/rdd/RDD.scala ## @@ -1509,22 +1509,26 @@ abstract class RDD[T: ClassTag]( * @return an array of

[GitHub] [spark] SparkQA commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655504940 **[Test build #125365 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125365/testReport)** for PR 29039 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655504789 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #29002: [SPARK-32175][CORE] Fix the order between initialization for ExecutorPlugin and starting heartbeat thread

SparkQA commented on pull request #29002: URL: https://github.com/apache/spark/pull/29002#issuecomment-655505147 **[Test build #125316 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125316/testReport)** for PR 29002 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #29002: [SPARK-32175][CORE] Fix the order between initialization for ExecutorPlugin and starting heartbeat thread

SparkQA removed a comment on pull request #29002: URL: https://github.com/apache/spark/pull/29002#issuecomment-655337239 **[Test build #125316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125316/testReport)** for PR 29002 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655504789 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR commented on pull request #28859: [SPARK-32024][WEBUI] Update ApplicationStoreInfo.size during HistoryServerDiskManager initializing

HeartSaVioR commented on pull request #28859: URL: https://github.com/apache/spark/pull/28859#issuecomment-655504143 Thanks @zhli1142015 for the contribution! Merged into master, branch-3.0, branch-2.4 respectively. This is

[GitHub] [spark] HeartSaVioR closed pull request #28859: [SPARK-32024][WEBUI] Update ApplicationStoreInfo.size during HistoryServerDiskManager initializing

HeartSaVioR closed pull request #28859: URL: https://github.com/apache/spark/pull/28859 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655502223 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655502223 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] maropu commented on pull request #27983: [SPARK-32105][SQL]Refactor current ScriptTransformationExec code

maropu commented on pull request #27983: URL: https://github.com/apache/spark/pull/27983#issuecomment-655501312 Could you update the PR description, too? This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29040: [WIP] Use Hadoop 2.7 profile in AppVeyor SparkR build

AmplabJenkins removed a comment on pull request #29040: URL: https://github.com/apache/spark/pull/29040#issuecomment-655500970 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] zhli1142015 commented on pull request #28859: [SPARK-32024][WEBUI] Update ApplicationStoreInfo.size during HistoryServerDiskManager initializing

zhli1142015 commented on pull request #28859: URL: https://github.com/apache/spark/pull/28859#issuecomment-655501284 Thank you for your helping @HeartSaVioR . cc @gengliangwang , is it ok to merge this PR, or I need to run test again?

[GitHub] [spark] AmplabJenkins commented on pull request #29040: [WIP] Use Hadoop 2.7 profile in AppVeyor SparkR build

AmplabJenkins commented on pull request #29040: URL: https://github.com/apache/spark/pull/29040#issuecomment-655500970 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

HeartSaVioR commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655500750 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] SparkQA commented on pull request #29040: [WIP] Use Hadoop 2.7 profile in AppVeyor SparkR build

SparkQA commented on pull request #29040: URL: https://github.com/apache/spark/pull/29040#issuecomment-655500364 **[Test build #125362 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125362/testReport)** for PR 29040 at commit

[GitHub] [spark] SparkQA commented on pull request #29038: [MINOR][INFRA][R] Show the installed packages in R in a prettier way

SparkQA commented on pull request #29038: URL: https://github.com/apache/spark/pull/29038#issuecomment-655500319 **[Test build #125364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125364/testReport)** for PR 29038 at commit

[GitHub] [spark] SparkQA commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655500419 **[Test build #125363 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125363/testReport)** for PR 29039 at commit

[GitHub] [spark] HyukjinKwon opened a new pull request #29040: [WIP] Use Hadoop 2.7 profile in AppVeyor SparkR build

HyukjinKwon opened a new pull request #29040: URL: https://github.com/apache/spark/pull/29040 ### What changes were proposed in this pull request? https://ci.appveyor.com/project/ApacheSoftwareFoundation/spark/builds/33976604 AppVeyor build seems broken after we change the

[GitHub] [spark] HeartSaVioR commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

HeartSaVioR commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655497736 retest this, please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29038: [MINOR][INFRA][R] Show the installed packages in R in a prettier way

AmplabJenkins removed a comment on pull request #29038: URL: https://github.com/apache/spark/pull/29038#issuecomment-655497174 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins removed a comment on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655497233 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

AmplabJenkins commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655497233 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HeartSaVioR commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

HeartSaVioR commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655497019 Rationalization: the failures always come from classloader issue; ``` sbt.ForkMain$ForkError: org.mockito.exceptions.base.MockitoException: ClassCastException

[GitHub] [spark] AmplabJenkins commented on pull request #29038: [MINOR][INFRA][R] Show the installed packages in R in a prettier way

AmplabJenkins commented on pull request #29038: URL: https://github.com/apache/spark/pull/29038#issuecomment-655497174 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #29039: [WIP][SPARK-XXXXX] HiveSessionImplSuite flakiness fix via mocking instances earlier than initializing HiveSessionImpl

SparkQA commented on pull request #29039: URL: https://github.com/apache/spark/pull/29039#issuecomment-655496732 **[Test build #125361 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/125361/testReport)** for PR 29039 at commit

[GitHub] [spark] maropu commented on a change in pull request #27983: [SPARK-32105][SQL]Refactor current ScriptTransformationExec code

maropu commented on a change in pull request #27983: URL: https://github.com/apache/spark/pull/27983#discussion_r451514820 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/script/ScriptTransformbase.scala ## @@ -0,0 +1,167 @@ +/* + * Licensed to the Apache

[GitHub] [spark] HyukjinKwon commented on pull request #29038: [MINOR][INFRA][R] Show the installed packages in R in a prettier way

HyukjinKwon commented on pull request #29038: URL: https://github.com/apache/spark/pull/29038#issuecomment-655494623 The tests failure in AppVeyor is orthogonal. It still prints fine the package information: ``` [00:01:52] Package Version [00:01:52] arrow

<    4   5   6   7   8   9   10   11   12   13   >