[GitHub] [spark] AmplabJenkins removed a comment on issue #25418: [SPARK-28695][SS] Use CaseInsensitiveMap in KafkaSourceProvider to make source param handling more robust

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25418: [SPARK-28695][SS] Use CaseInsensitiveMap in KafkaSourceProvider to make source param handling more robust URL: https://github.com/apache/spark/pull/25418#issuecomment-520420110 Test PASSed. Refer to this link for build results (access rig

[GitHub] [spark] SparkQA removed a comment on issue #25418: [SPARK-28695][SS] Use CaseInsensitiveMap in KafkaSourceProvider to make source param handling more robust

2019-08-12 Thread GitBox
SparkQA removed a comment on issue #25418: [SPARK-28695][SS] Use CaseInsensitiveMap in KafkaSourceProvider to make source param handling more robust URL: https://github.com/apache/spark/pull/25418#issuecomment-520411806 **[Test build #108971 has started](https://amplab.cs.berkeley.edu/jen

[GitHub] [spark] AmplabJenkins commented on issue #25398: [SPARK-28659][SQL] Use data source if convertible in insert overwrite directory

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25398: [SPARK-28659][SQL] Use data source if convertible in insert overwrite directory URL: https://github.com/apache/spark/pull/25398#issuecomment-520421689 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #25398: [SPARK-28659][SQL] Use data source if convertible in insert overwrite directory

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25398: [SPARK-28659][SQL] Use data source if convertible in insert overwrite directory URL: https://github.com/apache/spark/pull/25398#issuecomment-520421694 Test PASSed. Refer to this link for build results (access rights to CI server needed): http

[GitHub] [spark] cloud-fan commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-08-12 Thread GitBox
cloud-fan commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#discussion_r312920338 ## File path: core/src/test/scala/org/apache/spark/scheduler/

[GitHub] [spark] cloud-fan commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-08-12 Thread GitBox
cloud-fan commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#discussion_r312920818 ## File path: core/src/test/scala/org/apache/spark/shuffle/so

[GitHub] [spark] AmplabJenkins removed a comment on issue #25398: [SPARK-28659][SQL] Use data source if convertible in insert overwrite directory

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25398: [SPARK-28659][SQL] Use data source if convertible in insert overwrite directory URL: https://github.com/apache/spark/pull/25398#issuecomment-520421694 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25398: [SPARK-28659][SQL] Use data source if convertible in insert overwrite directory

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25398: [SPARK-28659][SQL] Use data source if convertible in insert overwrite directory URL: https://github.com/apache/spark/pull/25398#issuecomment-520421689 Merged build finished. Test PASSed. --

[GitHub] [spark] cloud-fan commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files

2019-08-12 Thread GitBox
cloud-fan commented on a change in pull request #24892: [SPARK-25341][Core] Support rolling back a shuffle map stage and re-generate the shuffle files URL: https://github.com/apache/spark/pull/24892#discussion_r312922279 ## File path: core/src/main/java/org/apache/spark/shuffle/api

[GitHub] [spark] skonto commented on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql'

2019-08-12 Thread GitBox
skonto commented on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql' URL: https://github.com/apache/spark/pull/25360#issuecomment-520423877 @HyukjinKwon done.

[GitHub] [spark] AmplabJenkins commented on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql'

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql' URL: https://github.com/apache/spark/pull/25360#issuecomment-520424585 Test PASSed. Refer to this link for build results (access rights to CI se

[GitHub] [spark] AmplabJenkins commented on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql'

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql' URL: https://github.com/apache/spark/pull/25360#issuecomment-520424573 Merged build finished. Test PASSed. ---

[GitHub] [spark] AmplabJenkins removed a comment on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql'

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql' URL: https://github.com/apache/spark/pull/25360#issuecomment-520424573 Merged build finished. Test PASSed. ---

[GitHub] [spark] AmplabJenkins removed a comment on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql'

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql' URL: https://github.com/apache/spark/pull/25360#issuecomment-520424585 Test PASSed. Refer to this link for build results (access rights

[GitHub] [spark] SparkQA commented on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql'

2019-08-12 Thread GitBox
SparkQA commented on issue #25360: [SPARK-28280][PYTHON][SQL][TESTS][FOLLOW-UP] Add UDF cases into group by clause in 'udf-group-by.sql' URL: https://github.com/apache/spark/pull/25360#issuecomment-520425350 **[Test build #108973 has started](https://amplab.cs.berkeley.edu/jenkins/job/Spar

[GitHub] [spark] srowen commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location, only if it is empty or does not exists.

2019-08-12 Thread GitBox
srowen commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location,only if it is empty or does not exists. URL: https://github.com/apache/spark/pull/22466#discussion_r312924860 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catal

[GitHub] [spark] cloud-fan commented on a change in pull request #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
cloud-fan commented on a change in pull request #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#discussion_r312925274 ## File path: external/avro/src/test/scala/org/apache/spark/sql/avro/AvroCatalys

[GitHub] [spark] srowen commented on issue #25331: [SPARK-27768][SQL] Infinity, -Infinity, NaN should be recognized in a case insensitive manner.

2019-08-12 Thread GitBox
srowen commented on issue #25331: [SPARK-27768][SQL] Infinity, -Infinity, NaN should be recognized in a case insensitive manner. URL: https://github.com/apache/spark/pull/25331#issuecomment-520425599 I agree now, put a line in the migration guide ---

[GitHub] [spark] cloud-fan commented on a change in pull request #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
cloud-fan commented on a change in pull request #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#discussion_r312925637 ## File path: python/pyspark/sql/avro/functions.py ## @@ -69,26 +69,35

[GitHub] [spark] cloud-fan commented on a change in pull request #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
cloud-fan commented on a change in pull request #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#discussion_r312925637 ## File path: python/pyspark/sql/avro/functions.py ## @@ -69,26 +69,35

[GitHub] [spark] cloud-fan commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
cloud-fan commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520426316 makes sense to me, let's fix the build first.

[GitHub] [spark] srowen commented on a change in pull request #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect

2019-08-12 Thread GitBox
srowen commented on a change in pull request #25344: [WIP][SPARK-28151][SQL] Mapped ByteType to TinyINT for MsSQLServerDialect URL: https://github.com/apache/spark/pull/25344#discussion_r312926123 ## File path: external/docker-integration-tests/src/test/scala/org/apache/spark/sql/j

[GitHub] [spark] srowen closed pull request #25349: [SPARK-28538][UI] Document SQL page

2019-08-12 Thread GitBox
srowen closed pull request #25349: [SPARK-28538][UI] Document SQL page URL: https://github.com/apache/spark/pull/25349 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] sandeep-katta commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location, only if it is empty or does not exists.

2019-08-12 Thread GitBox
sandeep-katta commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location,only if it is empty or does not exists. URL: https://github.com/apache/spark/pull/22466#discussion_r312929766 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sq

[GitHub] [spark] AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520430600 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://

[GitHub] [spark] AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520430591 Merged build finished. Test PASSed. T

[GitHub] [spark] AmplabJenkins commented on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication .

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication . URL: https://github.com/apache/spark/pull/25201#issuecomment-520430705 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication .

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication . URL: https://github.com/apache/spark/pull/25201#issuecomment-520430717 Test PASSed. Refer to this link for build results (access rights to CI server needed): ht

[GitHub] [spark] AmplabJenkins removed a comment on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication .

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication . URL: https://github.com/apache/spark/pull/25201#issuecomment-520430705 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins removed a comment on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication .

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication . URL: https://github.com/apache/spark/pull/25201#issuecomment-520430717 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520430600 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520430591 Merged build finished. Test PASSed. --

[GitHub] [spark] SparkQA commented on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication .

2019-08-12 Thread GitBox
SparkQA commented on issue #25201: [SPARK-28419][SQL] Enable SparkThriftServer support proxy user's authentication . URL: https://github.com/apache/spark/pull/25201#issuecomment-520431473 **[Test build #108975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/

[GitHub] [spark] SparkQA commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
SparkQA commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520431444 **[Test build #108974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108974

[GitHub] [spark] srowen commented on issue #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos

2019-08-12 Thread GitBox
srowen commented on issue #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos URL: https://github.com/apache/spark/pull/25412#issuecomment-520433490 While I don't think we want to overcomplicate these examples by showing all the ways it could be used, this is p

[GitHub] [spark] abellina commented on a change in pull request #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
abellina commented on a change in pull request #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#discussion_r312937682 ## File path: resource-managers/yarn/src/main/

[GitHub] [spark] abellina commented on a change in pull request #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
abellina commented on a change in pull request #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#discussion_r312937740 ## File path: resource-managers/yarn/src/main/

[GitHub] [spark] AmplabJenkins commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520437008 Test PASSed. Refer to this link for build results (access rights t

[GitHub] [spark] AmplabJenkins commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520436998 Merged build finished. Test PASSed.

[GitHub] [spark] SparkQA commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
SparkQA commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520437910 **[Test build #108976 has started](https://amplab.cs.berkeley.edu/jenkins/

[GitHub] [spark] AmplabJenkins removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520437008 Test PASSed. Refer to this link for build results (access

[GitHub] [spark] AmplabJenkins removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520436998 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520440279 Merged build finished. Test PASSed. T

[GitHub] [spark] AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520440285 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://

[GitHub] [spark] AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520440285 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520440279 Merged build finished. Test PASSed. --

[GitHub] [spark] SparkQA commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
SparkQA commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520441222 **[Test build #108977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108977

[GitHub] [spark] xuanyuanking opened a new pull request #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun

2019-08-12 Thread GitBox
xuanyuanking opened a new pull request #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun URL: https://github.com/apache/spark/pull/25420 ## What changes were proposed in this pull request? It's another case for the indeterminate s

[GitHub] [spark] sandeep-katta commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location, only if it is empty or does not exists.

2019-08-12 Thread GitBox
sandeep-katta commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location,only if it is empty or does not exists. URL: https://github.com/apache/spark/pull/22466#discussion_r312946008 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sq

[GitHub] [spark] gaborgsomogyi commented on issue #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos

2019-08-12 Thread GitBox
gaborgsomogyi commented on issue #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos URL: https://github.com/apache/spark/pull/25412#issuecomment-520443270 @srowen +1 not overcomplicate. Not sure what you mean `this is possibly OK`? OK to give example on a depr

[GitHub] [spark] AmplabJenkins commented on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun URL: https://github.com/apache/spark/pull/25420#issuecomment-520443454 Test PASSed. Refer to this link for build results (access rights to CI server ne

[GitHub] [spark] xuanyuanking commented on a change in pull request #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun

2019-08-12 Thread GitBox
xuanyuanking commented on a change in pull request #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun URL: https://github.com/apache/spark/pull/25420#discussion_r312947461 ## File path: core/src/test/scala/org/apache/spark/schedu

[GitHub] [spark] AmplabJenkins commented on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun URL: https://github.com/apache/spark/pull/25420#issuecomment-520443446 Merged build finished. Test PASSed. --

[GitHub] [spark] SparkQA commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
SparkQA commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520444206 **[Test build #108974 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/10897

[GitHub] [spark] AmplabJenkins removed a comment on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun URL: https://github.com/apache/spark/pull/25420#issuecomment-520443454 Test PASSed. Refer to this link for build results (access rights to CI s

[GitHub] [spark] AmplabJenkins removed a comment on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun URL: https://github.com/apache/spark/pull/25420#issuecomment-520443446 Merged build finished. Test PASSed. --

[GitHub] [spark] SparkQA commented on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun

2019-08-12 Thread GitBox
SparkQA commented on issue #25420: [SPARK-28699][Core] Cache an indeterminate RDD could lead to incorrect result while stage rerun URL: https://github.com/apache/spark/pull/25420#issuecomment-520444288 **[Test build #108978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullR

[GitHub] [spark] SparkQA removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
SparkQA removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520431444 **[Test build #108974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilde

[GitHub] [spark] AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520444685 Test PASSed. Refer to this link for build results (access rights to CI server needed): h

[GitHub] [spark] AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520444685 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://a

[GitHub] [spark] AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520444673 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520444673 Merged build finished. Test PASSed. T

[GitHub] [spark] SparkQA commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
SparkQA commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520445308 **[Test build #108976 has finished](https://amplab.cs.berkeley.edu/jenkins

[GitHub] [spark] AmplabJenkins commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520445512 Merged build finished. Test PASSed.

[GitHub] [spark] srowen commented on issue #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos

2019-08-12 Thread GitBox
srowen commented on issue #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos URL: https://github.com/apache/spark/pull/25412#issuecomment-520445575 I would not illustrate a deprecated feature. I just mean probably OK to include something like this at all. ---

[GitHub] [spark] AmplabJenkins removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520445519 Test PASSed. Refer to this link for build results (access

[GitHub] [spark] SparkQA removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
SparkQA removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520437910 **[Test build #108976 has started](https://amplab.cs.berkeley.edu/

[GitHub] [spark] AmplabJenkins removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520445512 Merged build finished. Test PASSed.

[GitHub] [spark] AmplabJenkins commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25403: [SPARK-28679][YARN] changes to setResourceInformation to handle empty resources and reflection error handling URL: https://github.com/apache/spark/pull/25403#issuecomment-520445519 Test PASSed. Refer to this link for build results (access rights t

[GitHub] [spark] srowen commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location, only if it is empty or does not exists.

2019-08-12 Thread GitBox
srowen commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location,only if it is empty or does not exists. URL: https://github.com/apache/spark/pull/22466#discussion_r312950875 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catal

[GitHub] [spark] gaborgsomogyi commented on issue #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos

2019-08-12 Thread GitBox
gaborgsomogyi commented on issue #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos URL: https://github.com/apache/spark/pull/25412#issuecomment-520449464 Well, just checked and Kafka broker 1.1.0 introduced DT so for 1.1.0- users this is the way, so I'm fine

[GitHub] [spark] Ngone51 commented on a change in pull request #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone

2019-08-12 Thread GitBox
Ngone51 commented on a change in pull request #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone URL: https://github.com/apache/spark/pull/25409#discussion_r312958930 ## File path: core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala ##

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos

2019-08-12 Thread GitBox
gaborgsomogyi commented on a change in pull request #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos URL: https://github.com/apache/spark/pull/25412#discussion_r312956190 ## File path: examples/src/main/java/org/apache/spark/examples/streaming/JavaDi

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos

2019-08-12 Thread GitBox
gaborgsomogyi commented on a change in pull request #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos URL: https://github.com/apache/spark/pull/25412#discussion_r312956496 ## File path: examples/src/main/scala/org/apache/spark/examples/streaming/Direc

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos

2019-08-12 Thread GitBox
gaborgsomogyi commented on a change in pull request #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos URL: https://github.com/apache/spark/pull/25412#discussion_r312955921 ## File path: examples/src/main/java/org/apache/spark/examples/streaming/JavaDi

[GitHub] [spark] gaborgsomogyi commented on a change in pull request #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos

2019-08-12 Thread GitBox
gaborgsomogyi commented on a change in pull request #25412: [SPARK-28691][EXAMPLES] DirectKafkaWordCount support kafka with kerberos URL: https://github.com/apache/spark/pull/25412#discussion_r312956370 ## File path: examples/src/main/scala/org/apache/spark/examples/streaming/Direc

[GitHub] [spark] AmplabJenkins removed a comment on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone URL: https://github.com/apache/spark/pull/25409#issuecomment-520453495 Merged build finished. Test PASSed. T

[GitHub] [spark] AmplabJenkins removed a comment on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone URL: https://github.com/apache/spark/pull/25409#issuecomment-520453503 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://

[GitHub] [spark] AmplabJenkins commented on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone URL: https://github.com/apache/spark/pull/25409#issuecomment-520453503 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.c

[GitHub] [spark] AmplabJenkins commented on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone URL: https://github.com/apache/spark/pull/25409#issuecomment-520453495 Merged build finished. Test PASSed. This is a

[GitHub] [spark] SparkQA commented on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone

2019-08-12 Thread GitBox
SparkQA commented on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone URL: https://github.com/apache/spark/pull/25409#issuecomment-520454467 **[Test build #108979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108979/testRep

[GitHub] [spark] Ngone51 commented on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone

2019-08-12 Thread GitBox
Ngone51 commented on issue #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone URL: https://github.com/apache/spark/pull/25409#issuecomment-520454670 also cc @tgravescs @jiangxb1987 @mengxr This is an

[GitHub] [spark] SparkQA commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
SparkQA commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520454762 **[Test build #108977 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/10897

[GitHub] [spark] SparkQA removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
SparkQA removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520441222 **[Test build #108977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilde

[GitHub] [spark] AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520455214 Test PASSed. Refer to this link for build results (access rights to CI server needed): h

[GitHub] [spark] AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520455199 Merged build finished. Test PASSed. T

[GitHub] [spark] AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520455199 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25419: [SPARK-28698][SQL] Allow user-specified output schema in function `to_avro` URL: https://github.com/apache/spark/pull/25419#issuecomment-520455214 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://a

[GitHub] [spark] sandeep-katta commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location, only if it is empty or does not exists.

2019-08-12 Thread GitBox
sandeep-katta commented on a change in pull request #22466: [SPARK-25464][SQL] Create Database to the location,only if it is empty or does not exists. URL: https://github.com/apache/spark/pull/22466#discussion_r312963310 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sq

[GitHub] [spark] Ngone51 commented on a change in pull request #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone

2019-08-12 Thread GitBox
Ngone51 commented on a change in pull request #25409: [SPARK-28414][WEBUI] UI updates to show resource info in Standalone URL: https://github.com/apache/spark/pull/25409#discussion_r312964642 ## File path: core/src/main/scala/org/apache/spark/resource/ResourceInformation.scala ###

[GitHub] [spark] MaxGekk opened a new pull request #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()`

2019-08-12 Thread GitBox
MaxGekk opened a new pull request #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()` URL: https://github.com/apache/spark/pull/25421 ## What changes were proposed in this pull request? Changed type of `sec` argument in the `make_timestamp()` function from `D

[GitHub] [spark] AmplabJenkins commented on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()` URL: https://github.com/apache/spark/pull/25421#issuecomment-520463197 Can one of the admins verify this patch? This

[GitHub] [spark] HeartSaVioR commented on a change in pull request #25407: [SPARK-28650][SS][DOC] Correct explanation of guarantee for ForeachWriter

2019-08-12 Thread GitBox
HeartSaVioR commented on a change in pull request #25407: [SPARK-28650][SS][DOC] Correct explanation of guarantee for ForeachWriter URL: https://github.com/apache/spark/pull/25407#discussion_r312972712 ## File path: docs/structured-streaming-programming-guide.md ## @@ -225

[GitHub] [spark] AmplabJenkins commented on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()` URL: https://github.com/apache/spark/pull/25421#issuecomment-520463730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.c

[GitHub] [spark] AmplabJenkins commented on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()`

2019-08-12 Thread GitBox
AmplabJenkins commented on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()` URL: https://github.com/apache/spark/pull/25421#issuecomment-520463724 Merged build finished. Test PASSed. This is a

[GitHub] [spark] SparkQA commented on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()`

2019-08-12 Thread GitBox
SparkQA commented on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()` URL: https://github.com/apache/spark/pull/25421#issuecomment-520464644 **[Test build #108980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/108980/testRep

[GitHub] [spark] AmplabJenkins removed a comment on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()` URL: https://github.com/apache/spark/pull/25421#issuecomment-520463724 Merged build finished. Test PASSed. T

[GitHub] [spark] AmplabJenkins removed a comment on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()` URL: https://github.com/apache/spark/pull/25421#issuecomment-520463730 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://

[GitHub] [spark] AmplabJenkins removed a comment on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()`

2019-08-12 Thread GitBox
AmplabJenkins removed a comment on issue #25421: [SPARK-28700][SQL] Use DECIMAL type for `sec` in `make_timestamp()` URL: https://github.com/apache/spark/pull/25421#issuecomment-520463197 Can one of the admins verify this patch? -

[GitHub] [spark] HeartSaVioR commented on a change in pull request #25407: [SPARK-28650][SS][DOC] Correct explanation of guarantee for ForeachWriter

2019-08-12 Thread GitBox
HeartSaVioR commented on a change in pull request #25407: [SPARK-28650][SS][DOC] Correct explanation of guarantee for ForeachWriter URL: https://github.com/apache/spark/pull/25407#discussion_r312977097 ## File path: docs/structured-streaming-programming-guide.md ## @@ -225

<    1   2   3   4   5   6   7   8   9   10   >