[GitHub] spark issue #21605: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92176/ Test PASSed. ---

[GitHub] spark issue #21605: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21605 **[Test build #92176 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92176/testReport)** for PR 21605 at commit

[GitHub] spark pull request #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidati...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21594#discussion_r197247925 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -105,24 +105,58 @@ class CacheManager extends Logging {

[GitHub] spark pull request #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidati...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21594#discussion_r197247763 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -105,24 +105,58 @@ class CacheManager extends Logging {

[GitHub] spark pull request #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidati...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21594#discussion_r197247440 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2971,7 +2971,7 @@ class Dataset[T] private[sql]( * @since 1.6.0

[GitHub] spark pull request #21594: [SPARK-24596][SQL] Non-cascading Cache Invalidati...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21594#discussion_r197247632 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/CacheManager.scala --- @@ -105,24 +105,58 @@ class CacheManager extends Logging {

[GitHub] spark pull request #21599: [SPARK-24598][SQL] Overflow on arithmetic operati...

2018-06-21 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/21599#discussion_r197246916 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -128,17 +128,31 @@ abstract class

[GitHub] spark pull request #21537: [SPARK-24505][SQL] Convert strings in codegen to ...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21537#discussion_r197245629 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/javaCode.scala --- @@ -256,6 +283,22 @@ object EmptyBlock

[GitHub] spark pull request #21537: [SPARK-24505][SQL] Convert strings in codegen to ...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21537#discussion_r197245172 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -1004,26 +1012,29 @@ case class Cast(child:

[GitHub] spark pull request #21602: [SPARK-24613][SQL] Cache with UDF could not be ma...

2018-06-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21602 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21602 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92174/ Test PASSed. ---

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21565: [SPARK-24558][Core]wrong Idle Timeout value is used in c...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21565 **[Test build #92179 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92179/testReport)** for PR 21565 at commit

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21602 **[Test build #92174 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92174/testReport)** for PR 21602 at commit

[GitHub] spark issue #21565: [SPARK-24558][Core]wrong Idle Timeout value is used in c...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21565 **[Test build #92178 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92178/testReport)** for PR 21565 at commit

[GitHub] spark issue #21565: [SPARK-24558][Core]wrong Idle Timeout value is used in c...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21565 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21565: [SPARK-24558][Core]wrong Idle Timeout value is us...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21565#discussion_r197237620 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -488,9 +488,16 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #21599: [SPARK-24598][SQL] Overflow on arithmetic operati...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21599#discussion_r197235683 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala --- @@ -128,17 +128,31 @@ abstract class

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197235630 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21577: [SPARK-24589][core] Correctly identify tasks in o...

2018-06-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21577 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21577: [SPARK-24589][core] Correctly identify tasks in output c...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21577 > I pushed the change for that in: vanzin/spark@e6a862e I like it, it's simpler to use task id to replace stage attempt id and task attempt id. For safety we should do it in master only

[GitHub] spark pull request #21482: [SPARK-24393][SQL] SQL builtin: isinf

2018-06-21 Thread henryr
Github user henryr commented on a diff in the pull request: https://github.com/apache/spark/pull/21482#discussion_r197234189 --- Diff: python/pyspark/sql/functions.py --- @@ -468,6 +468,18 @@ def input_file_name(): return Column(sc._jvm.functions.input_file_name())

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-21 Thread markhamstra
Github user markhamstra commented on the issue: https://github.com/apache/spark/pull/21598 > so we can't just change the default value in a feature release Agreed. Once a particular interface and behavior is in our released public API, then we effectively have a contract not

[GitHub] spark issue #21587: [SPARK-24588][SS] streaming join should require HashClus...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21587 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/382/

[GitHub] spark issue #21587: [SPARK-24588][SS] streaming join should require HashClus...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21587 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21598 @rxin yes we have, I think they are all listed in the [2.4 migration guide](https://github.com/apache/spark/blob/master/docs/sql-programming-guide.md#upgrading-from-spark-sql-23-to-24)

[GitHub] spark issue #21587: [SPARK-24588][SS] streaming join should require HashClus...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21587 **[Test build #92177 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92177/testReport)** for PR 21587 at commit

[GitHub] spark issue #21587: [SPARK-24588][SS] streaming join should require HashClus...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21587 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21587: [SPARK-24588][SS] streaming join should require HashClus...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21587 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/381/

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197229189 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21598 Do we have other "legacy" configs that we haven't released and can change to match this prefix? It's pretty nice to have a single prefix for stuff like this. ---

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197229074 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark issue #21577: [SPARK-24589][core] Correctly identify tasks in output c...

2018-06-21 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21577 I will --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21558: [SPARK-24552][SQL] Use task ID instead of attempt number...

2018-06-21 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21558 If you are working on this, I'll merge the other one and wait for you and continue to investigate in parallel --- - To

[GitHub] spark pull request #21061: [SPARK-23914][SQL] Add array_union function

2018-06-21 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21061#discussion_r197228219 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -2355,3 +2355,347 @@ case class

[GitHub] spark pull request #21598: [SPARK-24605][SQL] size(null) returns null instea...

2018-06-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21598#discussion_r197227370 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1314,6 +1314,13 @@ object SQLConf { "Other column

[GitHub] spark issue #21558: [SPARK-24552][SQL] Use task ID instead of attempt number...

2018-06-21 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21558 Ah ok, was looking at my own version as well. There are other things we should update for v2 as well, other functions with the variable names, description in DataWriterFactory.java, etc.

[GitHub] spark pull request #21558: [SPARK-24552][SQL] Use task ID instead of attempt...

2018-06-21 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21558#discussion_r197222759 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala --- @@ -110,7 +108,7 @@ object

[GitHub] spark issue #21599: [SPARK-24598][SQL] Overflow on arithmetic operations ret...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21599 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92172/ Test FAILed. ---

[GitHub] spark issue #21599: [SPARK-24598][SQL] Overflow on arithmetic operations ret...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21599 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21599: [SPARK-24598][SQL] Overflow on arithmetic operations ret...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21599 **[Test build #92172 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92172/testReport)** for PR 21599 at commit

[GitHub] spark issue #21558: [SPARK-24552][SQL] Use task ID instead of attempt number...

2018-06-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21558 FYI, I'm preparing my own version of this PR with the remaining feedback addressed. Ryan was on paternity leave and I don't know whether he's done yet, so he may not be that responsive.

[GitHub] spark issue #21527: [SPARK-24519] Make the threshold for highly compressed m...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21527 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21527: [SPARK-24519] Make the threshold for highly compressed m...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21527 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92169/ Test PASSed. ---

[GitHub] spark issue #21527: [SPARK-24519] Make the threshold for highly compressed m...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21527 **[Test build #92169 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92169/testReport)** for PR 21527 at commit

[GitHub] spark issue #21577: [SPARK-24589][core] Correctly identify tasks in output c...

2018-06-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21577 So anyone wants to do the actual merging? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21598: [SPARK-24605][SQL] size(null) returns null instead of -1

2018-06-21 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/21598 This is not a "bug" and there is no "right" behavior in APIs. It's been defined as -1 since the very beginning (when was it added?), so we can't just change the default value in a feature release.

[GitHub] spark issue #21565: [SPARK-24558][Core]wrong Idle Timeout value is used in c...

2018-06-21 Thread sandeep-katta
Github user sandeep-katta commented on the issue: https://github.com/apache/spark/pull/21565 @cloud-fan can you please review this small piece of code and merge this PR --- - To unsubscribe, e-mail:

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20944 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20944 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92171/ Test PASSed. ---

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20944 **[Test build #92171 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92171/testReport)** for PR 20944 at commit

[GitHub] spark issue #21588: [SPARK-24590][BUILD] Make Jenkins tests passed with hado...

2018-06-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21588 Let me phrase the question a different way: your title says "Make jenkins tests passed" [sic]. If you check this in, and we enable a jenkins job for hadoop 3, will it pass? I'm 100% sure

[GitHub] spark issue #21548: [SPARK-24518][CORE] Using Hadoop credential provider API...

2018-06-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21548 Could you update the summary so that it doesn't sound like this is an existing security issue? --- - To unsubscribe, e-mail:

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20944 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20944 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92170/ Test PASSed. ---

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20944 **[Test build #92170 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92170/testReport)** for PR 20944 at commit

[GitHub] spark issue #21596: [SPARK-24601] Bump Jackson version

2018-06-21 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/21596 Please, make sure that performance doesn't degrade after upgrading Jackson. You can check that by

[GitHub] spark pull request #21603: [SPARK-17091][SQL] Add rule to convert IN predica...

2018-06-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/21603#discussion_r197191390 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilters.scala --- @@ -270,6 +270,11 @@ private[parquet]

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16677 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92168/ Test FAILed. ---

[GitHub] spark issue #16677: [SPARK-19355][SQL] Use map output statistics to improve ...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16677 **[Test build #92168 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92168/testReport)** for PR 16677 at commit

[GitHub] spark issue #21605: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21605 **[Test build #92176 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92176/testReport)** for PR 21605 at commit

[GitHub] spark issue #21605: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21605 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/380/

[GitHub] spark issue #21605: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21605 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21605: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-21 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21605 cc @cloud-fan @daniel-shields @WenboZhao --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-21 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21449 ok so I created https://github.com/apache/spark/pull/21605 for the fix proposed by @daniel-shields. I'd like to leave this open in order to go on with the discussion for a long-term better fix.

[GitHub] spark pull request #21605: [SPARK-24385][SQL] Resolve self-join condition am...

2018-06-21 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/21605 [SPARK-24385][SQL] Resolve self-join condition ambiguity for EqualNullSafe ## What changes were proposed in this pull request? In Dataset.join we have a small hack for resolving ambiguity

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21282 **[Test build #92175 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92175/testReport)** for PR 21282 at commit

[GitHub] spark pull request #21548: [SPARK-24518][CORE] Using Hadoop credential provi...

2018-06-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21548#discussion_r197182301 --- Diff: core/src/main/scala/org/apache/spark/SSLOptions.scala --- @@ -179,9 +185,11 @@ private[spark] object SSLOptions extends Logging {

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-21 Thread WenboZhao
Github user WenboZhao commented on the issue: https://github.com/apache/spark/pull/21449 I like the proposal by @daniel-shields. If we could get it fixed soon, we will be able to catch up the Spark 2.3.2 release. ---

[GitHub] spark issue #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwrite a p...

2018-06-21 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/21257 some overall thought * I think this is only happening on a successful job commit, not abort. This is the desired action? * if something goes wrong here, is failing the entire job

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21602 **[Test build #92174 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92174/testReport)** for PR 21602 at commit

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21602 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21602 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/379/

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-06-21 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r197177156 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -235,4 +244,41 @@ class

[GitHub] spark issue #21602: [SPARK-24613][SQL] Cache with UDF could not be matched w...

2018-06-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21602 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-06-21 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r197176461 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +210,23 @@ case

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-06-21 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r197176292 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +210,23 @@ case

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-06-21 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r197174835 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +210,23 @@ case

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-06-21 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r197173180 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -235,4 +244,41 @@ class

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-06-21 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r197172859 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -235,4 +244,41 @@ class

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21282 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/92173/ Test FAILed. ---

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21282 **[Test build #92173 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92173/testReport)** for PR 21282 at commit

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21282 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21282: [SPARK-23934][SQL] Adding map_from_entries function

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21282 **[Test build #92173 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92173/testReport)** for PR 21282 at commit

[GitHub] spark issue #21599: [SPARK-24598][SQL] Overflow on arithmetic operations ret...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21599 **[Test build #92172 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92172/testReport)** for PR 21599 at commit

[GitHub] spark issue #21599: [SPARK-24598][SQL] Overflow on arithmetic operations ret...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/378/

[GitHub] spark issue #21599: [SPARK-24598][SQL] Overflow on arithmetic operations ret...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19492: [SPARK-22228][SQL] Add support for array...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19492 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19492: [SPARK-22228][SQL] Add support for array...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19492 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/377/

[GitHub] spark issue #21577: [SPARK-24589][core] Correctly identify tasks in output c...

2018-06-21 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21577 +1 this is a bit of a side while looking through the scenarios I filed: https://issues.apache.org/jira/browse/SPARK-24622 . shouldn't be a problem here though with this fix. ---

[GitHub] spark issue #20640: [SPARK-19755][Mesos] Blacklist is always active for Meso...

2018-06-21 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/20640 fyi I just merged SPARK-16330, which is currently very yarn specific, but I think you could easily refactor that code to reuse a lot for mesos too. ---

[GitHub] spark pull request #21068: [SPARK-16630][YARN] Blacklist a node if executors...

2018-06-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21068 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21068: [SPARK-16630][YARN] Blacklist a node if executors won't ...

2018-06-21 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/21068 merged to master. Thanks @attilapiros ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21546: [WIP][SPARK-23030][SQL][PYTHON] Use Arrow stream ...

2018-06-21 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/21546#discussion_r197149246 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowConverters.scala --- @@ -183,34 +182,131 @@ private[sql] object

[GitHub] spark issue #21601: [SPARK-24610] fix reading small files via wholeTextFiles

2018-06-21 Thread dhruve
Github user dhruve commented on the issue: https://github.com/apache/spark/pull/21601 @vanzin Can you review this PR? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20944 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/376/

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-06-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20944 **[Test build #92171 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/92171/testReport)** for PR 20944 at commit

[GitHub] spark issue #20944: [SPARK-23831][SQL] Add org.apache.derby to IsolatedClien...

2018-06-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20944 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   >