[GitHub] spark issue #21131: [SPARK-23433][CORE] Late zombie task completions update ...

2018-07-17 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21131 a late LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21131: [SPARK-23433][CORE] Late zombie task completions ...

2018-07-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21131#discussion_r203257926 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -764,6 +769,19 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #21488: SPARK-18057 Update structured streaming kafka fro...

2018-07-17 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/spark/pull/21488#discussion_r203256766 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaOffsetReader.scala --- @@ -115,7 +116,7 @@ private[kafka010] class

[GitHub] spark issue #21386: [SPARK-23928][SQL][WIP] Add shuffle collection function.

2018-07-17 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/21386 Okay, I'll take this over, and ping you when I submit a PR to ask a review. Thanks! --- - To unsubscribe, e-mail:

[GitHub] spark issue #21795: [SPARK-24840][SQL] do not use dummy filter to switch cod...

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21795 **[Test build #93215 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93215/testReport)** for PR 21795 at commit

[GitHub] spark issue #21795: [SPARK-24165][SQL][followup] Fixing conditional expressi...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21795 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21795: [SPARK-24165][SQL][followup] Fixing conditional expressi...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21795 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93212/ Test PASSed. ---

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #93212 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93212/testReport)** for PR 21488 at commit

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21800 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21800 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93210/ Test PASSed. ---

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21800 **[Test build #93210 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93210/testReport)** for PR 21800 at commit

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-17 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r203252809 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -726,8 +726,9 @@ private[execution] final class

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-07-17 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r203251175 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-07-17 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r203250619 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -166,6 +170,34 @@ private[spark] class ChunkedByteBuffer(var

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21669 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93211/ Test FAILed. ---

[GitHub] spark issue #21669: [SPARK-23257][K8S][WIP] Kerberos Support for Spark on K8...

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21669 **[Test build #93211 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93211/testReport)** for PR 21669 at commit

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21800 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93209/ Test PASSed. ---

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21800 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21801: [SPARK-24386][SPARK-24768][BUILD][FOLLOWUP] Fix l...

2018-07-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21801#discussion_r203249250 --- Diff: external/avro/src/test/resources/log4j.properties --- @@ -46,4 +46,4 @@ log4j.additivity.org.apache.hadoop.hive.metastore.RetryingHMSHandler=false

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21800 **[Test build #93209 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93209/testReport)** for PR 21800 at commit

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20146 Ah, I know why. Because the latest commit https://github.com/apache/spark/pull/20146/commits/a6551b02a10428d66e0dadcfcb5a8da3798ec814 doesn't contain the changes to trigger the test. Rebasing

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21542 H this happened to me too and I just pushed https://github.com/apache/spark/commit/fc2e18963efdf4b50258f85c8779122742876910. Mine was Java 8. There's reproducer in the PR description. ---

[GitHub] spark pull request #21801: [SPARK-24386][SPARK-24768][BUILD][FOLLOWUP] Fix l...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21801#discussion_r203248740 --- Diff: external/avro/src/test/resources/log4j.properties --- @@ -46,4 +46,4 @@

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93213/ Test FAILed. ---

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #93213 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93213/testReport)** for PR 21488 at commit

[GitHub] spark issue #21801: [SPARK-24386][SPARK-24768][BUILD][FOLLOWUP] Fix lint-jav...

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21801 **[Test build #93214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93214/testReport)** for PR 21801 at commit

[GitHub] spark issue #21801: [SPARK-24386][SPARK-24768][BUILD][FOLLOWUP] Fix lint-jav...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21801 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21801: [SPARK-24386][SPARK-24768][BUILD][FOLLOWUP] Fix lint-jav...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21801 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21801: [SPARK-24386][SPARK-24768][BUILD][FOLLOWUP] Fix l...

2018-07-17 Thread ueshin
GitHub user ueshin opened a pull request: https://github.com/apache/spark/pull/21801 [SPARK-24386][SPARK-24768][BUILD][FOLLOWUP] Fix lint-java and Scala 2.12 build. ## What changes were proposed in this pull request? This pr fixes lint-java and Scala 2.12 build.

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-07-17 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r20324 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #20146: [SPARK-11215][ML] Add multiple columns support to String...

2018-07-17 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20146 it's odd appveyer tests are not getting triggered. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-07-17 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r203245221 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -166,6 +170,34 @@ private[spark] class ChunkedByteBuffer(var chunks:

[GitHub] spark pull request #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-07-17 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21710#discussion_r203245118 --- Diff: R/pkg/R/mllib_fpm.R --- @@ -154,3 +160,74 @@ setMethod("write.ml", signature(object = "FPGrowthModel", path = "character"),

[GitHub] spark pull request #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-07-17 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21710#discussion_r203245010 --- Diff: R/pkg/R/generics.R --- @@ -1415,6 +1415,13 @@ setGeneric("spark.freqItemsets", function(object) { standardGeneric("spark.freqI #'

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-07-17 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r203244832 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -166,6 +170,34 @@ private[spark] class ChunkedByteBuffer(var chunks:

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21305 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/93208/ Test PASSed. ---

[GitHub] spark issue #21305: [SPARK-24251][SQL] Add AppendData logical plan.

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21305 **[Test build #93208 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/93208/testReport)** for PR 21305 at commit

[GitHub] spark issue #21542: [SPARK-24529][Build][test-maven] Add spotbugs into maven...

2018-07-17 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21542 "permission" stuff might be Java 9 related? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21787: [SPARK-24568] Code refactoring for DataType equalsXXX me...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21787 Constant variables and squashing the logic into one function look not worth enough and overkill. Less duplication is good of course but it doesn't look worth enough for both. I would focus on

[GitHub] spark issue #21787: [SPARK-24568] Code refactoring for DataType equalsXXX me...

2018-07-17 Thread swapnilushinde
Github user swapnilushinde commented on the issue: https://github.com/apache/spark/pull/21787 true. Currently we have just 3 variations of comparing two datatypes for equality. Adding even one more equality function would easily cause writing same repetitive code which would negate

[GitHub] spark pull request #21795: [SPARK-24165][SQL][followup] Fixing conditional e...

2018-07-17 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/21795#discussion_r203242468 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2336,46 +2336,40 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark issue #21772: [SPARK-24809] [SQL] Serializing LongHashedRelation in ex...

2018-07-17 Thread liutang123
Github user liutang123 commented on the issue: https://github.com/apache/spark/pull/21772 @hvanhovell Thanks for reviewing. Losing data because the variable **cursor** in executor is 0 and serialization depends on it. I will add an UT later. ---

[GitHub] spark pull request #21772: [SPARK-24809] [SQL] Serializing LongHashedRelatio...

2018-07-17 Thread liutang123
Github user liutang123 commented on a diff in the pull request: https://github.com/apache/spark/pull/21772#discussion_r203241485 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -726,8 +726,9 @@ private[execution] final class

[GitHub] spark pull request #18138: [SPARK-20915][SQL] Make lpad/rpad with empty pad ...

2018-07-17 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/18138 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19804: [WIP][SPARK-22573][SQL] Shouldn't inferFilters if it con...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19804 Thank you @wangyum. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21795: [SPARK-24165][SQL][followup] Fixing conditional e...

2018-07-17 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21795#discussion_r203240428 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala --- @@ -2336,46 +2336,40 @@ class DataFrameSuite extends QueryTest with

[GitHub] spark issue #21784: [SPARK-24182][YARN][FOLLOW-UP] Turn off noisy log output

2018-07-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21784 It's noisy when type something: ![spark-24128](https://user-images.githubusercontent.com/5399861/42857022-8afee254-8a7a-11e8-8ee0-47a19af99fa7.gif) ---

[GitHub] spark issue #21792: [SPARK-23231][ML][DOC] Add doc for string indexer orderi...

2018-07-17 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/21792 @srowen I think we need to update the docs 1, Current doc in `StringIndexer` is somewhat misleading: "The indices are in `[0, numLabels)`, ordered by label frequencies, so the most frequent

[GitHub] spark issue #21765: [MINOR][CORE] Add test cases for RDD.cartesian

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21765 **[Test build #4218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4218/testReport)** for PR 21765 at commit

[GitHub] spark issue #21789: [SPARK-24829][SQL]CAST AS FLOAT inconsistent with Hive

2018-07-17 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/21789 yes, only in STS, i will update the title --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19804: [WIP][SPARK-22573][SQL] Shouldn't inferFilters if it con...

2018-07-17 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/19804 Thanks @HyukjinKwon, already close some. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21789: [SPARK-24829][SQL]CAST AS FLOAT inconsistent with...

2018-07-17 Thread zuotingbing
Github user zuotingbing commented on a diff in the pull request: https://github.com/apache/spark/pull/21789#discussion_r203239370 --- Diff: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/HiveThriftServer2Suites.scala --- @@ -766,6 +774,14 @@ class

[GitHub] spark issue #21754: [SPARK-24705][SQL] Cannot reuse an exchange operator wit...

2018-07-17 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21754 ping --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #15945: [SPARK-12978][SQL] Merge unnecessary partial aggregates

2018-07-17 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/15945 I'll close for now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #15945: [SPARK-12978][SQL] Merge unnecessary partial aggr...

2018-07-17 Thread maropu
Github user maropu closed the pull request at: https://github.com/apache/spark/pull/15945 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16605: [SPARK-18884][SQL] Throw an exception in compile time if...

2018-07-17 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/16605 I'll close for now --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #16605: [SPARK-18884][SQL] Throw an exception in compile ...

2018-07-17 Thread maropu
Github user maropu closed the pull request at: https://github.com/apache/spark/pull/16605 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21460: [SPARK-23442][SQL] Increase reading tasks when re...

2018-07-17 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/21460 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19804: [WIP][SPARK-22573][SQL] Shouldn't inferFilters if...

2018-07-17 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/19804 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20248: [SPARK-23058][SQL] Show non printable field delim...

2018-07-17 Thread wangyum
Github user wangyum closed the pull request at: https://github.com/apache/spark/pull/20248 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19804: [WIP][SPARK-22573][SQL] Shouldn't inferFilters if it con...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19804 @wangyum, I think you could leave close some of PRs that you are currently not working on and have no explicit plan yet to start to work on soon. You could reopen or create new one when you

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-07-17 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r203237484 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBufferFileRegion.scala --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #18544: [SPARK-21318][SQL]Improve exception message thrown by `l...

2018-07-17 Thread stanzhai
Github user stanzhai commented on the issue: https://github.com/apache/spark/pull/18544 cc @gatorsmile changes in `sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala` has been reverted. ---

[GitHub] spark issue #21192: [SPARK-24118][SQL] Flexible format for the lineSep optio...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21192 To me, yes. If you find some times, I would appreciate if you take a look at https://github.com/apache/spark/pull/21192#issuecomment-391405633 too so that we can review each other whoever make

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-07-17 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r203236014 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -166,6 +170,34 @@ private[spark] class ChunkedByteBuffer(var

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r203235747 --- Diff: R/pkg/R/context.R --- @@ -437,3 +437,33 @@ setCheckpointDir <- function(directory) { sc <- getSparkContext()

[GitHub] spark pull request #21589: [SPARK-24591][CORE] Number of cores and executors...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21589#discussion_r203235671 --- Diff: R/pkg/R/context.R --- @@ -437,3 +437,33 @@ setCheckpointDir <- function(directory) { sc <- getSparkContext()

[GitHub] spark pull request #21440: [SPARK-24307][CORE] Support reading remote cached...

2018-07-17 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21440#discussion_r203235292 --- Diff: core/src/main/scala/org/apache/spark/util/io/ChunkedByteBuffer.scala --- @@ -17,17 +17,21 @@ package org.apache.spark.util.io

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/21800 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21784: [SPARK-24182][YARN][FOLLOW-UP] Turn off noisy log output

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21784 Hm, yea. I don't think find this super noisy though to be honest. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19819: [SPARK-22606][Streaming]Add threadId to the Cache...

2018-07-17 Thread eatoncys
Github user eatoncys closed the pull request at: https://github.com/apache/spark/pull/19819 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set s...

2018-07-17 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21785 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set spotbugs...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21785 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21785: [SPARK-24529][BUILD][test-maven][FOLLOW-UP] Set spotbugs...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21785 @kiszk, let me just push this in since it fixes an actual issue and the build passes; however, please make a followup if you see something else to fix. ---

[GitHub] spark pull request #21635: [SPARK-24594][YARN] Introducing metrics for YARN

2018-07-17 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21635#discussion_r203232034 --- Diff: docs/monitoring.md --- @@ -435,6 +435,7 @@ set of sinks to which metrics are reported. The following instances are currentl * `executor`:

[GitHub] spark issue #19819: [SPARK-22606][Streaming]Add threadId to the CachedKafkaC...

2018-07-17 Thread lvdongr
Github user lvdongr commented on the issue: https://github.com/apache/spark/pull/19819 I've seen your PR: https://github.com/apache/spark/pull/20997, a good solution @gaborgsomogyi --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-07-17 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21710#discussion_r203229835 --- Diff: mllib/src/main/scala/org/apache/spark/ml/r/PrefixSpanWrapper.scala --- @@ -0,0 +1,42 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-07-17 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21710#discussion_r203229794 --- Diff: R/pkg/tests/fulltests/test_mllib_fpm.R --- @@ -82,4 +82,26 @@ test_that("spark.fpGrowth", { })

[GitHub] spark pull request #21710: [SPARK-24207][R]add R API for PrefixSpan

2018-07-17 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21710#discussion_r203229733 --- Diff: R/pkg/R/generics.R --- @@ -1415,6 +1415,13 @@ setGeneric("spark.freqItemsets", function(object) { standardGeneric("spark.freqI #' @rdname

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r203229065 --- Diff: python/pyspark/sql/readwriter.py --- @@ -895,6 +895,8 @@ def csv(self, path, mode=None, compression=None, sep=None, quote=None, escape=No

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r203228930 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -512,6 +513,43 @@ class CSVSuite extends

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r203228844 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -512,6 +513,43 @@ class CSVSuite extends

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r203228679 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -512,6 +513,43 @@ class CSVSuite extends

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r203228640 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -512,6 +513,43 @@ class CSVSuite extends

[GitHub] spark pull request #21635: [SPARK-24594][YARN] Introducing metrics for YARN

2018-07-17 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21635#discussion_r203228423 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMasterSource.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r203228403 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/csv/CSVSuite.scala --- @@ -512,6 +513,43 @@ class CSVSuite extends

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r203228243 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -146,7 +148,12 @@ private[csv] class

[GitHub] spark pull request #20949: [SPARK-19018][SQL] Add support for custom encodin...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20949#discussion_r203227873 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -625,6 +625,7 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #21787: [SPARK-24568] Code refactoring for DataType equalsXXX me...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21787 But the current code looks less readable and adding more lines. I would rather leave this as was. --- - To unsubscribe,

[GitHub] spark pull request #21798: [SPARK-24836][SQL] New option for Avro datasource...

2018-07-17 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21798#discussion_r203226401 --- Diff: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala --- @@ -276,10 +274,15 @@ private[avro] object AvroFileFormat {

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21800 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21800 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/1081/ ---

[GitHub] spark issue #21800: [SPARK-24825][K8S][TEST] Kubernetes integration tests bu...

2018-07-17 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21800 Test PASSed. Refer to this link for build results (access rights to CI server needed):

  1   2   3   4   5   6   7   >