[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r140351256 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadSupport.scala --- @@ -63,9 +74,22 @@ private[parquet] class

[GitHub] spark issue #19294: [SPARK-21549][CORE] Respect OutputFormats with no output...

2017-09-21 Thread mridulm
Github user mridulm commented on the issue: https://github.com/apache/spark/pull/19294 @szhem Did you try this patch with sql ? A cursory look at `org.apache.spark.sql.execution.datasources.FileFormatWriter` and `org.apache.spark.sql.execution.datasources.InsertIntoHadoopFsRelatio

[GitHub] spark issue #19313: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19313 **[Test build #82049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82049/testReport)** for PR 19313 at commit [`3a6b4ef`](https://github.com/apache/spark/commit/3a

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17298 **[Test build #82050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82050/testReport)** for PR 17298 at commit [`009e7c4`](https://github.com/apache/spark/commit/00

[GitHub] spark issue #11105: [SPARK-12469][CORE] Data Property accumulators for Spark

2017-09-21 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/11105 So if there is interest (cc @squito ) I'm open to update this and go with the warning signs in the doc approach if there is reviewer interest. Otherwise I'll kick the tires on this again when were c

[GitHub] spark issue #17298: [SPARK-19094][WIP][PySpark] Plumb through logging for IJ...

2017-09-21 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/17298 Jenkins retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark pull request #19313: [SPARK-21928][CORE] Set classloader on Serializer...

2017-09-21 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/19313 [SPARK-21928][CORE] Set classloader on SerializerManager's private kryo ## What changes were proposed in this pull request? We have to make sure that SerializerManager's private instance of

[GitHub] spark issue #9207: [SPARK-11171][SPARK-11237][SPARK-11241][ML] Try adding PM...

2017-09-21 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/9207 @MLNick: Do you have the bandwith to revisit this? I'm open to refactoring to a more plug-gable approach if we've got the review bandwidth for it. --- --

[GitHub] spark issue #18994: [SPARK-21784][SQL] Adds support for defining information...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18994 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18994: [SPARK-21784][SQL] Adds support for defining information...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18994 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82039/ Test PASSed. ---

[GitHub] spark issue #18994: [SPARK-21784][SQL] Adds support for defining information...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18994 **[Test build #82039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82039/testReport)** for PR 18994 at commit [`ea39601`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #19312: [SPARK-22072][SPARK-22071][BUILD]Improve release build s...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19312 **[Test build #82048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82048/testReport)** for PR 19312 at commit [`aa4cbf6`](https://github.com/apache/spark/commit/aa

[GitHub] spark pull request #19312: [SPARK-22072][SPARK-22071][BUILD]Improve release ...

2017-09-21 Thread holdenk
GitHub user holdenk opened a pull request: https://github.com/apache/spark/pull/19312 [SPARK-22072][SPARK-22071][BUILD]Improve release build scripts ## What changes were proposed in this pull request? Check JDK version (with javac) and use SPARK_VERSION for publish-release

[GitHub] spark issue #19312: [SPARK-22072][SPARK-22071][BUILD]Improve release build s...

2017-09-21 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19312 cc @JoshRosen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark pull request #19294: [SPARK-21549][CORE] Respect OutputFormats with no...

2017-09-21 Thread mridulm
Github user mridulm commented on a diff in the pull request: https://github.com/apache/spark/pull/19294#discussion_r140342523 --- Diff: core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala --- @@ -57,6 +57,11 @@ class HadoopMapReduceCommitProtocol(j

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16578 **[Test build #82047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82047/testReport)** for PR 16578 at commit [`38cec5f`](https://github.com/apache/spark/commit/38

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread markhamstra
Github user markhamstra commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140340018 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r140338751 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala --- @@ -77,20 +77,21 @@ trait QueryPlanConstra

[GitHub] spark pull request #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/16578#discussion_r140338601 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/QueryPlanConstraints.scala --- @@ -77,20 +77,21 @@ trait QueryPlanConstra

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140337893 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #19308: [SPARK-22092] Reallocation in OffHeapColumnVector.reserv...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19308 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19308: [SPARK-22092] Reallocation in OffHeapColumnVector.reserv...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19308 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82038/ Test PASSed. ---

[GitHub] spark issue #19308: [SPARK-22092] Reallocation in OffHeapColumnVector.reserv...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19308 **[Test build #82038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82038/testReport)** for PR 19308 at commit [`8ce2a68`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-09-21 Thread pgandhi999
Github user pgandhi999 commented on the issue: https://github.com/apache/spark/pull/19270 I believe you mean the opposite of what you wrote. My changes are visible in the web ui(while the app is running) and not in the SHS(once the job is done). Yep I see that and am working on the fi

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-21 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r140336969 --- Diff: common/unsafe/src/main/java/org/apache/spark/sql/catalyst/expressions/HiveHasher.java --- @@ -38,6 +39,10 @@ public static int hashLong(long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-21 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r140336704 --- Diff: common/unsafe/src/main/java/org/apache/spark/sql/catalyst/expressions/HiveHasher.java --- @@ -38,6 +39,10 @@ public static int hashLong(long

[GitHub] spark pull request #19222: [SPARK-10399][CORE][SQL] Introduce multiple Memor...

2017-09-21 Thread tejasapatil
Github user tejasapatil commented on a diff in the pull request: https://github.com/apache/spark/pull/19222#discussion_r140336092 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/array/ByteArrayMethods.java --- @@ -46,6 +47,42 @@ public static int roundNumberOfBytesTo

[GitHub] spark issue #19311: [SPARK-22083][CORE] Release locks in MemoryStore.evictBl...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19311 **[Test build #82046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82046/testReport)** for PR 19311 at commit [`1ff270a`](https://github.com/apache/spark/commit/1f

[GitHub] spark pull request #19311: [SPARK-22083][CORE] Release locks in MemoryStore....

2017-09-21 Thread squito
GitHub user squito opened a pull request: https://github.com/apache/spark/pull/19311 [SPARK-22083][CORE] Release locks in MemoryStore.evictBlocksToFreeSpace ## What changes were proposed in this pull request? MemoryStore.evictBlocksToFreeSpace acquires write locks for all th

[GitHub] spark pull request #19232: [SPARK-22009][ML] Using treeAggregate improve som...

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19232 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19288: [SPARK-22075][ML] GBTs unpersist datasets cached ...

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19288 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16578 **[Test build #82045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82045/testReport)** for PR 16578 at commit [`2524a48`](https://github.com/apache/spark/commit/25

[GitHub] spark issue #19232: [SPARK-22009][ML] Using treeAggregate improve some algs

2017-09-21 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19232 I'm gonna merge this as a non-trivial win. I think the benefit outweighs concerns. We can add to the change later. --- - To unsub

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140332294 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19250 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82037/ Test PASSed. ---

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19250 **[Test build #82037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82037/testReport)** for PR 19250 at commit [`a869c6e`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #19305: [SPARK-22088][SQL] Incorrect scalastyle comment c...

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19305 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19305: [SPARK-22088][SQL] Incorrect scalastyle comment causes w...

2017-09-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19305 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19305: [SPARK-22088][SQL] Incorrect scalastyle comment causes w...

2017-09-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19305 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.ap

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19222 **[Test build #82044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82044/testReport)** for PR 19222 at commit [`4606ecf`](https://github.com/apache/spark/commit/46

[GitHub] spark pull request #19272: [Spark-21842][Mesos] Support Kerberos ticket rene...

2017-09-21 Thread kalvinnchau
Github user kalvinnchau commented on a diff in the pull request: https://github.com/apache/spark/pull/19272#discussion_r140326376 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCredentialRenewer.scala --- @@ -0,0 +1,150 @@ +/*

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82043/testReport)** for PR 18747 at commit [`3a6edc2`](https://github.com/apache/spark/commit/3a

[GitHub] spark issue #19310: [SPARK-18136] Fix SPARK_JARS_DIR for Python pip install ...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19310 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19310: [SPARK-18136] Fix SPARK_JARS_DIR for Python pip i...

2017-09-21 Thread jsnowacki
GitHub user jsnowacki opened a pull request: https://github.com/apache/spark/pull/19310 [SPARK-18136] Fix SPARK_JARS_DIR for Python pip install on Windows ## What changes were proposed in this pull request? Fix for setup of `SPARK_JARS_DIR` on Windows as it looks for `%SPAR

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18659 **[Test build #82042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82042/testReport)** for PR 18659 at commit [`53926cc`](https://github.com/apache/spark/commit/53

[GitHub] spark issue #18659: [SPARK-21190][PYSPARK][WIP] Python Vectorized UDFs

2017-09-21 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18659 Thanks @ueshin , that works to allow the tests to pass. I do worry that it might cause some other issues and I would much prefer we upgrade Arrow to handle this, but I'll push this and we can di

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140321640 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19308: [SPARK-22092] Reallocation in OffHeapColumnVector...

2017-09-21 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19308#discussion_r140320919 --- Diff: sql/core/src/main/java/org/apache/spark/sql/execution/vectorized/OffHeapColumnVector.java --- @@ -517,6 +517,7 @@ public void loadBytes(Column

[GitHub] spark issue #13143: [SPARK-15359] [Mesos] Mesos dispatcher should handle DRI...

2017-09-21 Thread ArtRand
Github user ArtRand commented on the issue: https://github.com/apache/spark/pull/13143 @skonto @susanxhuynh @devaraj-kavali are people still interested in this? I was just playing around with this code to clean up ZK state.. Would be happy to try this when I have a few cycles. ---

[GitHub] spark issue #19309: [SPARK-19558][sql] Add config key to register QueryExecu...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19309 **[Test build #82041 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82041/testReport)** for PR 19309 at commit [`c52fd71`](https://github.com/apache/spark/commit/c5

[GitHub] spark pull request #19309: [SPARK-19558][sql] Add config key to register Que...

2017-09-21 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/19309 [SPARK-19558][sql] Add config key to register QueryExecutionListeners automatically. This change adds a new SQL config key that is equivalent to SparkContext's "spark.extraListeners", allowing u

[GitHub] spark issue #19307: [SPARK-22087][SPARK-14650][WIP][BUILD][REPL][CORE] Compi...

2017-09-21 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19307 Good to see this passes, as I expected, because it means we can also commit this chunk of progress towards 2.12 without any downside for 2.11. Right now this means that the 2.12 build does c

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19204 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19204 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82040/ Test PASSed. ---

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19204 **[Test build #82040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82040/testReport)** for PR 19204 at commit [`8735c4c`](https://github.com/apache/spark/commit/8

[GitHub] spark pull request #19291: Branch 2.1

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19291 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19296: Branch 2.2

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19296 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19307: [SPARK-22087][SPARK-14650][WIP][BUILD][REPL][CORE] Compi...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19307 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82036/ Test PASSed. ---

[GitHub] spark issue #19307: [SPARK-22087][SPARK-14650][WIP][BUILD][REPL][CORE] Compi...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19307 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19307: [SPARK-22087][SPARK-14650][WIP][BUILD][REPL][CORE] Compi...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19307 **[Test build #82036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82036/testReport)** for PR 19307 at commit [`99f082a`](https://github.com/apache/spark/commit/9

[GitHub] spark pull request #19280: [SPARK-21928][CORE] Set classloader on Serializer...

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19280 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19280: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19280 Didn't merge to 2.1, please open a PR against that branch if you want it there. --- - To unsubscribe, e-mail: reviews-unsubscr...

[GitHub] spark issue #19280: [SPARK-21928][CORE] Set classloader on SerializerManager...

2017-09-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19280 LGTM, merging to master / 2.2 / 2.1. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #19204: [SPARK-21981][PYTHON][ML] Added Python interface for Clu...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19204 **[Test build #82040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82040/testReport)** for PR 19204 at commit [`8735c4c`](https://github.com/apache/spark/commit/87

[GitHub] spark issue #18704: [SPARK-20783][SQL] Create ColumnVector to abstract exist...

2017-09-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18704 @michal-databricks do you have any thoughts? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comm

[GitHub] spark issue #18994: [SPARK-21784][SQL] Adds support for defining information...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18994 **[Test build #82039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82039/testReport)** for PR 18994 at commit [`ea39601`](https://github.com/apache/spark/commit/ea

[GitHub] spark issue #18994: [SPARK-21784][SQL] Adds support for defining information...

2017-09-21 Thread sureshthalamati
Github user sureshthalamati commented on the issue: https://github.com/apache/spark/pull/18994 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark pull request #19306: [SPARK-21977][HOTFIX] Adjust EnsureStatefulOpPart...

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19306 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19306: [SPARK-21977][HOTFIX] Adjust EnsureStatefulOpPartitionin...

2017-09-21 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19306 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@

[GitHub] spark issue #19308: [SPARK-22092] Reallocation in OffHeapColumnVector.reserv...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19308 **[Test build #82038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82038/testReport)** for PR 19308 at commit [`8ce2a68`](https://github.com/apache/spark/commit/8c

[GitHub] spark issue #19308: [SPARK-22092] Reallocation in OffHeapColumnVector.reserv...

2017-09-21 Thread ala
Github user ala commented on the issue: https://github.com/apache/spark/pull/19308 @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.a

[GitHub] spark pull request #19308: [SPARK-22092] Reallocation in OffHeapColumnVector...

2017-09-21 Thread ala
GitHub user ala opened a pull request: https://github.com/apache/spark/pull/19308 [SPARK-22092] Reallocation in OffHeapColumnVector.reserveInternal corrupts array data ## What changes were proposed in this pull request? `OffHeapColumnVector.reserveInternal()` will only copy

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140296863 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #19290: [WIP][SPARK-22063][R] Upgrades lintr to latest commit sh...

2017-09-21 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19290 isn't it the other way around? https://github.com/jimhester/lintr/compare/v1.0.1?expand=1 --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19194: [SPARK-20589] Allow limiting task concurrency per...

2017-09-21 Thread dhruve
Github user dhruve commented on a diff in the pull request: https://github.com/apache/spark/pull/19194#discussion_r140293577 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -619,6 +625,47 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #19306: [SPARK-21977][HOTFIX] Adjust EnsureStatefulOpPartitionin...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19306 **[Test build #3928 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3928/testReport)** for PR 19306 at commit [`af017f0`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #17902: [SPARK-20641][core] Add key-value store abstracti...

2017-09-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/17902#discussion_r140290617 --- Diff: common/kvstore/src/main/java/org/apache/spark/kvstore/KVStoreSerializer.java --- @@ -0,0 +1,86 @@ +/* + * Licensed to the Apache Software F

[GitHub] spark issue #19305: [SPARK-22088][SQL] Incorrect scalastyle comment causes w...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19305 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82035/ Test PASSed. ---

[GitHub] spark issue #19305: [SPARK-22088][SQL] Incorrect scalastyle comment causes w...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19305 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19305: [SPARK-22088][SQL] Incorrect scalastyle comment causes w...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19305 **[Test build #82035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82035/testReport)** for PR 19305 at commit [`79b135a`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #19290: [WIP][SPARK-22063][R] Upgrades lintr to latest commit sh...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19290 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82034/ Test FAILed. ---

[GitHub] spark issue #19290: [WIP][SPARK-22063][R] Upgrades lintr to latest commit sh...

2017-09-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19290 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19290: [WIP][SPARK-22063][R] Upgrades lintr to latest commit sh...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19290 **[Test build #82034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82034/testReport)** for PR 19290 at commit [`49a166d`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #19250: [SPARK-12297] Table timezone correction for Timestamps

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19250 **[Test build #82037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82037/testReport)** for PR 19250 at commit [`a869c6e`](https://github.com/apache/spark/commit/a8

[GitHub] spark issue #19306: [SPARK-21977][HOTFIX] Adjust EnsureStatefulOpPartitionin...

2017-09-21 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/19306 Thanks @srowen . LGTM! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #19234: [SPARK-22010][PySpark] Change fromInternal method of Tim...

2017-09-21 Thread maver1ck
Github user maver1ck commented on the issue: https://github.com/apache/spark/pull/19234 It was introduced with this PEP. https://www.python.org/dev/peps/pep-0495/ --- - To unsubscribe, e-mail: reviews-unsubscr...

[GitHub] spark issue #19307: [SPARK-22087][SPARK-14650][WIP][BUILD][REPL][CORE] Compi...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19307 **[Test build #82036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82036/testReport)** for PR 19307 at commit [`99f082a`](https://github.com/apache/spark/commit/99

[GitHub] spark issue #19145: [spark-21933][yarn] Spark Streaming request more executo...

2017-09-21 Thread klion26
Github user klion26 commented on the issue: https://github.com/apache/spark/pull/19145 @jerryshao thank you for your comment, I will try to find how MR/TEZ handle this --- - To unsubscribe, e-mail: reviews-unsubscr.

[GitHub] spark issue #19307: [SPARK-22087][SPARK-14650][WIP][BUILD][REPL][CORE] Compi...

2017-09-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19307 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark pull request #18924: [SPARK-14371] [MLLIB] OnlineLDAOptimizer should n...

2017-09-21 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18924#discussion_r140249325 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAOptimizer.scala --- @@ -503,17 +518,15 @@ final class OnlineLDAOptimizer extends LD

[GitHub] spark issue #19290: [WIP][SPARK-22063][R] Upgrades lintr to latest commit sh...

2017-09-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19290 To @shivaram: > does that also happen with lintr-1.0.1 ? I wonder if we are seeing some specific performance slowdown because of not using a release (it might be good to follow up with

[GitHub] spark pull request #19269: [SPARK-22026][SQL][WIP] data source v2 write path

2017-09-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19269#discussion_r140248789 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apac

[GitHub] spark pull request #19269: [SPARK-22026][SQL][WIP] data source v2 write path

2017-09-21 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19269#discussion_r140248731 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceV2Writer.java --- @@ -0,0 +1,71 @@ +/* + * Licensed to the Apac

[GitHub] spark issue #19304: [SPARK-22086][DOCS] Add expression description for CASE ...

2017-09-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19304 Thanks @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #19304: [SPARK-22086][DOCS] Add expression description fo...

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19304 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19304: [SPARK-22086][DOCS] Add expression description for CASE ...

2017-09-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19304 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #15544: [SPARK-17997] [SQL] Add an aggregation function f...

2017-09-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15544 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #15544: [SPARK-17997] [SQL] Add an aggregation function for coun...

2017-09-21 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/15544 LGTM, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark issue #19306: [SPARK-21977][HOTFIX] Adjust EnsureStatefulOpPartitionin...

2017-09-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19306 **[Test build #3928 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3928/testReport)** for PR 19306 at commit [`af017f0`](https://github.com/apache/spark/commit/a

<    1   2   3   4   5   >