[GitHub] spark issue #22516: [SPARK-25468]Highlight current page index in the spark u...

2018-09-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22516 I think the change is fine, if it works and if it maybe causes the same behavior in the whole UI. --- - To unsubscribe, e-mail

[GitHub] spark issue #22520: [SPARK-25509][Core]Windows doesn't support POSIX permiss...

2018-09-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22520 I wouldn't worry unless it's been several days. Depending on urgency and impact, I leave some time before merging to make sure there aren't more comments

[GitHub] spark pull request #22549: [SPARK-25533][CORE][WEBUI]AppSummary should hold ...

2018-09-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22549#discussion_r220383364 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -388,10 +388,11 @@ private[spark] class AppStatusListener

[GitHub] spark issue #22520: [SPARK-25509][Core]Windows doesn't support POSIX permiss...

2018-09-25 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22520 Yeah, I like your suggestion to just have one code path here that works for all platforms. No need to have used `PosixFilePermissions` in the first place as there was a utility method for setting

[GitHub] spark pull request #22453: [SPARK-20937][DOCS] Describe spark.sql.parquet.wr...

2018-09-25 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22453#discussion_r220200276 --- Diff: docs/sql-programming-guide.md --- @@ -1002,6 +1002,21 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark issue #22466: [SPARK-25464][SQL]On dropping the Database it will drop ...

2018-09-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22466 Owp, I've been misreading that several times. Right. Well by analogy, if a database has a non default LOCATION then so do it's tables, and they are treated like EXTERNAL tables. Dropping the DB

[GitHub] spark issue #22466: [SPARK-25464][SQL]On dropping the Database it will drop ...

2018-09-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22466 That link says Hive does support EXTERNAL. What am I missing? Well, in any event we aren't contemplating a behavior change here. If you delete a table with LOCATION specified, what should

[GitHub] spark pull request #22473: [SPARK-25449][CORE] Heartbeat shouldn't include a...

2018-09-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22473#discussion_r219941854 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -83,6 +83,17 @@ package object config { private[spark] val

[GitHub] spark pull request #21816: [SPARK-24794][CORE] Driver launched through rest ...

2018-09-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21816#discussion_r219940432 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/StandaloneRestServer.scala --- @@ -138,6 +138,10 @@ private[rest] class

[GitHub] spark pull request #21816: [SPARK-24794][CORE] Driver launched through rest ...

2018-09-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/21816#discussion_r219940865 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/StandaloneRestServer.scala --- @@ -138,6 +138,10 @@ private[rest] class

[GitHub] spark pull request #22534: [SPARK-25514][SQL] Pretty JSON

2018-09-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22534#discussion_r219901199 --- Diff: python/pyspark/sql/readwriter.py --- @@ -776,7 +776,7 @@ def saveAsTable(self, name, format=None, mode=None, partitionBy=None, **options

[GitHub] spark issue #22061: [SPARK-25079][PYTHON] preparing for python 3.5 bump

2018-09-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22061 @shaneknapp should this go in now? replacing 3.4? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #21858: [SPARK-24899][SQL][DOC] Add example of monotonically_inc...

2018-09-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21858 @jaceklaskowski would you like to update this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22526: [SPARK-25502][CORE][WEBUI]Empty Page when page nu...

2018-09-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22526#discussion_r219869516 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -685,7 +685,10 @@ private[ui] class TaskDataSource( private var

[GitHub] spark issue #22466: [SPARK-25464][SQL]On dropping the Database it will drop ...

2018-09-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22466 There is ... see https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-ManagedandExternalTables I think Spark conflates the two. It's rare (?) but possible

[GitHub] spark issue #22466: [SPARK-25464][SQL]On dropping the Database it will drop ...

2018-09-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22466 We should look at Spark documentation, and Hive, if any, to figure out what the right behavior is here. Spark generally follows Hive. See https://cwiki.apache.org/confluence/display/Hive

[GitHub] spark pull request #22526: [SPARK-25502][CORE][WEBUI]Empty Page when page nu...

2018-09-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22526#discussion_r219827419 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -685,7 +685,15 @@ private[ui] class TaskDataSource( private var

[GitHub] spark pull request #22453: [SPARK-20937][DOCS] Describe spark.sql.parquet.wr...

2018-09-24 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22453#discussion_r219827092 --- Diff: docs/sql-programming-guide.md --- @@ -1002,6 +1002,21 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark issue #21524: [SPARK-24212][ML][doc] Add the example and user guide fo...

2018-09-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21524 @tengpeng there's no deadline, but 2.4 is being released imminently and doc/example improvements can go in this week, probably. If it's going to be a while before you can update, we generally close

[GitHub] spark pull request #22453: [SPARK-20937][DOCS] Describe spark.sql.parquet.wr...

2018-09-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22453#discussion_r219722694 --- Diff: docs/sql-programming-guide.md --- @@ -1002,6 +1002,15 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark pull request #22453: [SPARK-20937][DOCS] Describe spark.sql.parquet.wr...

2018-09-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22453#discussion_r219717918 --- Diff: docs/sql-programming-guide.md --- @@ -1002,6 +1002,15 @@ Configuration of Parquet can be done using the `setConf` method on `SparkSession

[GitHub] spark issue #22463: remove annotation @Experimental

2018-09-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22463 Yeah, I think Experimental is over-used in the APIs. They just never get un-marked and lots of pretty old stuff, that de facto is just not changeable now, is still labeled this way. This seems

[GitHub] spark pull request #22473: [SPARK-25449][CORE] Heartbeat shouldn't include a...

2018-09-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22473#discussion_r219717779 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -83,6 +83,17 @@ package object config { private[spark] val

[GitHub] spark pull request #22520: [SPARK-25509][Core]Windows doesn't support POSIX ...

2018-09-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22520#discussion_r219717720 --- Diff: core/src/main/scala/org/apache/spark/deploy/history/FsHistoryProvider.scala --- @@ -133,9 +133,15 @@ private[history] class FsHistoryProvider(conf

[GitHub] spark issue #22516: [SPARK-25468]Highlight current page index in the history...

2018-09-23 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22516 Rather than modify jquery, can we override this in Spark-specific CSS? otherwise we might lose customizations when updating jquery

[GitHub] spark pull request #22526: [SPARK-25502][WEBUI]Empty Page when page number e...

2018-09-23 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22526#discussion_r219715489 --- Diff: core/src/main/scala/org/apache/spark/ui/jobs/StagePage.scala --- @@ -685,7 +685,15 @@ private[ui] class TaskDataSource( private var

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2018-09-22 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/13440 Yeah I take your point that the trait Impurity already defines two methods, only one of which is implemented for each of the subclasses. It's already a funky design that probably should have been

[GitHub] spark issue #22466: [SPARK-25464][SQL]On dropping the Database it will drop ...

2018-09-21 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22466 See JIRA, I don't think this should be merged. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-20 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22399 I think this is fine (though see https://github.com/apache/spark/pull/22399/files#r218121426 ). As a practical matter I might wait until we're done with the 2.4 release to start a lot of changes

[GitHub] spark issue #21524: [SPARK-24212][ML][doc] Add the example and user guide fo...

2018-09-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21524 @tengpeng would you like to update this? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22454: [SPARK-25445][BUILD][FOLLOWUP] Resolve issues in ...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22454#discussion_r218567663 --- Diff: dev/create-release/release-build.sh --- @@ -446,6 +432,8 @@ if [[ "$1" == "publish-release" ]]; then # Clean-up

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218486554 --- Diff: dev/create-release/release-build.sh --- @@ -111,13 +111,17 @@ fi # different versions of Scala are supported. BASE_PROFILES="-P

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22446 Merged to master/2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22446 Yeah, there's no 'latest' link though. The JDK is pretty stable so a link to Java 8 when we're on Java 9 isn't going to be an issue

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218476955 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218476788 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218479034 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/TestYarnResourceTypeHelper.scala --- @@ -0,0 +1,103

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218479200 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/TestYarnResourceTypeHelper.scala --- @@ -0,0 +1,103

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218478615 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/TestYarnResourceTypeHelper.scala --- @@ -0,0 +1,103

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218479401 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/TestYarnResourceTypeHelper.scala --- @@ -0,0 +1,103

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218478292 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceTypeValidatorSuite.scala --- @@ -0,0 +1,150

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218477062 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocator.scala --- @@ -140,10 +140,19 @@ private[yarn] class

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218477315 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ClientSuite.scala --- @@ -35,18 +36,22 @@ import

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218478229 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceTypeValidatorSuite.scala --- @@ -0,0 +1,150

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r218477920 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/ResourceTypeHelperSuite.scala --- @@ -0,0 +1,152 @@ +/* + * Licensed

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218471225 --- Diff: dev/create-release/release-build.sh --- @@ -414,15 +437,15 @@ if [[ "$1" == "publish-release" ]]; then -DskipTes

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218470455 --- Diff: dev/create-release/release-build.sh --- @@ -111,13 +111,17 @@ fi # different versions of Scala are supported. BASE_PROFILES="-P

[GitHub] spark issue #22381: [SPARK-25394][CORE] Add an application status metrics so...

2018-09-18 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22381 Looks like different failures. I am not sure if these are known issues. I'd try again later today. --- - To unsubscribe, e-mail

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-18 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218458224 --- Diff: dev/create-release/release-build.sh --- @@ -414,15 +437,15 @@ if [[ "$1" == "publish-release" ]]; then -DskipTes

[GitHub] spark issue #22446: [SPARK-19550][DOC][FOLLOW-UP] Update tuning.md to use JD...

2018-09-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22446 Are these all the references to Java 7 docs? Also, does the advice need a bit of updating since we require Java 8? around GC

[GitHub] spark pull request #13440: [SPARK-15699] [ML] Implement a Chi-Squared test s...

2018-09-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/13440#discussion_r218208245 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/ChiSquared.scala --- @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #13440: [SPARK-15699] [ML] Implement a Chi-Squared test s...

2018-09-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/13440#discussion_r218209825 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Gini.scala --- @@ -71,6 +71,23 @@ object Gini extends Impurity { @Since("

[GitHub] spark pull request #13440: [SPARK-15699] [ML] Implement a Chi-Squared test s...

2018-09-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/13440#discussion_r218209381 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Impurity.scala --- @@ -52,6 +52,49 @@ trait Impurity extends Serializable

[GitHub] spark pull request #13440: [SPARK-15699] [ML] Implement a Chi-Squared test s...

2018-09-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/13440#discussion_r218208383 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/ChiSquared.scala --- @@ -0,0 +1,162 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #13440: [SPARK-15699] [ML] Implement a Chi-Squared test s...

2018-09-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/13440#discussion_r218209453 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/tree/impurity/Impurity.scala --- @@ -52,6 +52,49 @@ trait Impurity extends Serializable

[GitHub] spark pull request #13440: [SPARK-15699] [ML] Implement a Chi-Squared test s...

2018-09-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/13440#discussion_r218208123 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/impl/RandomForest.scala --- @@ -670,14 +670,32 @@ private[spark] object RandomForest extends Logging

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218079177 --- Diff: dev/create-release/release-build.sh --- @@ -112,11 +112,16 @@ fi # different versions of Scala are supported. BASE_PROFILES="-P

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218078222 --- Diff: dev/create-release/release-build.sh --- @@ -411,15 +435,15 @@ if [[ "$1" == "publish-release" ]]; then -DskipTes

[GitHub] spark pull request #22441: [SPARK-25445][BUILD] the release script should be...

2018-09-17 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22441#discussion_r218078496 --- Diff: dev/create-release/release-build.sh --- @@ -183,8 +188,17 @@ if [[ "$1" == "package" ]]; then # Updated f

[GitHub] spark issue #22436: [SPARK-24654][BUILD][FOLLOWUP] Update, fix LICENSE and N...

2018-09-17 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22436 Merged to master/2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22436: [SPARK-24654][BUILD][FOLLOWUP] Update, fix LICENSE and N...

2018-09-16 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22436 CC @cloud-fan This one doesn't block 2.4.0 but would be nice to have. Certainly if there's a second RC. --- - To unsubscribe, e

[GitHub] spark pull request #22436: [SPARK-24654][BUILD][FOLLOWUP] Update, fix LICENS...

2018-09-16 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/22436 [SPARK-24654][BUILD][FOLLOWUP] Update, fix LICENSE and NOTICE, and specialize for source vs binary ## What changes were proposed in this pull request? Fix location of licenses-binary

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-16 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22231 Yeah I noticed that. I think we should leave it, and, if somehow RC1 passes, we'll mark this as fixed for a later release

[GitHub] spark pull request #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in...

2018-09-16 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22396#discussion_r217926918 --- Diff: docs/sql-programming-guide.md --- @@ -1897,7 +1897,8 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark pull request #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in...

2018-09-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22396#discussion_r217897208 --- Diff: docs/sql-programming-guide.md --- @@ -1898,6 +1898,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see - Since

[GitHub] spark pull request #22425: [SPARK-23367][Build] Include python document styl...

2018-09-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22425#discussion_r217895197 --- Diff: dev/tox.ini --- @@ -14,6 +14,8 @@ # limitations under the License. [pycodestyle] -ignore=E226,E241,E305,E402,E722,E731,E741

[GitHub] spark pull request #22425: [SPARK-23367][Build] Include python document styl...

2018-09-15 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22425#discussion_r217895205 --- Diff: dev/tox.ini --- @@ -14,6 +14,8 @@ # limitations under the License. [pycodestyle] -ignore=E226,E241,E305,E402,E722,E731,E741

[GitHub] spark issue #19538: [SPARK-20393][WEBU UI][BACKPORT-2.0] Strengthen Spark to...

2018-09-15 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19538 @ambauma could you close it? we can't, directly --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-14 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22231 Merged to master/2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in...

2018-09-14 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22396#discussion_r217722539 --- Diff: docs/sql-programming-guide.md --- @@ -1898,6 +1898,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see - Since

[GitHub] spark pull request #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycode...

2018-09-14 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22231#discussion_r217719711 --- Diff: dev/run-tests-jenkins.py --- @@ -131,7 +131,7 @@ def run_tests(tests_timeout): ERROR_CODES["BLOCK_PYSPARK_UNIT_TESTS"]: 'Py

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22231 Oh, that line isn't the error, although the warning says it is! It's really line 134, which actually has escaped back-ticks. I think you can honestly remove all of these back-ticks

[GitHub] spark issue #22385: [SPARK-25400][CORE][TEST] Increase test timeouts

2018-09-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22385 Merged to master/2.4/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycode...

2018-09-13 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22231#discussion_r217428438 --- Diff: dev/run-tests-jenkins.py --- @@ -116,7 +116,7 @@ def run_tests(tests_timeout): failure_note_by_errcode = { # error

[GitHub] spark issue #22397: [SPARK-25170][DOC] Add list and short description of Spa...

2018-09-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22397 Merged to master/2.4 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22231: [SPARK-25238][PYTHON] lint-python: Upgrade pycodestyle t...

2018-09-13 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22231 Hm, I don't think the `# noqa` worked. That's the same failure. I thought it passed locally, hm. Well, I think that line should remove the noqa and remove the back-ticks. They aren't actually

[GitHub] spark pull request #22396: [SPARK-23425][SQL][FOLLOWUP] Support wildcards in...

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22396#discussion_r217209344 --- Diff: docs/sql-programming-guide.md --- @@ -1898,6 +1898,7 @@ working with timestamps in `pandas_udf`s to get the best performance, see - Since

[GitHub] spark issue #22381: [SPARK-25394][CORE] Add an application status metrics so...

2018-09-12 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22381 Yeah, this is bigger than adding one metric; raise it on dev@? I'm not sure who if anyone has an opinion on it. I'd rather hear more feedback on the history of these choices and what the right

[GitHub] spark pull request #22404: DO NOT MERGE

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22404#discussion_r217154410 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala --- @@ -692,5 +681,238 @@ private[spark] class PythonBroadcast(@transient var

[GitHub] spark pull request #22404: DO NOT MERGE

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22404#discussion_r217154988 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -193,19 +193,51 @@ private[spark] abstract class BasePythonRunner

[GitHub] spark pull request #22404: DO NOT MERGE

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22404#discussion_r217153676 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/SimpleDownloadFile.java --- @@ -0,0 +1,89 @@ +/* + * Licensed

[GitHub] spark pull request #22404: DO NOT MERGE

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22404#discussion_r217153478 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/SimpleDownloadFile.java --- @@ -0,0 +1,89 @@ +/* + * Licensed

[GitHub] spark pull request #22404: DO NOT MERGE

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22404#discussion_r217153096 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/DownloadFile.java --- @@ -0,0 +1,45 @@ +/* + * Licensed

[GitHub] spark pull request #22404: DO NOT MERGE

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22404#discussion_r217153301 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/DownloadFile.java --- @@ -0,0 +1,45 @@ +/* + * Licensed

[GitHub] spark pull request #22404: DO NOT MERGE

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22404#discussion_r217153554 --- Diff: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/SimpleDownloadFile.java --- @@ -0,0 +1,89 @@ +/* + * Licensed

[GitHub] spark pull request #22404: DO NOT MERGE

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22404#discussion_r217155746 --- Diff: python/pyspark/context.py --- @@ -499,19 +506,32 @@ def f(split, iterator): def _serialize_to_jvm(self, data, parallelism

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r217099139 --- Diff: dev/run-tests-jenkins.py --- @@ -115,7 +115,7 @@ def run_tests(tests_timeout): os.path.join

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r217100731 --- Diff: dev/create-release/generate-contributors.py --- @@ -88,7 +88,7 @@ def print_indented(_list): def is_release(commit_title

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r217114473 --- Diff: dev/merge_spark_pr.py --- @@ -403,7 +403,7 @@ def standardize_jira_ref(text): # Extract spark component(s): # Look

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r217114250 --- Diff: dev/create-release/generate-contributors.py --- @@ -67,7 +67,7 @@ print("Release tag: %s" % RELEASE_TAG) print("Previous

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-12 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r217036427 --- Diff: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/ExternalShuffleIntegrationSuite.java --- @@ -133,37 +133,37 @@ private

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r216886496 --- Diff: python/pyspark/sql/functions.py --- @@ -283,7 +283,8 @@ def approxCountDistinct(col, rsd=None): @since(2.1) def

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r216886472 --- Diff: python/pyspark/ml/feature.py --- @@ -303,7 +303,7 @@ def _create_model(self, java_model): class BucketedRandomProjectionLSHModel

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22400#discussion_r216886428 --- Diff: dev/run-tests-jenkins.py --- @@ -115,7 +115,7 @@ def run_tests(tests_timeout): os.path.join

[GitHub] spark pull request #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warni...

2018-09-11 Thread srowen
GitHub user srowen opened a pull request: https://github.com/apache/spark/pull/22400 [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for pycodestyle 2.4 (This change is a subset of the changes needed for the JIRA; see https://github.com/apache/spark/pull/22231

[GitHub] spark issue #22400: [SPARK-25238][PYTHON] lint-python: Fix W605 warnings for...

2018-09-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22400 CC @cclauss @holdenk --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #22393: Axe deprecated doc refs

2018-09-11 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22393 CC @felixcheung I'd also prefix the title with `[MINOR][DOCS]` to match our conventions --- - To unsubscribe, e-mail

[GitHub] spark pull request #22398: [SPARK-23820][CORE] Enable use of long form of ca...

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22398#discussion_r216882861 --- Diff: docs/configuration.md --- @@ -746,6 +746,13 @@ Apart from these, the following properties are also available, and may be useful *Warning

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216882375 --- Diff: launcher/src/main/java/org/apache/spark/launcher/AbstractAppHandle.java --- @@ -72,11 +74,7 @@ public void stop() { @Override public

[GitHub] spark pull request #22399: [SPARK-25408] Move to mode ideomatic Java8

2018-09-11 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22399#discussion_r216881904 --- Diff: common/network-common/src/test/java/org/apache/spark/network/ChunkFetchIntegrationSuite.java --- @@ -143,61 +143,62 @@ public void releaseBuffers

<    4   5   6   7   8   9   10   11   12   13   >