[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-30 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164824439 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -594,12 +606,24 @@ private[spark] class AppStatusListener(

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20435 **[Test build #86840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86840/testReport)** for PR 20435 at commit

[GitHub] spark issue #20438: [SPARK-23272][SQL] add calendar interval type support to...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20438 **[Test build #86825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86825/testReport)** for PR 20438 at commit

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/20386 @cloud-fan, is the intent to get this into 2.3.0? If so, I'll make time to review it today. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20386 **[Test build #86829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86829/testReport)** for PR 20386 at commit

[GitHub] spark issue #20428: [SPARK-23261] [PySpark] Rename Pandas UDFs

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20428 Will submit a new PR to 2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20167: [SPARK-16501] [MESOS] Allow providing Mesos princ...

2018-01-30 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/20167#discussion_r164825718 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -71,40 +74,64 @@ trait

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/397/

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20435 **[Test build #86839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86839/testReport)** for PR 20435 at commit

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/20435 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20343: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQueryS...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20343 @maropu Yeah. As long as the queries are different, we should keep both versions. This is to help the others understand we fully support TPC-DS queries without the changes. Thanks! ---

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20295 **[Test build #86836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86836/testReport)** for PR 20295 at commit

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86822/ Test PASSed. ---

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164806777 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -236,9 +238,29 @@ public MapData getMap(int ordinal) {

[GitHub] spark pull request #20427: [SPARK-23260][SPARK-23262][SQL] several data sour...

2018-01-30 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20427#discussion_r164806676 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/SessionConfigSupport.java --- @@ -25,7 +25,7 @@ * session. */

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20386 @rdblue The target is 2.3 release. Thanks for your time! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20435 cc @jose-torres --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86829/ Test PASSed. ---

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20435 **[Test build #86832 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86832/testReport)** for PR 20435 at commit

[GitHub] spark issue #20436: [MINOR] Fix typos in dev/* scripts.

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20436 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86828/ Test PASSed. ---

[GitHub] spark issue #20436: [MINOR] Fix typos in dev/* scripts.

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20436 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20435 **[Test build #86838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86838/testReport)** for PR 20435 at commit

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20435 cc @zsxwing @marmbrus too --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/20435 cc @marmbrus --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20435 LGTM to adding the new package of partitioning/distribution. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20422: [SPARK-23253][Core][Shuffle]Only write shuffle temporary...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20422 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86831/ Test PASSed. ---

[GitHub] spark issue #20422: [SPARK-23253][Core][Shuffle]Only write shuffle temporary...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20422 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20434: [SPARK-23267] [SQL] Increase spark.sql.codegen.hu...

2018-01-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20434#discussion_r164845127 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -660,12 +660,13 @@ object SQLConf { val

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20386 **[Test build #86822 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86822/testReport)** for PR 20386 at commit

[GitHub] spark pull request #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer....

2018-01-30 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20386#discussion_r164805751 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/sources/ConsoleWriterSuite.scala --- @@ -34,9 +33,9 @@ class

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20295 **[Test build #86837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86837/testReport)** for PR 20295 at commit

[GitHub] spark pull request #20427: [SPARK-23260][SPARK-23262][SQL] several data sour...

2018-01-30 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/20427#discussion_r164807449 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/SessionConfigSupport.java --- @@ -25,7 +25,7 @@ * session. */

[GitHub] spark pull request #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer....

2018-01-30 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20386#discussion_r164810169 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceWriter.java --- @@ -63,32 +68,42 @@ DataWriterFactory

[GitHub] spark issue #20436: [MINOR] Fix typos in dev/* scripts.

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20436 **[Test build #86828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86828/testReport)** for PR 20436 at commit

[GitHub] spark issue #20428: [SPARK-23261] [PySpark] Rename Pandas UDFs

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20428 Let me manually push it to 2.3. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20295 **[Test build #86836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86836/testReport)** for PR 20295 at commit

[GitHub] spark issue #20438: [SPARK-23272][SQL] add calendar interval type support to...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20438 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20438: [SPARK-23272][SQL] add calendar interval type support to...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20438 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86825/ Test PASSed. ---

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164802008 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -236,9 +238,29 @@ public MapData getMap(int ordinal) {

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164807808 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -236,9 +238,29 @@ public MapData getMap(int ordinal) {

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20435 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/398/

[GitHub] spark issue #20343: [SPARK-23167][SQL] Add TPCDS queries v2.7 in TPCDSQueryS...

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20343 Thanks for submitting the PR https://github.com/apache/spark/pull/20433. It sounds like there are still some test failure. Will review it after 2.3 release. Thanks! ---

[GitHub] spark issue #20422: [SPARK-23253][Core][Shuffle]Only write shuffle temporary...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20422 **[Test build #86831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86831/testReport)** for PR 20422 at commit

[GitHub] spark issue #20434: [SPARK-23267] [SQL] Increase spark.sql.codegen.hugeMetho...

2018-01-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20434 @gatorsmile . In the original PR, https://github.com/apache/spark/pull/18810, there was a microbenchmark. Can we have the result on the same benchmark here, too? ---

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/395/

[GitHub] spark issue #20295: [SPARK-23011] Support alternative function form with gro...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20295 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20428: [SPARK-23261] [PySpark] Rename Pandas UDFs

2018-01-30 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20428 Yes. We need to backport it to 2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20435 Streaming part LGTM; I have no particular opinion or context on the distribution stuff. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164717772 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -195,6 +196,7 @@ * struct field. */ public

[GitHub] spark issue #20434: [SPARK-23267] [SQL] Increase spark.sql.codegen.hugeMetho...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20434 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86813/ Test PASSed. ---

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86809/ Test PASSed. ---

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20386 **[Test build #86809 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86809/testReport)** for PR 20386 at commit

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20428: [SPARK-23261] [PySpark] Rename Pandas UDFs

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20428 **[Test build #86804 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86804/testReport)** for PR 20428 at commit

[GitHub] spark issue #20428: [SPARK-23261] [PySpark] Rename Pandas UDFs

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20428 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86804/ Test PASSed. ---

[GitHub] spark issue #20428: [SPARK-23261] [PySpark] Rename Pandas UDFs

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20428 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/383/

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20386 **[Test build #86822 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86822/testReport)** for PR 20386 at commit

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20404 Yup, sorry for rushing the suggestion. Wanted to show an idea. BTW, if we should replace it if it's already stopped, I assume we need the synchronization with `defaultSession` anyway if I

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20386 **[Test build #86823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86823/testReport)** for PR 20386 at commit

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer.commit ...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20386 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/384/

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164729250 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -235,10 +237,30 @@ public MapData getMap(int ordinal) {

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164729429 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -235,10 +237,30 @@ public MapData getMap(int ordinal) {

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164729684 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -235,10 +237,30 @@ public MapData getMap(int ordinal) {

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164728359 --- Diff: common/unsafe/src/main/java/org/apache/spark/unsafe/types/CalendarInterval.java --- @@ -87,7 +87,7 @@ public static CalendarInterval

[GitHub] spark issue #20404: [SPARK-23228][PYSPARK] Add Python Created jsparkSession ...

2018-01-30 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20404 Thanks all for your comments. I think @felixcheung 's case really makes thing complex, I'm not sure if user will use it in such way. I will try to address it. Appreciate your comments! ---

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread jaceklaskowski
Github user jaceklaskowski commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164729086 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnVector.java --- @@ -235,10 +237,30 @@ public MapData getMap(int ordinal) {

[GitHub] spark pull request #20438: [SPARK-23272][SQL] add calendar interval type sup...

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20438#discussion_r164730300 --- Diff: sql/core/src/main/java/org/apache/spark/sql/vectorized/ColumnarArray.java --- @@ -135,9 +135,7 @@ public UTF8String getUTF8String(int ordinal)

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20435 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20435 **[Test build #86824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86824/testReport)** for PR 20435 at commit

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20442 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86848/ Test PASSed. ---

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20442 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20442: [SPARK-23265][SQL]Update multi-column error handling log...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20442 **[Test build #86848 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86848/testReport)** for PR 20442 at commit

[GitHub] spark issue #20441: [SPARK-23275][SQL] hive/tests have been failing when run...

2018-01-30 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20441 For the record, what was the error? I got this running on OS X from master with `-Phive` before this change, but was that same problem? ``` HiveWindowFunctionQuerySuite: *** RUN

[GitHub] spark pull request #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer....

2018-01-30 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20386#discussion_r164919092 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/writer/DataSourceWriter.java --- @@ -63,32 +68,42 @@ DataWriterFactory

[GitHub] spark issue #20428: [SPARK-23261] [PySpark] Rename Pandas UDFs

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20428 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20435: [SPARK-23268][SQL]Reorganize packages in data source V2

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20435 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/385/

[GitHub] spark pull request #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer....

2018-01-30 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20386#discussion_r164737921 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/sources/ConsoleWriterSuite.scala --- @@ -34,9 +33,9 @@ class

[GitHub] spark pull request #20436: [MINOR] Fix typos in dev/* scripts.

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20436#discussion_r164758616 --- Diff: dev/lint-python --- @@ -60,9 +60,9 @@ export "PYLINT_HOME=$PYTHONPATH" export "PATH=$PYTHONPATH:$PATH" # There is no need to

[GitHub] spark pull request #20408: [SPARK-23189][Core][Web UI] Reflect stage level b...

2018-01-30 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/20408#discussion_r164766057 --- Diff: core/src/main/resources/org/apache/spark/ui/static/executorspage.js --- @@ -416,8 +422,7 @@ $(document).ready(function () {

[GitHub] spark pull request #20386: [SPARK-23202][SQL] Break down DataSourceV2Writer....

2018-01-30 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20386#discussion_r164918470 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/streaming/writer/StreamWriter.java --- @@ -32,40 +32,44 @@

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20226 **[Test build #86850 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86850/testReport)** for PR 20226 at commit

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/406/

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20378: [SPARK-11222][Build][Python] Python document styl...

2018-01-30 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/20378#discussion_r164920796 --- Diff: dev/lint-python --- @@ -83,6 +84,53 @@ else rm "$PEP8_REPORT_PATH" fi + Python Document Style Checks +

[GitHub] spark issue #20415: [SPARK-23247][SQL]combines Unsafe operations and statist...

2018-01-30 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/20415 @cloud-fan Can you help me to review it. thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20444: [SPARK-23274] [SQL] Fix ReplaceExceptWithFilter w...

2018-01-30 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/20444 [SPARK-23274] [SQL] Fix ReplaceExceptWithFilter when the right's Filter contains the references that are not in the left output ## What changes were proposed in this pull request? This PR

[GitHub] spark issue #20441: [SPARK-23275][SQL] hive/tests have been failing when run...

2018-01-30 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/20441 @srowen Hello, yeah, i saw the same error. Quite a few errors like ``` java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

[GitHub] spark pull request #20378: [SPARK-11222][Build][Python] Python document styl...

2018-01-30 Thread rekhajoshm
Github user rekhajoshm commented on a diff in the pull request: https://github.com/apache/spark/pull/20378#discussion_r164926603 --- Diff: dev/lint-python --- @@ -83,6 +84,53 @@ else rm "$PEP8_REPORT_PATH" fi + Python Document Style Checks +

[GitHub] spark issue #20444: [SPARK-23274] [SQL] Fix ReplaceExceptWithFilter when the...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20444 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20444: [SPARK-23274] [SQL] Fix ReplaceExceptWithFilter when the...

2018-01-30 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20444 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/407/

[GitHub] spark issue #20444: [SPARK-23274] [SQL] Fix ReplaceExceptWithFilter when the...

2018-01-30 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20444 **[Test build #86851 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86851/testReport)** for PR 20444 at commit

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-30 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20378 I looked at the example output to see see what the errors were. Specifically looking at `broadcast.__init__` ``` def __init__(self, sc=None, value=None, pickle_registry=None,

<    1   2   3   4   5   6   7   >