[GitHub] spark pull request #19452: [SPARK-22136][SS] Evaluate one-sided conditions e...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19452#discussion_r144435539 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinExec.scala --- @@ -221,43 +237,36 @@ case class Streaming

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/19467 Whoops, i missed one comment. Not LGTM. Need tests in PlannerSuite that tests whether EnsureRequirements respects numPartitions in ClusteredDistribution. ---

[GitHub] spark pull request #19467: [SPARK-22238] Fix plan resolution bug caused by E...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19467#discussion_r144403807 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/EnsureRequirements.scala --- @@ -50,7 +50,8 @@ case class EnsureRequirements(conf: S

[GitHub] spark pull request #19467: [SPARK-22238] Fix plan resolution bug caused by E...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19467#discussion_r144434463 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StatefulOperatorTest.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request #19467: [SPARK-22238] Fix plan resolution bug caused by E...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19467#discussion_r144434390 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/StatefulOperatorTest.scala --- @@ -0,0 +1,49 @@ +/* + * Licensed to the Apache Softwa

[GitHub] spark pull request #19467: [SPARK-22238] Fix plan resolution bug caused by E...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19467#discussion_r144433715 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRDDSuite.scala --- @@ -214,7 +214,7 @@ class StateStoreRDDSuite ext

[GitHub] spark pull request #19467: [SPARK-22238] Fix plan resolution bug caused by E...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19467#discussion_r144402006 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/physical/partitioning.scala --- @@ -243,8 +246,9 @@ case class HashPartitioning(expr

[GitHub] spark pull request #19467: [SPARK-22238] Fix plan resolution bug caused by E...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19467#discussion_r144408406 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/streaming/EnsureStatefulOpPartitioningSuite.scala --- @@ -53,7 +53,7 @@ class EnsureStatefulOpPartition

[GitHub] spark pull request #19467: [SPARK-22238] Fix plan resolution bug caused by E...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19467#discussion_r144406045 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala --- @@ -43,10 +43,11 @@ case class StatefulOperatorStateInf

[GitHub] spark pull request #19467: [SPARK-22238] Fix plan resolution bug caused by E...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19467#discussion_r144406113 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/statefulOperators.scala --- @@ -239,7 +240,7 @@ case class StateStoreRestoreExec(

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82705/ Test FAILed. ---

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82705 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82705/testReport)** for PR 19467 at commit [`5122117`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #18029: [SPARK-20168] [DStream] Add changes to use kinesis fetch...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18029 **[Test build #82708 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82708/testReport)** for PR 18029 at commit [`ac0ae53`](https://github.com/apache/spark/commit/ac

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82704/ Test PASSed. ---

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82704 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82704/testReport)** for PR 19467 at commit [`407d76c`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82703/ Test PASSed. ---

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19451 **[Test build #82703 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82703/testReport)** for PR 19451 at commit [`24b6b6c`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19448 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82702/ Test PASSed. ---

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19448 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19448 **[Test build #82702 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82702/testReport)** for PR 19448 at commit [`f486263`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-10-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19270 > from my understanding SPARK-20657 is proposing to not using the rest api Not really; I just don't know enough JS to actually make the change to use the REST api, but I do believe using it i

[GitHub] spark issue #19390: [SPARK-18935][MESOS] Fix dynamic reservations on mesos

2017-10-12 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/19390 @susanxhuynh could you review pls? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-m

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-10-12 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19270 Right so from my understanding https://issues.apache.org/jira/browse/SPARK-20657 is proposing to not using the rest api and force more on the backing store which exists only for history server bu

[GitHub] spark issue #19486: [SPARK-22268][BUILD] Fix lint-java

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19486 **[Test build #82707 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82707/testReport)** for PR 19486 at commit [`7c321fc`](https://github.com/apache/spark/commit/7c

[GitHub] spark pull request #19486: [SPARK-22268][BUILD] Fix lint-java

2017-10-12 Thread ash211
GitHub user ash211 opened a pull request: https://github.com/apache/spark/pull/19486 [SPARK-22268][BUILD] Fix lint-java ## What changes were proposed in this pull request? Fix java style issues ## How was this patch tested? Run `./dev/lint-java` locally sin

[GitHub] spark issue #19452: [SPARK-22136][SS] Evaluate one-sided conditions early in...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19452 **[Test build #82706 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82706/testReport)** for PR 19452 at commit [`347b92d`](https://github.com/apache/spark/commit/34

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82705 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82705/testReport)** for PR 19467 at commit [`5122117`](https://github.com/apache/spark/commit/51

[GitHub] spark issue #19485: [SPARK-20055] [Docs] Added documentation for loading csv...

2017-10-12 Thread jomach
Github user jomach commented on the issue: https://github.com/apache/spark/pull/19485 Yes I will do it. give me some days please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional com

[GitHub] spark issue #19470: [SPARK-14387][SPARK-18355][SQL] Use Spark schema to read...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19470 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82701/ Test PASSed. ---

[GitHub] spark issue #19470: [SPARK-14387][SPARK-18355][SQL] Use Spark schema to read...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19470 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19448 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82700/ Test PASSed. ---

[GitHub] spark issue #19470: [SPARK-14387][SPARK-18355][SQL] Use Spark schema to read...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19470 **[Test build #82701 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82701/testReport)** for PR 19470 at commit [`8ac1acf`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19448 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19448 **[Test build #82700 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82700/testReport)** for PR 19448 at commit [`42afccb`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #19478: [SPARK-22263][SQL]Refactor deterministic as lazy value

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19478 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19478: [SPARK-22263][SQL]Refactor deterministic as lazy value

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19478 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82698/ Test PASSed. ---

[GitHub] spark issue #19478: [SPARK-22263][SQL]Refactor deterministic as lazy value

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19478 **[Test build #82698 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82698/testReport)** for PR 19478 at commit [`03e88b0`](https://github.com/apache/spark/commit/0

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-12 Thread sathiyapk
Github user sathiyapk commented on the issue: https://github.com/apache/spark/pull/19451 @gengliangwang thanks! I'll squash my commits into 2 if everything is fine.. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19464 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19464 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82696/ Test PASSed. ---

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19464 **[Test build #82696 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82696/testReport)** for PR 19464 at commit [`527b367`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82704 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82704/testReport)** for PR 19467 at commit [`407d76c`](https://github.com/apache/spark/commit/40

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-12 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/19451 LGTM. @gatorsmile please help check again. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addit

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19451 **[Test build #82703 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82703/testReport)** for PR 19451 at commit [`24b6b6c`](https://github.com/apache/spark/commit/24

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19448 **[Test build #82702 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82702/testReport)** for PR 19448 at commit [`f486263`](https://github.com/apache/spark/commit/f4

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19480: [SPARK-22226][SQL] splitExpression can create too many m...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19480 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82697/ Test PASSed. ---

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82699/ Test FAILed. ---

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82699 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82699/testReport)** for PR 19467 at commit [`3f51c5c`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #19480: [SPARK-22226][SQL] splitExpression can create too many m...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19480 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19480: [SPARK-22226][SQL] splitExpression can create too many m...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19480 **[Test build #82697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82697/testReport)** for PR 19480 at commit [`76b5489`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #19452: [SPARK-22136][SS] Evaluate one-sided conditions e...

2017-10-12 Thread joseph-torres
Github user joseph-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/19452#discussion_r144396781 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinHelper.scala --- @@ -66,6 +67,60 @@ object Strea

[GitHub] spark issue #19478: [SPARK-22263][SQL]Refactor deterministic as lazy value

2017-10-12 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/19478 @viirya @hvanhovell @gatorsmile Thanks, I have attached the performance result in the description in this PR. Overall I don't see any downside of the code change. Also it is p

[GitHub] spark issue #18972: [SPARK-21720][SQL] Fix 64KB JVM bytecode limit problem w...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18972 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82695/ Test PASSed. ---

[GitHub] spark issue #18972: [SPARK-21720][SQL] Fix 64KB JVM bytecode limit problem w...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18972 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18972: [SPARK-21720][SQL] Fix 64KB JVM bytecode limit problem w...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18972 **[Test build #82695 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82695/testReport)** for PR 18972 at commit [`c9bc395`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19270 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82693/ Test PASSed. ---

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19270 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19270 **[Test build #82693 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82693/testReport)** for PR 19270 at commit [`ef90c2a`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19464 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19464 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82694/ Test FAILed. ---

[GitHub] spark pull request #19448: [SPARK-22217] [SQL] ParquetFileFormat to support ...

2017-10-12 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/19448#discussion_r144388430 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -138,6 +138,10 @@ class ParquetFileForma

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/19448 Still +1 from me as well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #19464: [SPARK-22233] [core] Allow user to filter out empty spli...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19464 **[Test build #82694 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82694/testReport)** for PR 19464 at commit [`4dcfd83`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #18460: [SPARK-21247][SQL] Type comparison should respect case-s...

2017-10-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/18460 Thanks you, @gatorsmile . Now, it's simplified more. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19470: [SPARK-14387][SPARK-18355][SQL] Use Spark schema to read...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19470 **[Test build #82701 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82701/testReport)** for PR 19470 at commit [`8ac1acf`](https://github.com/apache/spark/commit/8a

[GitHub] spark pull request #19470: [SPARK-14387][SPARK-18355][SQL] Use Spark schema ...

2017-10-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/19470#discussion_r144387023 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2050,4 +2050,80 @@ class SQLQuerySuite extends Quer

[GitHub] spark issue #19478: [SPARK-22263][SQL]Refactor deterministic as lazy value

2017-10-12 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19478 Compare the total optimization time for the TPC-DS queries? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19448 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82692/ Test PASSed. ---

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19448 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19448 **[Test build #82692 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82692/testReport)** for PR 19448 at commit [`c93eb1b`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #19448: [SPARK-22217] [SQL] ParquetFileFormat to support ...

2017-10-12 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/19448#discussion_r144381367 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -138,6 +138,10 @@ class ParquetFi

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19448 **[Test build #82700 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82700/testReport)** for PR 19448 at commit [`42afccb`](https://github.com/apache/spark/commit/42

[GitHub] spark issue #19448: [SPARK-22217] [SQL] ParquetFileFormat to support arbitra...

2017-10-12 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19448 LGTM pending Jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: revi

[GitHub] spark pull request #19448: [SPARK-22217] [SQL] ParquetFileFormat to support ...

2017-10-12 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19448#discussion_r144379591 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -138,6 +138,10 @@ class ParquetFileF

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-12 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18747 @cloud-fan could you please review this since I resolved a confclit? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache

[GitHub] spark issue #19470: [SPARK-14387][SPARK-18355][SQL] Use Spark schema to read...

2017-10-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19470 Hi, @gatorsmile . - **For mismatched column orders**: [SPARK-22267](https://issues.apache.org/jira/browse/SPARK-22267) shows the current status of ORC. - **For mismatched column types*

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82691/ Test PASSed. ---

[GitHub] spark issue #18747: [SPARK-20822][SQL] Generate code to directly get value f...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82691 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82691/testReport)** for PR 18747 at commit [`8134e73`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #19427: Reset spark.driver.bindAddress when starting a Checkpoin...

2017-10-12 Thread ssaavedra
Github user ssaavedra commented on the issue: https://github.com/apache/spark/pull/19427 Should I create the appropriate issue in JIRA? I'm not sure if there is any automation which does that. --- - To unsubscribe,

[GitHub] spark pull request #19448: [SPARK-22217] [SQL] ParquetFileFormat to support ...

2017-10-12 Thread steveloughran
Github user steveloughran commented on a diff in the pull request: https://github.com/apache/spark/pull/19448#discussion_r144375059 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFileFormat.scala --- @@ -138,6 +138,11 @@ class ParquetFi

[GitHub] spark issue #19470: [SPARK-14387][SPARK-18355][SQL] Use Spark schema to read...

2017-10-12 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/19470 To be clear, I'll file another JIRA about ORC status on mismatched column orders. --- - To unsubscribe, e-mail: reviews-un

[GitHub] spark pull request #19452: [SPARK-22136][SS] Evaluate one-sided conditions e...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19452#discussion_r144372504 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinExec.scala --- @@ -387,9 +400,10 @@ case class StreamingS

[GitHub] spark pull request #19452: [SPARK-22136][SS] Evaluate one-sided conditions e...

2017-10-12 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/19452#discussion_r144371994 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/StreamingSymmetricHashJoinHelper.scala --- @@ -66,6 +67,60 @@ object StreamingSymme

[GitHub] spark issue #19478: [SPARK-22263][SQL]Refactor deterministic as lazy value

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19478 **[Test build #82698 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82698/testReport)** for PR 19478 at commit [`03e88b0`](https://github.com/apache/spark/commit/03

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82699 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82699/testReport)** for PR 19467 at commit [`3f51c5c`](https://github.com/apache/spark/commit/3f

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-10-12 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19270 The main change I'm talking about is https://issues.apache.org/jira/browse/SPARK-20657 (code at https://github.com/vanzin/spark/pull/41). I did not change the format of the tables, but how data is l

[GitHub] spark issue #19270: [SPARK-21809] : Change Stage Page to use datatables to s...

2017-10-12 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/19270 @vanzin I haven't followed the prs from SPARK-18085 closely, you keep mentioning it here but don't give anymore context as to what that has to do with this change. Did you change the format of ta

[GitHub] spark issue #19263: [SPARK-22050][CORE] Allow BlockUpdated events to be opti...

2017-10-12 Thread michaelmior
Github user michaelmior commented on the issue: https://github.com/apache/spark/pull/19263 @vanzin Just wanted to ping since I believe I addressed your concerns. Let me know if it looks like any other changes are required. ---

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-12 Thread MrBago
Github user MrBago commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r144365856 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,256 @@ +/* + * Licensed to the Apache Software Foundation (

[GitHub] spark pull request #19464: [SPARK-22233] [core] Allow user to filter out emp...

2017-10-12 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19464#discussion_r144365104 --- Diff: core/src/test/scala/org/apache/spark/FileSuite.scala --- @@ -510,4 +510,83 @@ class FileSuite extends SparkFunSuite with LocalSparkContext {

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82689/ Test FAILed. ---

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82690/ Test FAILed. ---

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19467 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19467: [SPARK-22238] Fix plan resolution bug caused by EnsureSt...

2017-10-12 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19467 **[Test build #82690 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82690/testReport)** for PR 19467 at commit [`48d1f25`](https://github.com/apache/spark/commit/4

<    1   2   3   4   5   6   7   >