[GitHub] spark issue #19933: [SPARK-22744][CORE] Add a configuration to show the appl...

2017-12-09 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19933
  
Can one of the admins verify this patch?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19933: [SPARK-22744][CORE] Add a configuration to show t...

2017-12-09 Thread LantaoJin 
GitHub user LantaoJin opened a pull request:

https://github.com/apache/spark/pull/19933

[SPARK-22744][CORE] Add a configuration to show the application submi…

…t hostname

## What changes were proposed in this pull request?

In MapReduce, we can get the submit hostname via checking the value of 
configuration **mapreduce.job.submithostname**. It can help infra team or other 
people to do support work and debug. Bu in Spark, the information is not 
included.

Then, I suggest to introduce a new configuration parameter, 
**spark.submit.hostname**, which will be set automatically by default, but it 
also allows to set this to an user defined hostname if needed (e.g, using an 
user node instead of Livy server node).

## How was this patch tested?

unit tests


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/LantaoJin/spark SPARK-22744

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/19933.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #19933


commit 616a02d3cb90d92712ee6de9647a0ccc81200d2a
Author: LantaoJin 
Date:   2017-12-09T09:05:10Z

[SPARK-22744][CORE] Add a configuration to show the application submit 
hostname




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19920: [SPARK-21672][CORE] Remove SHS-specific application / at...

2017-12-09 Thread smurakozi 
Github user smurakozi commented on the issue:

https://github.com/apache/spark/pull/19920
  
Thanks for your help @vanzin 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19676: [SPARK-14516][FOLLOWUP] Adding ClusteringEvaluator to ex...

2017-12-09 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19676
  
**[Test build #84681 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84681/testReport)**
 for PR 19676 at commit 
[`feb619d`](https://github.com/apache/spark/commit/feb619d657f6ff66dec240ee4619e6f53208ac18).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19594
  
**[Test build #84682 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84682/testReport)**
 for PR 19594 at commit 
[`e69e213`](https://github.com/apache/spark/commit/e69e21348b4cde2abaec9dbb46381caf1ed3a1a4).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread wzhfy 
Github user wzhfy commented on the issue:

https://github.com/apache/spark/pull/19594
  
retest this please


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19932
  
**[Test build #84680 has 
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84680/testReport)**
 for PR 19932 at commit 
[`48b81b5`](https://github.com/apache/spark/commit/48b81b5065808ffeff99142a03cd59bf54a9ea5d).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19932: [SPARK-22745][SQL] read partition stats from Hive

2017-12-09 Thread wzhfy 
GitHub user wzhfy opened a pull request:

https://github.com/apache/spark/pull/19932

[SPARK-22745][SQL] read partition stats from Hive

## What changes were proposed in this pull request?

Currently Spark can read table stats (e.g. `totalSize, numRows`) from Hive, 
we can also support to read partition stats from Hive using the same logic.

## How was this patch tested?

Added a new test case and modified an existing test case.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/wzhfy/spark read_hive_partition_stats

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/19932.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #19932


commit 48b81b5065808ffeff99142a03cd59bf54a9ea5d
Author: Zhenhua Wang 
Date:   2017-12-09T08:32:48Z

read partition stats




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19911: [SPARK-22729][SQL] Add getTruncateQuery to JdbcDi...

2017-12-09 Thread danielvdende 
Github user danielvdende commented on a diff in the pull request:

https://github.com/apache/spark/pull/19911#discussion_r155916550
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ---
@@ -147,14 +157,6 @@ abstract class JdbcDialect extends Serializable {
 case arrayValue: Array[Any] => 
arrayValue.map(compileValue).mkString(", ")
 case _ => value
   }
-
-  /**
-   * Return Some[true] iff `TRUNCATE TABLE` causes cascading default.
-   * Some[true] : TRUNCATE TABLE causes cascading.
-   * Some[false] : TRUNCATE TABLE does not cause cascading.
-   * None: The behavior of TRUNCATE TABLE is unknown (default).
-   */
-  def isCascadingTruncateTable(): Option[Boolean] = None
--- End diff --

Although, custom third party dialects may want it. 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19911: [SPARK-22729][SQL] Add getTruncateQuery to JdbcDi...

2017-12-09 Thread danielvdende 
Github user danielvdende commented on a diff in the pull request:

https://github.com/apache/spark/pull/19911#discussion_r155916248
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/jdbc/JdbcDialects.scala ---
@@ -147,14 +157,6 @@ abstract class JdbcDialect extends Serializable {
 case arrayValue: Array[Any] => 
arrayValue.map(compileValue).mkString(", ")
 case _ => value
   }
-
-  /**
-   * Return Some[true] iff `TRUNCATE TABLE` causes cascading default.
-   * Some[true] : TRUNCATE TABLE causes cascading.
-   * Some[false] : TRUNCATE TABLE does not cause cascading.
-   * None: The behavior of TRUNCATE TABLE is unknown (default).
-   */
-  def isCascadingTruncateTable(): Option[Boolean] = None
--- End diff --

As a middle ground, could it be marked deprecated? Seeing as we don't 
really need it anymore...


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19594
  
Test FAILed.
Refer to this link for build results (access rights to CI server needed): 
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84679/
Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread AmplabJenkins 
Github user AmplabJenkins commented on the issue:

https://github.com/apache/spark/pull/19594
  
Merged build finished. Test FAILed.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19594: [SPARK-21984] [SQL] Join estimation based on equi-height...

2017-12-09 Thread SparkQA 
Github user SparkQA commented on the issue:

https://github.com/apache/spark/pull/19594
  
**[Test build #84679 has 
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84679/testReport)**
 for PR 19594 at commit 
[`e69e213`](https://github.com/apache/spark/commit/e69e21348b4cde2abaec9dbb46381caf1ed3a1a4).
 * This patch **fails due to an unknown error code, -9**.
 * This patch merges cleanly.
 * This patch adds no public classes.


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

<    1   2