[jira] [Created] (SPARK-15814) Aggregator can return null result

Wenchen Fan created SPARK-15814: --- Summary: Aggregator can return null result Key: SPARK-15814 URL: https://issues.apache.org/jira/browse/SPARK-15814 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-9623) RandomForestRegressor: provide variance of predictions

[ https://issues.apache.org/jira/browse/SPARK-9623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15320036#comment-15320036 ] Yanbo Liang commented on SPARK-9623: [~MechCoder] I'm not working on this, please feel free to take

[jira] [Commented] (SPARK-15369) Investigate selectively using Jython for parts of PySpark

[ https://issues.apache.org/jira/browse/SPARK-15369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319966#comment-15319966 ] holdenk commented on SPARK-15369: - WIP design document

[jira] [Commented] (SPARK-15813) Spark Dyn Allocation Cancel log message misleading

[ https://issues.apache.org/jira/browse/SPARK-15813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319930#comment-15319930 ] Apache Spark commented on SPARK-15813: -- User 'peterableda' has created a pull request for this

[jira] [Assigned] (SPARK-15813) Spark Dyn Allocation Cancel log message misleading

[ https://issues.apache.org/jira/browse/SPARK-15813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15813: Assignee: (was: Apache Spark) > Spark Dyn Allocation Cancel log message misleading >

[jira] [Assigned] (SPARK-15813) Spark Dyn Allocation Cancel log message misleading

[ https://issues.apache.org/jira/browse/SPARK-15813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15813: Assignee: Apache Spark > Spark Dyn Allocation Cancel log message misleading >

[jira] [Updated] (SPARK-15813) Spark Dyn Allocation Cancel log message misleading

[ https://issues.apache.org/jira/browse/SPARK-15813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Ableda updated SPARK-15813: - Description: *Driver requested* message is logged before the *Canceling* message but has the

[jira] [Updated] (SPARK-15813) Spark Dyn Allocation Cancel log message misleading

[ https://issues.apache.org/jira/browse/SPARK-15813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Ableda updated SPARK-15813: - Description: *Driver requested* message is logged before the *Canceling* message but has the

[jira] [Created] (SPARK-15813) Spark Dyn Allocation Cancel log message misleading

Peter Ableda created SPARK-15813: Summary: Spark Dyn Allocation Cancel log message misleading Key: SPARK-15813 URL: https://issues.apache.org/jira/browse/SPARK-15813 Project: Spark Issue

[jira] [Closed] (SPARK-15755) java.lang.NullPointerException when run spark 2.0 setting spark.serializer=org.apache.spark.serializer.KryoSerializer

[ https://issues.apache.org/jira/browse/SPARK-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell closed SPARK-15755. - Resolution: Duplicate > java.lang.NullPointerException when run spark 2.0 setting >

[jira] [Commented] (SPARK-15802) SparkSQL connection fail using shell command "bin/beeline -u "jdbc:hive2://*.*.*.*:10000/default""

[ https://issues.apache.org/jira/browse/SPARK-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319908#comment-15319908 ] marymwu commented on SPARK-15802: - looking forward to your reply, thanks > SparkSQL connection fail

[jira] [Commented] (SPARK-15802) SparkSQL connection fail using shell command "bin/beeline -u "jdbc:hive2://*.*.*.*:10000/default""

[ https://issues.apache.org/jira/browse/SPARK-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319906#comment-15319906 ] marymwu commented on SPARK-15802: - what's the right protocol? how to specify it ? > SparkSQL connection

[jira] [Assigned] (SPARK-14485) Task finished cause fetch failure when its executor has already been removed by driver

[ https://issues.apache.org/jira/browse/SPARK-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-14485: -- Assignee: iward > Task finished cause fetch failure when its executor has already

[jira] [Assigned] (SPARK-15755) java.lang.NullPointerException when run spark 2.0 setting spark.serializer=org.apache.spark.serializer.KryoSerializer

[ https://issues.apache.org/jira/browse/SPARK-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15755: Assignee: (was: Apache Spark) > java.lang.NullPointerException when run spark 2.0

[jira] [Assigned] (SPARK-15755) java.lang.NullPointerException when run spark 2.0 setting spark.serializer=org.apache.spark.serializer.KryoSerializer

[ https://issues.apache.org/jira/browse/SPARK-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15755: Assignee: Apache Spark > java.lang.NullPointerException when run spark 2.0 setting >

[jira] [Commented] (SPARK-15755) java.lang.NullPointerException when run spark 2.0 setting spark.serializer=org.apache.spark.serializer.KryoSerializer

[ https://issues.apache.org/jira/browse/SPARK-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319866#comment-15319866 ] Apache Spark commented on SPARK-15755: -- User 'marymwu' has created a pull request for this issue:

[jira] [Created] (SPARK-15812) Allow sorting on aggregated streaming dataframe when the output mode is Complete

Tathagata Das created SPARK-15812: - Summary: Allow sorting on aggregated streaming dataframe when the output mode is Complete Key: SPARK-15812 URL: https://issues.apache.org/jira/browse/SPARK-15812

[jira] [Resolved] (SPARK-15517) Add support for complete output mode

[ https://issues.apache.org/jira/browse/SPARK-15517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-15517. --- Resolution: Fixed Fix Version/s: 2.0.0 > Add support for complete output mode >

[jira] [Commented] (SPARK-15046) When running hive-thriftserver with yarn on a secure cluster the workers fail with java.lang.NumberFormatException

[ https://issues.apache.org/jira/browse/SPARK-15046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319817#comment-15319817 ] Jie Huang commented on SPARK-15046: --- OK, I see. Thanks [~tleftwich]. If so, it seems we'd better to use

[jira] [Updated] (SPARK-15789) Allow reserved keywords in most places

[ https://issues.apache.org/jira/browse/SPARK-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-15789: Assignee: Herman van Hovell > Allow reserved keywords in most places >

[jira] [Resolved] (SPARK-15789) Allow reserved keywords in most places

[ https://issues.apache.org/jira/browse/SPARK-15789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15789. - Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13534

[jira] [Updated] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franklyn Dsouza updated SPARK-15811: Description: I've built spark-2.0-preview (8f5a04b) with scala-2.10 using the following

[jira] [Updated] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franklyn Dsouza updated SPARK-15811: Description: I've built spark-2.0-preview (8f5a04b) with scala-2.10 using the following

[jira] [Updated] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-15811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Franklyn Dsouza updated SPARK-15811: Description: I've built spark-2.0-preview (8f5a04b) with scala-2.10 using the following

[jira] [Created] (SPARK-15811) UDFs do not work in Spark 2.0-preview built with scala 2.10

Franklyn Dsouza created SPARK-15811: --- Summary: UDFs do not work in Spark 2.0-preview built with scala 2.10 Key: SPARK-15811 URL: https://issues.apache.org/jira/browse/SPARK-15811 Project: Spark

[jira] [Commented] (SPARK-15804) Manually added metadata not saving with parquet

[ https://issues.apache.org/jira/browse/SPARK-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319706#comment-15319706 ] kevin yu commented on SPARK-15804: -- I will submit a PR soon. Thanks. > Manually added metadata not

[jira] [Resolved] (SPARK-15580) Add ContinuousQueryInfo to make ContinuousQueryListener events serializable

[ https://issues.apache.org/jira/browse/SPARK-15580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-15580. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13335

[jira] [Resolved] (SPARK-14485) Task finished cause fetch failure when its executor has already been removed by driver

[ https://issues.apache.org/jira/browse/SPARK-14485?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-14485. Resolution: Fixed Fix Version/s: 2.0.0 > Task finished cause fetch failure when its

[jira] [Commented] (SPARK-11106) Should ML Models contains single models or Pipelines?

[ https://issues.apache.org/jira/browse/SPARK-11106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319685#comment-15319685 ] Xusen Yin commented on SPARK-11106: --- RFormula is easy to use, but it may not always do right things.

[jira] [Comment Edited] (SPARK-15780) Support mapValues on KeyValueGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319594#comment-15319594 ] koert kuipers edited comment on SPARK-15780 at 6/7/16 10:34 PM: also see

[jira] [Commented] (SPARK-15780) Support mapValues on KeyValueGroupedDataset

[ https://issues.apache.org/jira/browse/SPARK-15780?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319594#comment-15319594 ] koert kuipers commented on SPARK-15780: --- also see this discussion:

[jira] [Assigned] (SPARK-14816) Update MLlib, GraphX, SparkR websites for 2.0

[ https://issues.apache.org/jira/browse/SPARK-14816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang reassigned SPARK-14816: --- Assignee: Yanbo Liang > Update MLlib, GraphX, SparkR websites for 2.0 >

[jira] [Resolved] (SPARK-13590) Document the behavior of spark.ml logistic regression and AFT survival regression when there are constant features

[ https://issues.apache.org/jira/browse/SPARK-13590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-13590. - Resolution: Fixed Fix Version/s: 2.0.0 > Document the behavior of spark.ml logistic

[jira] [Updated] (SPARK-14816) Update MLlib, GraphX, SparkR websites for 2.0

[ https://issues.apache.org/jira/browse/SPARK-14816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-14816: Assignee: (was: Yanbo Liang) > Update MLlib, GraphX, SparkR websites for 2.0 >

[jira] [Resolved] (SPARK-15674) Deprecates "CREATE TEMPORARY TABLE USING...", use "CREATE TEMPORARY VIEW USING..." instead.

[ https://issues.apache.org/jira/browse/SPARK-15674?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-15674. --- Resolution: Resolved Assignee: Sean Zhong > Deprecates "CREATE TEMPORARY TABLE

[jira] [Updated] (SPARK-15810) Aggregator doesn't play nice with Option

[ https://issues.apache.org/jira/browse/SPARK-15810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] koert kuipers updated SPARK-15810: -- Description: {noformat} val ds1 = List(("a", 1), ("a", 2), ("a", 3)).toDS val ds2

[jira] [Created] (SPARK-15810) Aggregator doesn't play nice with Option

koert kuipers created SPARK-15810: - Summary: Aggregator doesn't play nice with Option Key: SPARK-15810 URL: https://issues.apache.org/jira/browse/SPARK-15810 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-9623) RandomForestRegressor: provide variance of predictions

[ https://issues.apache.org/jira/browse/SPARK-9623?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319450#comment-15319450 ] Manoj Kumar commented on SPARK-9623: [~yanboliang] Are you still working on this? Would you mind if I

[jira] [Updated] (SPARK-14279) Improve the spark build to pick the version information from the pom file and add git commit information

[ https://issues.apache.org/jira/browse/SPARK-14279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-14279: --- Fix Version/s: (was: 2.1.0) 2.0.0 > Improve the spark build to pick

[jira] [Created] (SPARK-15809) PySpark SQL UDF default returnType

Vladimir Feinberg created SPARK-15809: - Summary: PySpark SQL UDF default returnType Key: SPARK-15809 URL: https://issues.apache.org/jira/browse/SPARK-15809 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-15808) Wrong Results or Strange Errors In Append-mode DataFrame Writing Due to Mismatched File Formats

[ https://issues.apache.org/jira/browse/SPARK-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15808: Assignee: Apache Spark > Wrong Results or Strange Errors In Append-mode DataFrame Writing

[jira] [Commented] (SPARK-15808) Wrong Results or Strange Errors In Append-mode DataFrame Writing Due to Mismatched File Formats

[ https://issues.apache.org/jira/browse/SPARK-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319156#comment-15319156 ] Apache Spark commented on SPARK-15808: -- User 'gatorsmile' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15808) Wrong Results or Strange Errors In Append-mode DataFrame Writing Due to Mismatched File Formats

[ https://issues.apache.org/jira/browse/SPARK-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15808: Assignee: (was: Apache Spark) > Wrong Results or Strange Errors In Append-mode

[jira] [Created] (SPARK-15808) Wrong Results or Strange Errors In Append-mode DataFrame Writing Due to Mismatched File Formats

Xiao Li created SPARK-15808: --- Summary: Wrong Results or Strange Errors In Append-mode DataFrame Writing Due to Mismatched File Formats Key: SPARK-15808 URL: https://issues.apache.org/jira/browse/SPARK-15808

[jira] [Updated] (SPARK-15808) Wrong Results or Strange Errors In Append-mode DataFrame Writing Due to Mismatched File Formats

[ https://issues.apache.org/jira/browse/SPARK-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-15808: Description: Example 1: PARQUET -> CSV {noformat} createDF(0,

[jira] [Updated] (SPARK-15808) Wrong Results or Strange Errors In Append-mode DataFrame Writing Due to Mismatched File Formats

[ https://issues.apache.org/jira/browse/SPARK-15808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-15808: Description: Example 1: PARQUET -> CSV {noformat} createDF(0,

[jira] [Updated] (SPARK-15804) Manually added metadata not saving with parquet

[ https://issues.apache.org/jira/browse/SPARK-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Charlie Evans updated SPARK-15804: -- Description: Adding metadata with col().as(_, metadata) then saving the resultant dataframe

[jira] [Commented] (SPARK-15785) Add initialModel param to Gaussian Mixture Model (GMM) in spark.ml

[ https://issues.apache.org/jira/browse/SPARK-15785?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319100#comment-15319100 ] Gayathri Murali commented on SPARK-15785: - I will work on this. Thanks! > Add initialModel param

[jira] [Assigned] (SPARK-15807) Support varargs for distinct/dropDuplicates in Dataset/DataFrame

[ https://issues.apache.org/jira/browse/SPARK-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15807: Assignee: Apache Spark > Support varargs for distinct/dropDuplicates in Dataset/DataFrame

[jira] [Assigned] (SPARK-15807) Support varargs for distinct/dropDuplicates in Dataset/DataFrame

[ https://issues.apache.org/jira/browse/SPARK-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15807: Assignee: (was: Apache Spark) > Support varargs for distinct/dropDuplicates in

[jira] [Commented] (SPARK-15807) Support varargs for distinct/dropDuplicates in Dataset/DataFrame

[ https://issues.apache.org/jira/browse/SPARK-15807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15319090#comment-15319090 ] Apache Spark commented on SPARK-15807: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Created] (SPARK-15807) Support varargs for distinct/dropDuplicates in Dataset/DataFrame

Dongjoon Hyun created SPARK-15807: - Summary: Support varargs for distinct/dropDuplicates in Dataset/DataFrame Key: SPARK-15807 URL: https://issues.apache.org/jira/browse/SPARK-15807 Project: Spark

[jira] [Commented] (SPARK-15804) Manually added metadata not saving with parquet

[ https://issues.apache.org/jira/browse/SPARK-15804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318897#comment-15318897 ] Takeshi Yamamuro commented on SPARK-15804: -- `MetadataBuilder` is one of developer apis, so is

[jira] [Resolved] (SPARK-15760) Documentation missing for package-related config options

[ https://issues.apache.org/jira/browse/SPARK-15760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-15760. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.0.0 >

[jira] [Resolved] (SPARK-15684) Not mask startsWith and endsWith in R

[ https://issues.apache.org/jira/browse/SPARK-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-15684. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request

[jira] [Updated] (SPARK-15684) Not mask startsWith and endsWith in R

[ https://issues.apache.org/jira/browse/SPARK-15684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-15684: -- Assignee: Miao Wang > Not mask startsWith and endsWith in R >

[jira] [Commented] (SPARK-15799) Release SparkR on CRAN

[ https://issues.apache.org/jira/browse/SPARK-15799?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318769#comment-15318769 ] Shivaram Venkataraman commented on SPARK-15799: --- I dont think there are any license issues

[jira] [Assigned] (SPARK-15805) update the whole sql programming guide

[ https://issues.apache.org/jira/browse/SPARK-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15805: Assignee: (was: Apache Spark) > update the whole sql programming guide >

[jira] [Commented] (SPARK-15805) update the whole sql programming guide

[ https://issues.apache.org/jira/browse/SPARK-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318766#comment-15318766 ] Apache Spark commented on SPARK-15805: -- User 'WeichenXu123' has created a pull request for this

[jira] [Assigned] (SPARK-15805) update the whole sql programming guide

[ https://issues.apache.org/jira/browse/SPARK-15805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15805: Assignee: Apache Spark > update the whole sql programming guide >

[jira] [Assigned] (SPARK-15806) Update doc for SPARK_MASTER_IP

[ https://issues.apache.org/jira/browse/SPARK-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15806: Assignee: (was: Apache Spark) > Update doc for SPARK_MASTER_IP >

[jira] [Commented] (SPARK-15806) Update doc for SPARK_MASTER_IP

[ https://issues.apache.org/jira/browse/SPARK-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318761#comment-15318761 ] Apache Spark commented on SPARK-15806: -- User 'bomeng' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15806) Update doc for SPARK_MASTER_IP

[ https://issues.apache.org/jira/browse/SPARK-15806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15806: Assignee: Apache Spark > Update doc for SPARK_MASTER_IP > --

[jira] [Created] (SPARK-15806) Update doc for SPARK_MASTER_IP

Bo Meng created SPARK-15806: --- Summary: Update doc for SPARK_MASTER_IP Key: SPARK-15806 URL: https://issues.apache.org/jira/browse/SPARK-15806 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-15805) update the whole sql programming guide

Weichen Xu created SPARK-15805: -- Summary: update the whole sql programming guide Key: SPARK-15805 URL: https://issues.apache.org/jira/browse/SPARK-15805 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-15801) spark-submit --num-executors switch also works without YARN

[ https://issues.apache.org/jira/browse/SPARK-15801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318721#comment-15318721 ] Marcelo Vanzin commented on SPARK-15801: I'm not really sure of how standalone works these days

[jira] [Commented] (SPARK-15652) Missing org.apache.spark.launcher.SparkAppHandle.Listener notification if SparkSubmit JVM shutsdown

[ https://issues.apache.org/jira/browse/SPARK-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318711#comment-15318711 ] Marcelo Vanzin commented on SPARK-15652: I'm a little worried about that because it touches a

[jira] [Commented] (SPARK-15755) java.lang.NullPointerException when run spark 2.0 setting spark.serializer=org.apache.spark.serializer.KryoSerializer

[ https://issues.apache.org/jira/browse/SPARK-15755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318692#comment-15318692 ] Bo Meng commented on SPARK-15755: - Could you provide a test case to reproduce the issue? >

[jira] [Commented] (SPARK-15652) Missing org.apache.spark.launcher.SparkAppHandle.Listener notification if SparkSubmit JVM shutsdown

[ https://issues.apache.org/jira/browse/SPARK-15652?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318683#comment-15318683 ] Subroto Sanyal commented on SPARK-15652: hi [~vanzin] Can this be merged to 1.6 branch? >

[jira] [Assigned] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15730: Assignee: (was: Apache Spark) > [Spark SQL] the value of 'hiveconf' parameter in

[jira] [Commented] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318654#comment-15318654 ] Cheng Hao commented on SPARK-15730: --- [~jameszhouyi], can you please verify this fixing? > [Spark SQL]

[jira] [Commented] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318648#comment-15318648 ] Apache Spark commented on SPARK-15730: -- User 'chenghao-intel' has created a pull request for this

[jira] [Assigned] (SPARK-15730) [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI don't take effect in spark-sql session

[ https://issues.apache.org/jira/browse/SPARK-15730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15730: Assignee: Apache Spark > [Spark SQL] the value of 'hiveconf' parameter in Spark-sql CLI

[jira] [Resolved] (SPARK-13570) pyspark save with partitionBy is very slow

[ https://issues.apache.org/jira/browse/SPARK-13570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13570. --- Resolution: Incomplete > pyspark save with partitionBy is very slow >

[jira] [Commented] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318558#comment-15318558 ] Sean Owen commented on SPARK-15796: --- I'm not sure what you mean about storing RDDs that don't fit in

[jira] [Comment Edited] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318530#comment-15318530 ] Gabor Feher edited comment on SPARK-15796 at 6/7/16 2:15 PM: - MEMORY_ONLY

[jira] [Commented] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318530#comment-15318530 ] Gabor Feher commented on SPARK-15796: - MEMORY_ONLY caching works in a way that when a partition

[jira] [Commented] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318526#comment-15318526 ] Sean Owen commented on SPARK-15796: --- To leave a little extra room and to match the old behavior -- yeah

[jira] [Commented] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318523#comment-15318523 ] Daniel Darabos commented on SPARK-15796: > The only argument against it was that it's specific to

[jira] [Commented] (SPARK-15564) App name is the main class name in Spark streaming jobs

[ https://issues.apache.org/jira/browse/SPARK-15564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318520#comment-15318520 ] Sean Owen commented on SPARK-15564: --- On further review, I don't see how there's a null appName here.

[jira] [Commented] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318496#comment-15318496 ] Sean Owen commented on SPARK-15796: --- Yeah, sounds like we should change the default min cache size so

[jira] [Commented] (SPARK-15065) HiveSparkSubmitSuite's "set spark.sql.warehouse.dir" is flaky

[ https://issues.apache.org/jira/browse/SPARK-15065?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318492#comment-15318492 ] Pete Robbins commented on SPARK-15065: -- I think this may be related to

[jira] [Commented] (SPARK-15796) Spark 1.6 default memory settings can cause heavy GC when caching

[ https://issues.apache.org/jira/browse/SPARK-15796?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318486#comment-15318486 ] Daniel Darabos commented on SPARK-15796: The example program takes less than a minute on Spark

[jira] [Resolved] (SPARK-15787) Display more helpful error messages for several invalid operations

[ https://issues.apache.org/jira/browse/SPARK-15787?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15787. --- Resolution: Duplicate Fix Version/s: (was: 1.2.1) Please comment on the other JIRA with

[jira] [Commented] (SPARK-15801) spark-submit --num-executors switch also works without YARN

[ https://issues.apache.org/jira/browse/SPARK-15801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318447#comment-15318447 ] Jonathan Taws commented on SPARK-15801: --- Indeed, I am getting the same behavior. After quickly

[jira] [Commented] (SPARK-15779) SQL context fails when Hive uses Tez as its default execution engine

[ https://issues.apache.org/jira/browse/SPARK-15779?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318443#comment-15318443 ] Alexandre Linte commented on SPARK-15779: - Thank you for your reply Zhang, You're right, I'm

[jira] [Commented] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318439#comment-15318439 ] Sean Owen commented on SPARK-15781: --- Yeah, I'd love for someone who really knows standalone to confirm

[jira] [Updated] (SPARK-15792) [SQL] Allows operator to change the verbosity in explain output.

[ https://issues.apache.org/jira/browse/SPARK-15792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15792: -- Assignee: Sean Zhong > [SQL] Allows operator to change the verbosity in explain output. >

[jira] [Comment Edited] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318426#comment-15318426 ] Jonathan Taws edited comment on SPARK-15781 at 6/7/16 1:00 PM: --- Then a

[jira] [Commented] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318426#comment-15318426 ] Jonathan Taws commented on SPARK-15781: --- Then a little sentence as this one could do the trick : If

[jira] [Created] (SPARK-15804) Manually added metadata not saving with parquet

Charlie Evans created SPARK-15804: - Summary: Manually added metadata not saving with parquet Key: SPARK-15804 URL: https://issues.apache.org/jira/browse/SPARK-15804 Project: Spark Issue

[jira] [Commented] (SPARK-15801) spark-submit --num-executors switch also works without YARN

[ https://issues.apache.org/jira/browse/SPARK-15801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318392#comment-15318392 ] Sean Owen commented on SPARK-15801: --- I get the result you get _without_ {{--num-executors}}. I've kind

[jira] [Reopened] (SPARK-15802) SparkSQL connection fail using shell command "bin/beeline -u "jdbc:hive2://*.*.*.*:10000/default""

[ https://issues.apache.org/jira/browse/SPARK-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-15802: --- oops, didn't yet mean to resolve > SparkSQL connection fail using shell command "bin/beeline -u >

[jira] [Resolved] (SPARK-15802) SparkSQL connection fail using shell command "bin/beeline -u "jdbc:hive2://*.*.*.*:10000/default""

[ https://issues.apache.org/jira/browse/SPARK-15802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15802. --- Resolution: Fixed Doesn't that just mean you used the wrong protocol, and when you specified the

[jira] [Commented] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318379#comment-15318379 ] Sean Owen commented on SPARK-15781: --- These are reasonable ideas, though I think the idea is to move

[jira] [Assigned] (SPARK-15803) Support with statement syntax for SparkSession

[ https://issues.apache.org/jira/browse/SPARK-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15803: Assignee: (was: Apache Spark) > Support with statement syntax for SparkSession >

[jira] [Commented] (SPARK-15803) Support with statement syntax for SparkSession

[ https://issues.apache.org/jira/browse/SPARK-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318375#comment-15318375 ] Apache Spark commented on SPARK-15803: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15803) Support with statement syntax for SparkSession

[ https://issues.apache.org/jira/browse/SPARK-15803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15803: Assignee: Apache Spark > Support with statement syntax for SparkSession >

[jira] [Created] (SPARK-15803) Support with statement syntax for SparkSession

Jeff Zhang created SPARK-15803: -- Summary: Support with statement syntax for SparkSession Key: SPARK-15803 URL: https://issues.apache.org/jira/browse/SPARK-15803 Project: Spark Issue Type:

[jira] [Comment Edited] (SPARK-15801) spark-submit --num-executors switch also works without YARN

[ https://issues.apache.org/jira/browse/SPARK-15801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15318185#comment-15318185 ] Jonathan Taws edited comment on SPARK-15801 at 6/7/16 10:13 AM: It is

  1   2   >