[jira] [Resolved] (SPARK-19924) Handle InvocationTargetException for all Hive Shim

2017-03-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19924. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17265

[jira] [Commented] (SPARK-19944) Move SQLConf from sql/core to sql/catalyst

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923519#comment-15923519 ] Apache Spark commented on SPARK-19944: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19944) Move SQLConf from sql/core to sql/catalyst

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19944: Assignee: Reynold Xin (was: Apache Spark) > Move SQLConf from sql/core to sql/catalyst >

[jira] [Assigned] (SPARK-19944) Move SQLConf from sql/core to sql/catalyst

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19944: Assignee: Apache Spark (was: Reynold Xin) > Move SQLConf from sql/core to sql/catalyst >

[jira] [Created] (SPARK-19944) Move SQLConf from sql/core to sql/catalyst

2017-03-13 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-19944: --- Summary: Move SQLConf from sql/core to sql/catalyst Key: SPARK-19944 URL: https://issues.apache.org/jira/browse/SPARK-19944 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16365) Ideas for moving "mllib-local" forward

2017-03-13 Thread Hollin Wilkins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923494#comment-15923494 ] Hollin Wilkins commented on SPARK-16365: Hey guys, Sorry for not chiming in for a while here. I

[jira] [Commented] (SPARK-19943) commons-collections has vulnerability: CVE-2015-6420

2017-03-13 Thread meiyoula (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19943?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923425#comment-15923425 ] meiyoula commented on SPARK-19943: -- [~srowen] Do you think this vulnerability affects spark? >

[jira] [Created] (SPARK-19943) commons-collections has vulnerability: CVE-2015-6420

2017-03-13 Thread meiyoula (JIRA)
meiyoula created SPARK-19943: Summary: commons-collections has vulnerability: CVE-2015-6420 Key: SPARK-19943 URL: https://issues.apache.org/jira/browse/SPARK-19943 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-10908) ClassCastException in HadoopRDD.getJobConf

2017-03-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923415#comment-15923415 ] Hyukjin Kwon commented on SPARK-10908: -- Please provide the environment or code [~nadenf]. Otherwise,

[jira] [Commented] (SPARK-16365) Ideas for moving "mllib-local" forward

2017-03-13 Thread Asher Krim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923406#comment-15923406 ] Asher Krim commented on SPARK-16365: Thanks for pointing me to this Jira [~josephkb], I somehow

[jira] [Resolved] (SPARK-6169) Shuffle based join

2017-03-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6169?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-6169. - Resolution: Invalid I am resolving this as I don't think this describes the concrete suggestion

[jira] [Commented] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

2017-03-13 Thread Karthik Palaniappan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923363#comment-15923363 ] Karthik Palaniappan commented on SPARK-19941: - To repro: Set up Spark on YARN (Hadoop 2.8).

[jira] [Resolved] (SPARK-19942) DataFrameWriter - CSV options "ignoreLeadingWhiteSpace" and "ignoreTrailingWhiteSpace" Not working

2017-03-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19942?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19942. -- Resolution: Duplicate > DataFrameWriter - CSV options "ignoreLeadingWhiteSpace" and >

[jira] [Commented] (SPARK-18579) spark-csv strips whitespace (pyspark)

2017-03-13 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923343#comment-15923343 ] Hyukjin Kwon commented on SPARK-18579: -- It is not implemented in spark-csv library (see

[jira] [Commented] (SPARK-18579) spark-csv strips whitespace (pyspark)

2017-03-13 Thread nirav patel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923338#comment-15923338 ] nirav patel commented on SPARK-18579: - I am having same issue. Created SPARK-19942. It seems like

[jira] [Created] (SPARK-19942) DataFrameWriter - CSV options "ignoreLeadingWhiteSpace" and "ignoreTrailingWhiteSpace" Not working

2017-03-13 Thread nirav patel (JIRA)
nirav patel created SPARK-19942: --- Summary: DataFrameWriter - CSV options "ignoreLeadingWhiteSpace" and "ignoreTrailingWhiteSpace" Not working Key: SPARK-19942 URL: https://issues.apache.org/jira/browse/SPARK-19942

[jira] [Commented] (SPARK-14174) Accelerate KMeans via Mini-Batch EM

2017-03-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923314#comment-15923314 ] Joseph K. Bradley commented on SPARK-14174: --- I'm fine with improving KMeans, but I'm still not

[jira] [Commented] (SPARK-14682) Provide evaluateEachIteration method or equivalent for spark.ml GBTs

2017-03-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923309#comment-15923309 ] Joseph K. Bradley commented on SPARK-14682: --- [~podongfeng] Sorry for the slow response. To

[jira] [Commented] (SPARK-19653) `Vector` Type Should Be A First-Class Citizen In Spark SQL

2017-03-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923307#comment-15923307 ] Joseph K. Bradley commented on SPARK-19653: --- I agree it'd be nice to make it easier to work

[jira] [Created] (SPARK-19941) Spark should not schedule tasks on executors on decommissioning YARN nodes

2017-03-13 Thread Karthik Palaniappan (JIRA)
Karthik Palaniappan created SPARK-19941: --- Summary: Spark should not schedule tasks on executors on decommissioning YARN nodes Key: SPARK-19941 URL: https://issues.apache.org/jira/browse/SPARK-19941

[jira] [Commented] (SPARK-4591) Algorithm/model parity for spark.ml (Scala)

2017-03-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923294#comment-15923294 ] Joseph K. Bradley commented on SPARK-4591: -- For the record: * Kernel Density: later, I'd say *

[jira] [Commented] (SPARK-19416) Dataset.schema is inconsistent with Dataset in handling columns with periods

2017-03-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923286#comment-15923286 ] Joseph K. Bradley commented on SPARK-19416: --- Hm, I'd call my synopsis above a "complaint" but

[jira] [Commented] (SPARK-10413) ML models should support prediction on single instances

2017-03-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923258#comment-15923258 ] Joseph K. Bradley commented on SPARK-10413: --- [~akrim] I agree this would be useful, but it will

[jira] [Comment Edited] (SPARK-10436) spark-submit overwrites spark.files defaults with the job script filename

2017-03-13 Thread Naresh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923175#comment-15923175 ] Naresh edited comment on SPARK-10436 at 3/13/17 11:30 PM: -- This issue got fixed

[jira] [Commented] (SPARK-10436) spark-submit overwrites spark.files defaults with the job script filename

2017-03-13 Thread Naresh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923175#comment-15923175 ] Naresh commented on SPARK-10436: This issue got fixed in spark 2.0.0 version.

[jira] [Commented] (SPARK-18608) Spark ML algorithms that check RDD cache level for internal caching double-cache data

2017-03-13 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923158#comment-15923158 ] yuhao yang commented on SPARK-18608: Thanks [~podongfeng] I'd say it's a better solution as it avoids

[jira] [Commented] (SPARK-19875) Map->filter on many columns gets stuck in constraint inference optimization code

2017-03-13 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19875?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923152#comment-15923152 ] Sameer Agarwal commented on SPARK-19875: I also verified with [~jay.pranavamurthi] that this is a

[jira] [Commented] (SPARK-19940) FPGrowthModel.transform should skip duplicated items

2017-03-13 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923149#comment-15923149 ] Maciej Szymkiewicz commented on SPARK-19940: cc [~yuhaoyan] > FPGrowthModel.transform

[jira] [Commented] (SPARK-19940) FPGrowthModel.transform should skip duplicated items

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923146#comment-15923146 ] Apache Spark commented on SPARK-19940: -- User 'zero323' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19940) FPGrowthModel.transform should skip duplicated items

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19940: Assignee: Apache Spark > FPGrowthModel.transform should skip duplicated items >

[jira] [Assigned] (SPARK-19940) FPGrowthModel.transform should skip duplicated items

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19940: Assignee: (was: Apache Spark) > FPGrowthModel.transform should skip duplicated items

[jira] [Issue Comment Deleted] (SPARK-14503) spark.ml Scala API for FPGrowth

2017-03-13 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Szymkiewicz updated SPARK-14503: --- Comment: was deleted (was: [~yuhaoyan] Sure thing. I'll try to do it later today.)

[jira] [Created] (SPARK-19940) FPGrowthModel.transform should skip duplicated items

2017-03-13 Thread Maciej Szymkiewicz (JIRA)
Maciej Szymkiewicz created SPARK-19940: -- Summary: FPGrowthModel.transform should skip duplicated items Key: SPARK-19940 URL: https://issues.apache.org/jira/browse/SPARK-19940 Project: Spark

[jira] [Commented] (SPARK-19634) Feature parity for descriptive statistics in MLlib

2017-03-13 Thread Timothy Hunter (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15923119#comment-15923119 ] Timothy Hunter commented on SPARK-19634: I was not able to finish it in time, but the bulk of the

[jira] [Updated] (SPARK-19282) RandomForestRegressionModel summary should expose getMaxDepth

2017-03-13 Thread Nick Lothian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Lothian updated SPARK-19282: - Component/s: PySpark > RandomForestRegressionModel summary should expose getMaxDepth >

[jira] [Updated] (SPARK-19282) RandomForestRegressionModel summary should expose getMaxDepth

2017-03-13 Thread Nick Lothian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Lothian updated SPARK-19282: - Summary: RandomForestRegressionModel summary should expose getMaxDepth (was: R

[jira] [Reopened] (SPARK-19282) R RandomForestRegressionModel summary should expose getMaxDepth

2017-03-13 Thread Nick Lothian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Lothian reopened SPARK-19282: -- This issue was raised against the Python model, not the R version. I agree the R model should be

[jira] [Commented] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922777#comment-15922777 ] Apache Spark commented on SPARK-19872: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19872: Assignee: Apache Spark > UnicodeDecodeError in Pyspark on sc.textFile read with

[jira] [Assigned] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19872: Assignee: (was: Apache Spark) > UnicodeDecodeError in Pyspark on sc.textFile read

[jira] [Assigned] (SPARK-19817) make it clear that `timeZone` option is a general option in DataFrameReader/Writer

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19817: Assignee: Apache Spark (was: Takuya Ueshin) > make it clear that `timeZone` option is a

[jira] [Commented] (SPARK-19817) make it clear that `timeZone` option is a general option in DataFrameReader/Writer

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922725#comment-15922725 ] Apache Spark commented on SPARK-19817: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19817) make it clear that `timeZone` option is a general option in DataFrameReader/Writer

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19817: Assignee: Takuya Ueshin (was: Apache Spark) > make it clear that `timeZone` option is a

[jira] [Commented] (SPARK-18057) Update structured streaming kafka from 10.0.1 to 10.2.0

2017-03-13 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18057?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922715#comment-15922715 ] Michael Armbrust commented on SPARK-18057: -- So to summarize, it'll be unfortunate if Kafka

[jira] [Commented] (SPARK-19912) String literals are not escaped while performing Hive metastore level partition pruning

2017-03-13 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922705#comment-15922705 ] Dongjoon Hyun commented on SPARK-19912: --- As [~smilegator] mentioned, this is related to the

[jira] [Assigned] (SPARK-19939) Add support for association rules in ML

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19939: Assignee: Apache Spark > Add support for association rules in ML >

[jira] [Commented] (SPARK-19939) Add support for association rules in ML

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15922676#comment-15922676 ] Apache Spark commented on SPARK-19939: -- User 'hhbyyh' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19939) Add support for association rules in ML

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19939: Assignee: (was: Apache Spark) > Add support for association rules in ML >

[jira] [Created] (SPARK-19939) Add support for association rules in ML

2017-03-13 Thread yuhao yang (JIRA)
yuhao yang created SPARK-19939: -- Summary: Add support for association rules in ML Key: SPARK-19939 URL: https://issues.apache.org/jira/browse/SPARK-19939 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-19936) Page rank example takes long time to complete

2017-03-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15917373#comment-15917373 ] Sean Owen commented on SPARK-19936: --- That's the question, yes. It sounds like communication problem

[jira] [Resolved] (SPARK-19789) Add the shortcut of .format("parquet").option("path", "/hdfs/path").partitionBy("col1", "col2").start()

2017-03-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19789?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19789. --- Resolution: Won't Fix > Add the shortcut of .format("parquet").option("path", >

[jira] [Resolved] (SPARK-19938) java.lang.ClassCastException: cannot assign instance of scala.collection.immutable.List$SerializationProxy to field

2017-03-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19938. --- Resolution: Duplicate This is a question you can answer with Google and is not a Spark issue. >

[jira] [Updated] (SPARK-11569) StringIndexer transform fails when column contains nulls

2017-03-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-11569: -- Shepherd: Joseph K. Bradley > StringIndexer transform fails when column contains nulls

[jira] [Commented] (SPARK-19933) TPCDS Q70 went wrong while explaining

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907774#comment-15907774 ] Apache Spark commented on SPARK-19933: -- User 'hvanhovell' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-11569) StringIndexer transform fails when column contains nulls

2017-03-13 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15813712#comment-15813712 ] Joseph K. Bradley edited comment on SPARK-11569 at 3/13/17 4:31 PM:

[jira] [Assigned] (SPARK-19933) TPCDS Q70 went wrong while explaining

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19933: Assignee: Apache Spark > TPCDS Q70 went wrong while explaining >

[jira] [Assigned] (SPARK-19933) TPCDS Q70 went wrong while explaining

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19933?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19933: Assignee: (was: Apache Spark) > TPCDS Q70 went wrong while explaining >

[jira] [Commented] (SPARK-16784) Configurable log4j settings

2017-03-13 Thread Josh Bacon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907766#comment-15907766 ] Josh Bacon commented on SPARK-16784: >From what I've seen, this limitation is also experienced on the

[jira] [Commented] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned persisted tables

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907750#comment-15907750 ] Apache Spark commented on SPARK-19887: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned persisted tables

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19887: Assignee: (was: Apache Spark) > __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL

[jira] [Assigned] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned persisted tables

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19887: Assignee: Apache Spark > __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition

[jira] [Commented] (SPARK-19887) __HIVE_DEFAULT_PARTITION__ is not interpreted as NULL partition value in partitioned persisted tables

2017-03-13 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19887?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907646#comment-15907646 ] Wenchen Fan commented on SPARK-19887: - This is actually the same behavior of hive, see

[jira] [Commented] (SPARK-7460) Provide DataFrame.zip (analog of RDD.zip) to merge two data frames

2017-03-13 Thread Chang chen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7460?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907632#comment-15907632 ] Chang chen commented on SPARK-7460: --- +1 for supporting {{DataFrame.zip}}, python guys of my company

[jira] [Commented] (SPARK-19938) java.lang.ClassCastException: cannot assign instance of scala.collection.immutable.List$SerializationProxy to field

2017-03-13 Thread srinivas thallam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907608#comment-15907608 ] srinivas thallam commented on SPARK-19938: -- The tests are working fine in IDE(intellij) but when

[jira] [Created] (SPARK-19938) java.lang.ClassCastException: cannot assign instance of scala.collection.immutable.List$SerializationProxy to field

2017-03-13 Thread srinivas thallam (JIRA)
srinivas thallam created SPARK-19938: Summary: java.lang.ClassCastException: cannot assign instance of scala.collection.immutable.List$SerializationProxy to field Key: SPARK-19938 URL:

[jira] [Comment Edited] (SPARK-19936) Page rank example takes long time to complete

2017-03-13 Thread Stan Teresen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907532#comment-15907532 ] Stan Teresen edited comment on SPARK-19936 at 3/13/17 2:53 PM: --- The cluster

[jira] [Commented] (SPARK-14503) spark.ml Scala API for FPGrowth

2017-03-13 Thread Maciej Szymkiewicz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907589#comment-15907589 ] Maciej Szymkiewicz commented on SPARK-14503: [~yuhaoyan] Sure thing. I'll try to do it later

[jira] [Commented] (SPARK-19933) TPCDS Q70 went wrong while explaining

2017-03-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907562#comment-15907562 ] Herman van Hovell commented on SPARK-19933: --- Here is a reproducible example: {noformat}

[jira] [Commented] (SPARK-17495) Hive hash implementation

2017-03-13 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907534#comment-15907534 ] Tejas Patil commented on SPARK-17495: - Yes. Thats a better way. At this point all datatypes are

[jira] [Commented] (SPARK-19936) Page rank example takes long time to complete

2017-03-13 Thread Stan Teresen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907532#comment-15907532 ] Stan Teresen commented on SPARK-19936: -- The cluster has plenty of the resource (3 r3.large AWS

[jira] [Reopened] (SPARK-17495) Hive hash implementation

2017-03-13 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil reopened SPARK-17495: - > Hive hash implementation > > > Key: SPARK-17495 >

[jira] [Commented] (SPARK-19933) TPCDS Q70 went wrong while explaining

2017-03-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907524#comment-15907524 ] Herman van Hovell commented on SPARK-19933: --- Yeah, this is on me. I added a rule that removes

[jira] [Commented] (SPARK-19937) Collect metrics of block sizes when shuffle.

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907481#comment-15907481 ] Apache Spark commented on SPARK-19937: -- User 'jinxing64' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19937) Collect metrics of block sizes when shuffle.

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19937: Assignee: Apache Spark > Collect metrics of block sizes when shuffle. >

[jira] [Assigned] (SPARK-19937) Collect metrics of block sizes when shuffle.

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19937: Assignee: (was: Apache Spark) > Collect metrics of block sizes when shuffle. >

[jira] [Commented] (SPARK-19936) Page rank example takes long time to complete

2017-03-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907475#comment-15907475 ] Sean Owen commented on SPARK-19936: --- That doesn't sound like a bug, but a problem in your environment.

[jira] [Updated] (SPARK-19936) Page rank example takes long time to complete

2017-03-13 Thread Stan Teresen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stan Teresen updated SPARK-19936: - Attachment: (was: pr.out) > Page rank example takes long time to complete >

[jira] [Updated] (SPARK-19936) Page rank example takes long time to complete

2017-03-13 Thread Stan Teresen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stan Teresen updated SPARK-19936: - Attachment: pr.out 2.stderr 1.stderr Page rank output and 2

[jira] [Issue Comment Deleted] (SPARK-19936) Page rank example takes long time to complete

2017-03-13 Thread Stan Teresen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stan Teresen updated SPARK-19936: - Comment: was deleted (was: Page Rank output) > Page rank example takes long time to complete >

[jira] [Created] (SPARK-19937) Collect metrics of block sizes when shuffle.

2017-03-13 Thread jin xing (JIRA)
jin xing created SPARK-19937: Summary: Collect metrics of block sizes when shuffle. Key: SPARK-19937 URL: https://issues.apache.org/jira/browse/SPARK-19937 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-19936) Page rank example takes long time to complete

2017-03-13 Thread Stan Teresen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19936?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stan Teresen updated SPARK-19936: - Attachment: pr.out Page Rank output > Page rank example takes long time to complete >

[jira] [Created] (SPARK-19936) Page rank example takes long time to complete

2017-03-13 Thread Stan Teresen (JIRA)
Stan Teresen created SPARK-19936: Summary: Page rank example takes long time to complete Key: SPARK-19936 URL: https://issues.apache.org/jira/browse/SPARK-19936 Project: Spark Issue Type:

[jira] [Commented] (SPARK-16394) Timestamp conversion error in pyspark.sql.Row because of timezones

2017-03-13 Thread Keith Bourgoin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907383#comment-15907383 ] Keith Bourgoin commented on SPARK-16394: PS: The above is from Spark 2.1.0 running on an Ubuntu

[jira] [Assigned] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19767: Assignee: Apache Spark > API Doc pages for Streaming with Kafka 0.10 not current >

[jira] [Assigned] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19767: Assignee: (was: Apache Spark) > API Doc pages for Streaming with Kafka 0.10 not

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907368#comment-15907368 ] Apache Spark commented on SPARK-19767: -- User 'nafshartous' has created a pull request for this

[jira] [Assigned] (SPARK-19925) SparkR spark.getSparkFiles fails on executor

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19925: Assignee: (was: Apache Spark) > SparkR spark.getSparkFiles fails on executor >

[jira] [Commented] (SPARK-19925) SparkR spark.getSparkFiles fails on executor

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907366#comment-15907366 ] Apache Spark commented on SPARK-19925: -- User 'yanboliang' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19925) SparkR spark.getSparkFiles fails on executor

2017-03-13 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19925: Assignee: Apache Spark > SparkR spark.getSparkFiles fails on executor >

[jira] [Resolved] (SPARK-19935) SparkSQL unsupports to create a hive table which is mapped for HBase table

2017-03-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19935. --- Resolution: Duplicate > SparkSQL unsupports to create a hive table which is mapped for HBase table >

[jira] [Assigned] (SPARK-19719) Structured Streaming write to Kafka

2017-03-13 Thread Prashant Sharma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19719?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Prashant Sharma reassigned SPARK-19719: --- Assignee: Tyson Condie > Structured Streaming write to Kafka >

[jira] [Comment Edited] (SPARK-19935) SparkSQL unsupports to create a hive table which is mapped for HBase table

2017-03-13 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907089#comment-15907089 ] Xiaochen Ouyang edited comment on SPARK-19935 at 3/13/17 9:47 AM: --

[jira] [Commented] (SPARK-19935) SparkSQL unsupports to create a hive table which is mapped for HBase table

2017-03-13 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907089#comment-15907089 ] Xiaochen Ouyang commented on SPARK-19935: - Throwing an operationNotAllowed exeception when we are

[jira] [Commented] (SPARK-19928) Incorrect error message when grouping function used with wrong types

2017-03-13 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907085#comment-15907085 ] Herman van Hovell commented on SPARK-19928: --- It does 'work' with one column (it gives a

[jira] [Commented] (SPARK-19935) SparkSQL unsupports to create a hive table which is mapped for HBase table

2017-03-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907076#comment-15907076 ] Sean Owen commented on SPARK-19935: --- What is the problem, and should this even be supported? >

[jira] [Updated] (SPARK-19934) code comments are not very clearly in BlackListTracker.scala

2017-03-13 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-19934: - Description: {code} def handleRemovedExecutor(executorId: String): Unit = { // We intentionally do

[jira] [Updated] (SPARK-19934) code comments are not very clearly in BlackListTracker.scala

2017-03-13 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang updated SPARK-19934: - Summary: code comments are not very clearly in BlackListTracker.scala (was: code comments not very

[jira] [Updated] (SPARK-19935) SparkSQL unsupports to create a hive table which is mapped for HBase table

2017-03-13 Thread Xiaochen Ouyang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiaochen Ouyang updated SPARK-19935: Description: SparkSQL unsupports the command as following: CREATE TABLE spark_test(key 

[jira] [Updated] (SPARK-19934) code comments not very clearly in BlackListTracker.scala

2017-03-13 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19934: -- Description: {code} def handleRemovedExecutor(executorId: String): Unit = { // We intentionally do

[jira] [Commented] (SPARK-19934) code comments not very clearly in BlackListTracker.scala

2017-03-13 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15907070#comment-15907070 ] zhoukang commented on SPARK-19934: -- Has update [~sowen] > code comments not very clearly in

  1   2   >