[jira] [Resolved] (SPARK-12655) GraphX does not unpersist RDDs

[ https://issues.apache.org/jira/browse/SPARK-12655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12655. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10713

[jira] [Updated] (SPARK-12655) GraphX does not unpersist RDDs

[ https://issues.apache.org/jira/browse/SPARK-12655?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12655: -- Assignee: Jason C Lee > GraphX does not unpersist RDDs > -- > >

[jira] [Commented] (SPARK-12675) Executor dies because of ClassCastException and causes timeout

[ https://issues.apache.org/jira/browse/SPARK-12675?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101567#comment-15101567 ] Himanshu Gupta commented on SPARK-12675: This issue is arising in Spark 1.5.2 as well.

[jira] [Commented] (SPARK-12739) Details of batch in Streaming tab uses two Duration columns

[ https://issues.apache.org/jira/browse/SPARK-12739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101644#comment-15101644 ] Jacek Laskowski commented on SPARK-12739: - Ok, I'll work on it. Thanks. > Details of batch in

[jira] [Resolved] (SPARK-2930) clarify docs on using webhdfs with spark.yarn.access.namenodes

[ https://issues.apache.org/jira/browse/SPARK-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-2930. -- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10699

[jira] [Assigned] (SPARK-12836) spark enable both driver run executor & write to HDFS

[ https://issues.apache.org/jira/browse/SPARK-12836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12836: Assignee: Apache Spark > spark enable both driver run executor & write to HDFS >

[jira] [Assigned] (SPARK-12836) spark enable both driver run executor & write to HDFS

[ https://issues.apache.org/jira/browse/SPARK-12836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12836: Assignee: (was: Apache Spark) > spark enable both driver run executor & write to HDFS

[jira] [Commented] (SPARK-12836) spark enable both driver run executor & write to HDFS

[ https://issues.apache.org/jira/browse/SPARK-12836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101630#comment-15101630 ] Apache Spark commented on SPARK-12836: -- User 'Astralidea' has created a pull request for this issue:

[jira] [Updated] (SPARK-2930) clarify docs on using webhdfs with spark.yarn.access.namenodes

[ https://issues.apache.org/jira/browse/SPARK-2930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2930: - Priority: Trivial (was: Minor) > clarify docs on using webhdfs with spark.yarn.access.namenodes >

[jira] [Created] (SPARK-12836) spark enable both driver run executor & write to HDFS

astralidea created SPARK-12836: -- Summary: spark enable both driver run executor & write to HDFS Key: SPARK-12836 URL: https://issues.apache.org/jira/browse/SPARK-12836 Project: Spark Issue

[jira] [Assigned] (SPARK-7683) Confusing behavior of fold function of RDD in pyspark

[ https://issues.apache.org/jira/browse/SPARK-7683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7683: --- Assignee: (was: Apache Spark) > Confusing behavior of fold function of RDD in pyspark >

[jira] [Commented] (SPARK-7683) Confusing behavior of fold function of RDD in pyspark

[ https://issues.apache.org/jira/browse/SPARK-7683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101728#comment-15101728 ] Apache Spark commented on SPARK-7683: - User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-7683) Confusing behavior of fold function of RDD in pyspark

[ https://issues.apache.org/jira/browse/SPARK-7683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-7683: --- Assignee: Apache Spark > Confusing behavior of fold function of RDD in pyspark >

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101742#comment-15101742 ] Krzysztof Gawryś commented on SPARK-10528: -- I have the same problem using spark 1.5.2 and

[jira] [Updated] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tien-Dung LE updated SPARK-12837: - Description: Executing a sql statement with a large number of partitions requires a high memory

[jira] [Created] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

Tien-Dung LE created SPARK-12837: Summary: Spark driver requires large memory space for serialized results even there are no data collected to the driver Key: SPARK-12837 URL:

[jira] [Commented] (SPARK-6817) DataFrame UDFs in R

[ https://issues.apache.org/jira/browse/SPARK-6817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101805#comment-15101805 ] Sun Rui commented on SPARK-6817: Spark is now supporting vectorized execution via Columnar batch. See

[jira] [Comment Edited] (SPARK-12786) Actor demo does not demonstrate usable code

[ https://issues.apache.org/jira/browse/SPARK-12786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101918#comment-15101918 ] Brian London edited comment on SPARK-12786 at 1/15/16 3:29 PM: --- Yeah,

[jira] [Commented] (SPARK-12786) Actor demo does not demonstrate usable code

[ https://issues.apache.org/jira/browse/SPARK-12786?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101918#comment-15101918 ] Brian London commented on SPARK-12786: -- Yeah, exactly. Because of the use of `AkkaUtil` the

[jira] [Commented] (SPARK-12834) Use type conversion instead of Ser/De of Pickle to transform JavaArray and JavaList

[ https://issues.apache.org/jira/browse/SPARK-12834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101809#comment-15101809 ] Apache Spark commented on SPARK-12834: -- User 'yinxusen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12834) Use type conversion instead of Ser/De of Pickle to transform JavaArray and JavaList

[ https://issues.apache.org/jira/browse/SPARK-12834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12834: Assignee: (was: Apache Spark) > Use type conversion instead of Ser/De of Pickle to

[jira] [Assigned] (SPARK-12834) Use type conversion instead of Ser/De of Pickle to transform JavaArray and JavaList

[ https://issues.apache.org/jira/browse/SPARK-12834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12834: Assignee: Apache Spark > Use type conversion instead of Ser/De of Pickle to transform

[jira] [Created] (SPARK-12838) fix a problem in PythonRDD.scala

zhanglu created SPARK-12838: --- Summary: fix a problem in PythonRDD.scala Key: SPARK-12838 URL: https://issues.apache.org/jira/browse/SPARK-12838 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-12838) fix a problem in PythonRDD.scala

[ https://issues.apache.org/jira/browse/SPARK-12838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-12838. --- Resolution: Invalid > fix a problem in PythonRDD.scala > - > >

[jira] [Updated] (SPARK-12834) Use type conversion instead of Ser/De of Pickle to transform JavaArray and JavaList

[ https://issues.apache.org/jira/browse/SPARK-12834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xusen Yin updated SPARK-12834: -- Description: According to the Ser/De code in Python side:

[jira] [Commented] (SPARK-11031) SparkR str() method on DataFrame objects

[ https://issues.apache.org/jira/browse/SPARK-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101936#comment-15101936 ] Shivaram Venkataraman commented on SPARK-11031: --- Resolved by

[jira] [Resolved] (SPARK-11031) SparkR str() method on DataFrame objects

[ https://issues.apache.org/jira/browse/SPARK-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-11031. --- Resolution: Fixed Fix Version/s: 2.0.0 1.6.1 >

[jira] [Updated] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-12807: --- Priority: Critical (was: Major) > Spark External Shuffle not working in Hadoop clusters

[jira] [Comment Edited] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093151#comment-15093151 ] Amir Gur edited comment on SPARK-10528 at 1/15/16 5:41 PM: --- Should this not be

[jira] [Commented] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102158#comment-15102158 ] Steve Loughran commented on SPARK-12807: We can replicate this intermittently. It all depends on

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102162#comment-15102162 ] Sean Owen commented on SPARK-10528: --- I'm not suggesting it's not a problem; I'm left wondering what

[jira] [Commented] (SPARK-12825) Spark-submit Jar URL loading fails on redirect

[ https://issues.apache.org/jira/browse/SPARK-12825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102163#comment-15102163 ] Alex Nederlof commented on SPARK-12825: --- It's a `307 temporary redirect` response code, returned by

[jira] [Comment Edited] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093151#comment-15093151 ] Amir Gur edited comment on SPARK-10528 at 1/15/16 5:45 PM: --- Should this not be

[jira] [Comment Edited] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093151#comment-15093151 ] Amir Gur edited comment on SPARK-10528 at 1/15/16 5:45 PM: --- Should this not be

[jira] [Commented] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102169#comment-15102169 ] Sean Owen commented on SPARK-12807: --- Yes in general I'd assume Spark's classes/dependencies are

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102178#comment-15102178 ] Amir Gur commented on SPARK-10528: -- Thanks [~kgawrys] for the confirmation. [~srowen], thanks for the

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102183#comment-15102183 ] Zhan Zhang commented on SPARK-5159: --- What happen if an user have a valid visit to a table, which will be

[jira] [Comment Edited] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102183#comment-15102183 ] Zhan Zhang edited comment on SPARK-5159 at 1/15/16 5:50 PM: What happen if an

[jira] [Comment Edited] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102178#comment-15102178 ] Amir Gur edited comment on SPARK-10528 at 1/15/16 5:54 PM: --- Thanks [~kgawrys]

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102190#comment-15102190 ] Sean Owen commented on SPARK-10528: --- I don't see a value in opening this, as there is no action in

[jira] [Comment Edited] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15093151#comment-15093151 ] Amir Gur edited comment on SPARK-10528 at 1/15/16 5:28 PM: --- Should this not be

[jira] [Commented] (SPARK-9625) SparkILoop creates sql context continuously, thousands of times

[ https://issues.apache.org/jira/browse/SPARK-9625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101992#comment-15101992 ] Alex Spencer commented on SPARK-9625: - I'm getting this same problem today - spark 1.3.0. I can't see

[jira] [Updated] (SPARK-11031) SparkR str() method on DataFrame objects

[ https://issues.apache.org/jira/browse/SPARK-11031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-11031: -- Assignee: Oscar D. Lara Yejas > SparkR str() method on DataFrame objects >

[jira] [Commented] (SPARK-6166) Add config to limit number of concurrent outbound connections for shuffle fetch

[ https://issues.apache.org/jira/browse/SPARK-6166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15101995#comment-15101995 ] Sanket Reddy commented on SPARK-6166: - Hi, I modified the code to fit the latest Spark build, I will

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102243#comment-15102243 ] Sean Owen commented on SPARK-10528: --- This JIRA tracks it already. More JIRAs don't help; they tend to

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102247#comment-15102247 ] Amir Gur commented on SPARK-10528: -- Sure, that's fine, let's find the spark level solution on this one

[jira] [Comment Edited] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102212#comment-15102212 ] Amir Gur edited comment on SPARK-10528 at 1/15/16 6:24 PM: --- As long as there is

[jira] [Commented] (SPARK-12783) Dataset map serialization error

[ https://issues.apache.org/jira/browse/SPARK-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102351#comment-15102351 ] Muthu Jayakumar commented on SPARK-12783: - I tried the following, but got similar error...

[jira] [Commented] (SPARK-12783) Dataset map serialization error

[ https://issues.apache.org/jira/browse/SPARK-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102361#comment-15102361 ] kevin yu commented on SPARK-12783: -- Hello Muthu: do the import first, it seems working. scala> import

[jira] [Created] (SPARK-12840) Support pass any object into codegen as reference

Davies Liu created SPARK-12840: -- Summary: Support pass any object into codegen as reference Key: SPARK-12840 URL: https://issues.apache.org/jira/browse/SPARK-12840 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12835) StackOverflowError when aggregating over column from window function

[ https://issues.apache.org/jira/browse/SPARK-12835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102358#comment-15102358 ] Herman van Hovell commented on SPARK-12835: --- Kalle, you are not wrong, this should work.

[jira] [Commented] (SPARK-10528) spark-shell throws java.lang.RuntimeException: The root scratch dir: /tmp/hive on HDFS should be writable.

[ https://issues.apache.org/jira/browse/SPARK-10528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102212#comment-15102212 ] Amir Gur commented on SPARK-10528: -- As long as there is no spark workaround + nor root-cause-hive-level

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102267#comment-15102267 ] Greg Senia commented on SPARK-5159: --- [~zhanzhang], [~luciano resende] and [~ilovesoup] I think this is

[jira] [Created] (SPARK-12839) Implement CoSelect for Feature Selection and Instance Selection

Morgan Funtowicz created SPARK-12839: Summary: Implement CoSelect for Feature Selection and Instance Selection Key: SPARK-12839 URL: https://issues.apache.org/jira/browse/SPARK-12839 Project:

[jira] [Commented] (SPARK-12839) Implement CoSelect for Feature Selection and Instance Selection

[ https://issues.apache.org/jira/browse/SPARK-12839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102295#comment-15102295 ] Morgan Funtowicz commented on SPARK-12839: -- If you feel interested by this feature, I would like

[jira] [Comment Edited] (SPARK-12783) Dataset map serialization error

[ https://issues.apache.org/jira/browse/SPARK-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102351#comment-15102351 ] Muthu Jayakumar edited comment on SPARK-12783 at 1/15/16 7:34 PM: -- I

[jira] [Commented] (SPARK-12833) Initial import of databricks/spark-csv

[ https://issues.apache.org/jira/browse/SPARK-12833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102407#comment-15102407 ] Apache Spark commented on SPARK-12833: -- User 'yhuai' has created a pull request for this issue:

[jira] [Commented] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102430#comment-15102430 ] Sean Owen commented on SPARK-12807: --- Are you asking if it's possible, a possible explanation, a

[jira] [Resolved] (SPARK-12667) Remove block manager's internal "external block store" API

[ https://issues.apache.org/jira/browse/SPARK-12667?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-12667. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 10752

[jira] [Commented] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102416#comment-15102416 ] Maciej Bryński commented on SPARK-12807: Sean, Maybe it's possible to compile YARN Shuffle with

[jira] [Created] (SPARK-12842) Add Hadoop 2.7 build profile

Josh Rosen created SPARK-12842: -- Summary: Add Hadoop 2.7 build profile Key: SPARK-12842 URL: https://issues.apache.org/jira/browse/SPARK-12842 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102441#comment-15102441 ] Maciej Bryński commented on SPARK-12807: I'm asking if it's possible. About running Spark

[jira] [Resolved] (SPARK-12833) Initial import of databricks/spark-csv

[ https://issues.apache.org/jira/browse/SPARK-12833?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12833. - Resolution: Fixed Fix Version/s: 2.0.0 > Initial import of databricks/spark-csv >

[jira] [Commented] (SPARK-12835) StackOverflowError when aggregating over column from window function

[ https://issues.apache.org/jira/browse/SPARK-12835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102386#comment-15102386 ] Herman van Hovell commented on SPARK-12835: --- I can reproduce your problem with the following

[jira] [Created] (SPARK-12841) UnresolvedException with cast

Michael Armbrust created SPARK-12841: Summary: UnresolvedException with cast Key: SPARK-12841 URL: https://issues.apache.org/jira/browse/SPARK-12841 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-12842) Add Hadoop 2.7 build profile

[ https://issues.apache.org/jira/browse/SPARK-12842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102424#comment-15102424 ] Apache Spark commented on SPARK-12842: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102441#comment-15102441 ] Maciej Bryński edited comment on SPARK-12807 at 1/15/16 8:43 PM: - I'm

[jira] [Created] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

Maciej Bryński created SPARK-12843: -- Summary: Spark should avoid scanning all partitions when limit is set Key: SPARK-12843 URL: https://issues.apache.org/jira/browse/SPARK-12843 Project: Spark

[jira] [Updated] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-12843: --- Description: SQL Query: {code} select * from table limit 100 {code} force Spark to scan all

[jira] [Updated] (SPARK-12701) Logging FileAppender should use join to ensure thread is finished

[ https://issues.apache.org/jira/browse/SPARK-12701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-12701: -- Fix Version/s: 1.6.1 > Logging FileAppender should use join to ensure thread is finished >

[jira] [Commented] (SPARK-10985) Avoid passing evicted blocks throughout BlockManager / CacheManager

[ https://issues.apache.org/jira/browse/SPARK-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102477#comment-15102477 ] Apache Spark commented on SPARK-10985: -- User 'JoshRosen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-10985) Avoid passing evicted blocks throughout BlockManager / CacheManager

[ https://issues.apache.org/jira/browse/SPARK-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10985: Assignee: (was: Apache Spark) > Avoid passing evicted blocks throughout BlockManager

[jira] [Assigned] (SPARK-10985) Avoid passing evicted blocks throughout BlockManager / CacheManager

[ https://issues.apache.org/jira/browse/SPARK-10985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-10985: Assignee: Apache Spark > Avoid passing evicted blocks throughout BlockManager /

[jira] [Commented] (SPARK-12783) Dataset map serialization error

[ https://issues.apache.org/jira/browse/SPARK-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102482#comment-15102482 ] Muthu Jayakumar commented on SPARK-12783: - Hello Kevin, Here is what I am seeing... from shell:

[jira] [Comment Edited] (SPARK-12783) Dataset map serialization error

[ https://issues.apache.org/jira/browse/SPARK-12783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102351#comment-15102351 ] Muthu Jayakumar edited comment on SPARK-12783 at 1/15/16 9:09 PM: -- I

[jira] [Commented] (SPARK-12624) When schema is specified, we should treat undeclared fields as null (in Python)

[ https://issues.apache.org/jira/browse/SPARK-12624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102473#comment-15102473 ] Maciej Bryński commented on SPARK-12624: [~davies] Isn't related to my comment here:

[jira] [Comment Edited] (SPARK-12624) When schema is specified, we should treat undeclared fields as null (in Python)

[ https://issues.apache.org/jira/browse/SPARK-12624?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102473#comment-15102473 ] Maciej Bryński edited comment on SPARK-12624 at 1/15/16 9:17 PM: -

[jira] [Commented] (SPARK-12835) StackOverflowError when aggregating over column from window function

[ https://issues.apache.org/jira/browse/SPARK-12835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102491#comment-15102491 ] Kalle Jepsen commented on SPARK-12835: -- The [traceback|http://pastebin.com/pRRCAben] really is

[jira] [Created] (SPARK-12844) Spark documentation should be more precise about the algebraic properties of functions in various transformations

Jimmy Lin created SPARK-12844: - Summary: Spark documentation should be more precise about the algebraic properties of functions in various transformations Key: SPARK-12844 URL:

[jira] [Created] (SPARK-12845) During join Spark should pushdown predicates to both tables

Maciej Bryński created SPARK-12845: -- Summary: During join Spark should pushdown predicates to both tables Key: SPARK-12845 URL: https://issues.apache.org/jira/browse/SPARK-12845 Project: Spark

[jira] [Updated] (SPARK-12845) During join Spark should pushdown predicates to both tables

[ https://issues.apache.org/jira/browse/SPARK-12845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-12845: --- Description: I have following issue. I'm connecting two tables with where condition {code}

[jira] [Updated] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-12843: --- Issue Type: Bug (was: Improvement) > Spark should avoid scanning all partitions when limit

[jira] [Updated] (SPARK-12030) Incorrect results when aggregate joined data

[ https://issues.apache.org/jira/browse/SPARK-12030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-12030: --- Attachment: (was: t1.tar.gz) > Incorrect results when aggregate joined data >

[jira] [Updated] (SPARK-12030) Incorrect results when aggregate joined data

[ https://issues.apache.org/jira/browse/SPARK-12030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-12030: --- Attachment: (was: spark.jpg) > Incorrect results when aggregate joined data >

[jira] [Updated] (SPARK-12030) Incorrect results when aggregate joined data

[ https://issues.apache.org/jira/browse/SPARK-12030?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej Bryński updated SPARK-12030: --- Attachment: (was: t2.tar.gz) > Incorrect results when aggregate joined data >

[jira] [Commented] (SPARK-12835) StackOverflowError when aggregating over column from window function

[ https://issues.apache.org/jira/browse/SPARK-12835?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102532#comment-15102532 ] Herman van Hovell commented on SPARK-12835: --- Thanks for that. The

[jira] [Updated] (SPARK-12149) Executor UI improvement suggestions - Color UI

[ https://issues.apache.org/jira/browse/SPARK-12149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-12149: -- Assignee: Alex Bozarth > Executor UI improvement suggestions - Color UI >

[jira] [Resolved] (SPARK-12716) Executor UI improvement suggestions - Totals

[ https://issues.apache.org/jira/browse/SPARK-12716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-12716. --- Resolution: Fixed Assignee: Alex Bozarth Fix Version/s: 2.0.0 > Executor UI

[jira] [Resolved] (SPARK-11925) Add PySpark missing methods for ml.feature during Spark 1.6 QA

[ https://issues.apache.org/jira/browse/SPARK-11925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-11925. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 9908

[jira] [Commented] (SPARK-12847) Remove StreamingListenerBus and post all Streaming events to the same thread as Spark events

[ https://issues.apache.org/jira/browse/SPARK-12847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102740#comment-15102740 ] Shixiong Zhu commented on SPARK-12847: -- Ah, I think this one should be a sub-task. Let me change it.

[jira] [Commented] (SPARK-12848) Parse number as decimal

[ https://issues.apache.org/jira/browse/SPARK-12848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102763#comment-15102763 ] Herman van Hovell commented on SPARK-12848: --- Assuming that we are talking about literals here.

[jira] [Commented] (SPARK-12840) Support pass any object into codegen as reference

[ https://issues.apache.org/jira/browse/SPARK-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102629#comment-15102629 ] Apache Spark commented on SPARK-12840: -- User 'davies' has created a pull request for this issue:

[jira] [Assigned] (SPARK-12840) Support pass any object into codegen as reference

[ https://issues.apache.org/jira/browse/SPARK-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12840: Assignee: Apache Spark (was: Davies Liu) > Support pass any object into codegen as

[jira] [Assigned] (SPARK-12840) Support pass any object into codegen as reference

[ https://issues.apache.org/jira/browse/SPARK-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12840: Assignee: Davies Liu (was: Apache Spark) > Support pass any object into codegen as

[jira] [Commented] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102654#comment-15102654 ] Sean Owen commented on SPARK-12807: --- I see, it's only the shuffle and only 1.6, and only happens to

[jira] [Updated] (SPARK-12840) Support passing arbitrary objects (not just expressions) into code generated classes

[ https://issues.apache.org/jira/browse/SPARK-12840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-12840: Description: As of now, our code generator only allows passing Expression objects into the

[jira] [Commented] (SPARK-12807) Spark External Shuffle not working in Hadoop clusters with Jackson 2.2.3

[ https://issues.apache.org/jira/browse/SPARK-12807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15102883#comment-15102883 ] Steve Loughran commented on SPARK-12807: There's a PR to shade in trunk; I'm going to do a 1.6 PR

[jira] [Closed] (SPARK-12704) we may repartition a relation even it's not needed

[ https://issues.apache.org/jira/browse/SPARK-12704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-12704. --- Resolution: Later Closing as later. We will revisit this when the time comes. > we may repartition

[jira] [Created] (SPARK-12851) Add the ability to understand tables bucketed by Hive

Reynold Xin created SPARK-12851: --- Summary: Add the ability to understand tables bucketed by Hive Key: SPARK-12851 URL: https://issues.apache.org/jira/browse/SPARK-12851 Project: Spark Issue

[jira] [Created] (SPARK-12852) Support create table DDL with bucketing

Reynold Xin created SPARK-12852: --- Summary: Support create table DDL with bucketing Key: SPARK-12852 URL: https://issues.apache.org/jira/browse/SPARK-12852 Project: Spark Issue Type: Sub-task

  1   2   >