[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-10 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324352#comment-15324352 ] Adam Roberts commented on SPARK-15822: -- The comment for baseOffset which has the corrupt value

[jira] [Assigned] (SPARK-15873) JdbcRDD to support more bound types other than Long and allow multiple bound occurrence for subqueries.

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15873: Assignee: Apache Spark > JdbcRDD to support more bound types other than Long and allow

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324475#comment-15324475 ] Adam Roberts commented on SPARK-15822: -- Herman, here's the application, note my HashedRelation

[jira] [Commented] (SPARK-15871) Add assertNotPartitioned check in DataFrameWriter

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324290#comment-15324290 ] Apache Spark commented on SPARK-15871: -- User 'lw-lin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15871) Add assertNotPartitioned check in DataFrameWriter

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15871: Assignee: Apache Spark > Add assertNotPartitioned check in DataFrameWriter >

[jira] [Commented] (SPARK-13587) Support virtualenv in PySpark

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324318#comment-15324318 ] Apache Spark commented on SPARK-13587: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Commented] (SPARK-15874) HBase rowkey optimization support for Hbase-handler

2016-06-10 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324385#comment-15324385 ] Weichen Xu commented on SPARK-15874: [~rxin]What do you think about it ? > HBase rowkey optimization

[jira] [Updated] (SPARK-15869) HTTP 500 and NPE on streaming batch details page

2016-06-10 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-15869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej BryƄski updated SPARK-15869: --- Component/s: Web UI > HTTP 500 and NPE on streaming batch details page >

[jira] [Commented] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324268#comment-15324268 ] Sean Owen commented on SPARK-15781: --- [~andrewor14] do you know? if no reply soon, let's just proceed

[jira] [Resolved] (SPARK-15837) PySpark ML Word2Vec should support maxSentenceLength

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-15837. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13578

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-10 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324331#comment-15324331 ] Pete Robbins commented on SPARK-15822: -- I'm still looking into this tracing back through the code

[jira] [Commented] (SPARK-15867) TABLESAMPLE BUCKET semantics don't match Hive's

2016-06-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1532#comment-1532 ] Herman van Hovell commented on SPARK-15867: --- This would change the behavior in comparison with

[jira] [Created] (SPARK-15870) DataFrame can't execute after uncacheTable.

2016-06-10 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-15870: - Summary: DataFrame can't execute after uncacheTable. Key: SPARK-15870 URL: https://issues.apache.org/jira/browse/SPARK-15870 Project: Spark Issue Type:

[jira] [Updated] (SPARK-15837) PySpark ML Word2Vec should support maxSentenceLength

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15837: -- Assignee: Weichen Xu > PySpark ML Word2Vec should support maxSentenceLength >

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324455#comment-15324455 ] Herman van Hovell commented on SPARK-15822: --- [~aroberts][~robbinspg] Can you share a

[jira] [Updated] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pete Robbins updated SPARK-15822: - Summary: segmentation violation in o.a.s.unsafe.types.UTF8String (was: segmentation violation

[jira] [Comment Edited] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324455#comment-15324455 ] Herman van Hovell edited comment on SPARK-15822 at 6/10/16 1:30 PM:

[jira] [Issue Comment Deleted] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15781: -- Comment: was deleted (was: By launching a session with {{SPARK_WORKER_INSTANCES}} set or just a

[jira] [Commented] (SPARK-15869) HTTP 500 and NPE on streaming batch details page

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15869?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324264#comment-15324264 ] Sean Owen commented on SPARK-15869: --- Presumably one of the OutputOpIdAndSparkJobId has a null

[jira] [Issue Comment Deleted] (SPARK-15781) Misleading deprecated property in standalone cluster configuration documentation

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15781?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-15781: -- Comment: was deleted (was: Oh I'm sorry, I put this comment on entirely the wrong JIRA -- too many

[jira] [Updated] (SPARK-15871) Add assertNotPartitioned check in DataFrameWriter

2016-06-10 Thread Liwei Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liwei Lin updated SPARK-15871: -- Description: Sometimes it doesn't make sense to specify partitioning parameters, e.g. when we write

[jira] [Created] (SPARK-15871) Add assertNotPartitioned check in DataFrameWriter

2016-06-10 Thread Liwei Lin (JIRA)
Liwei Lin created SPARK-15871: - Summary: Add assertNotPartitioned check in DataFrameWriter Key: SPARK-15871 URL: https://issues.apache.org/jira/browse/SPARK-15871 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-13587) Support virtualenv in PySpark

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-13587: Assignee: Apache Spark > Support virtualenv in PySpark > - >

[jira] [Updated] (SPARK-15874) HBase rowkey optimization support for Hbase-Storage-handler

2016-06-10 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15874: --- Summary: HBase rowkey optimization support for Hbase-Storage-handler (was: HBase rowkey

[jira] [Commented] (SPARK-15873) JdbcRDD to support more bound types other than Long and allow multiple bound occurrence for subqueries.

2016-06-10 Thread Cheng Wei (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324408#comment-15324408 ] Cheng Wei commented on SPARK-15873: --- In the original JdbcRDD, it only supports Long bound type and does

[jira] [Assigned] (SPARK-15870) DataFrame can't execute after uncacheTable.

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15870: Assignee: (was: Apache Spark) > DataFrame can't execute after uncacheTable. >

[jira] [Commented] (SPARK-15870) DataFrame can't execute after uncacheTable.

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324274#comment-15324274 ] Apache Spark commented on SPARK-15870: -- User 'ueshin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15870) DataFrame can't execute after uncacheTable.

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15870: Assignee: Apache Spark > DataFrame can't execute after uncacheTable. >

[jira] [Commented] (SPARK-13587) Support virtualenv in PySpark

2016-06-10 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324323#comment-15324323 ] Jeff Zhang commented on SPARK-13587: Sorry, guys, I am busy on other stuff recently and late for

[jira] [Comment Edited] (SPARK-13587) Support virtualenv in PySpark

2016-06-10 Thread Jeff Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13587?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324323#comment-15324323 ] Jeff Zhang edited comment on SPARK-13587 at 6/10/16 11:43 AM: -- Sorry, guys,

[jira] [Comment Edited] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-10 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324352#comment-15324352 ] Adam Roberts edited comment on SPARK-15822 at 6/10/16 12:16 PM: The

[jira] [Updated] (SPARK-15872) Dataset of Array of Custom case class throws MissingRequirementError

2016-06-10 Thread Petr Votava (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Petr Votava updated SPARK-15872: Description: example: {code:scala} import org.apache.spark.SparkContext import

[jira] [Commented] (SPARK-15873) JdbcRDD to support more bound types other than Long and allow multiple bound occurrence for subqueries.

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324388#comment-15324388 ] Apache Spark commented on SPARK-15873: -- User 'weicheng113' has created a pull request for this

[jira] [Assigned] (SPARK-15873) JdbcRDD to support more bound types other than Long and allow multiple bound occurrence for subqueries.

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15873: Assignee: (was: Apache Spark) > JdbcRDD to support more bound types other than Long

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String with spark.memory.offHeap.enabled=true

2016-06-10 Thread Adam Roberts (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324341#comment-15324341 ] Adam Roberts commented on SPARK-15822: -- Note that this segv happens regardless of

[jira] [Created] (SPARK-15872) Dataset of Array of Custom case class throws MissingRequirementError

2016-06-10 Thread Petr Votava (JIRA)
Petr Votava created SPARK-15872: --- Summary: Dataset of Array of Custom case class throws MissingRequirementError Key: SPARK-15872 URL: https://issues.apache.org/jira/browse/SPARK-15872 Project: Spark

[jira] [Updated] (SPARK-15874) HBase rowkey optimization support for Hbase-Storage-handler

2016-06-10 Thread Weichen Xu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Weichen Xu updated SPARK-15874: --- Description: Currently, Spark-SQL use `org.apache.hadoop.hive.hbase.HBaseStorageHandler` for Hbase

[jira] [Commented] (SPARK-15851) Spark 2.0 does not compile in Windows 7

2016-06-10 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324452#comment-15324452 ] Thomas Graves commented on SPARK-15851: --- Can we also please document that windows is support build

[jira] [Updated] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pete Robbins updated SPARK-15822: - Description: Executors fail with segmentation violation while running application with

[jira] [Commented] (SPARK-15861) pyspark mapPartitions with none generator functions / functors

2016-06-10 Thread Greg Bowyer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324865#comment-15324865 ] Greg Bowyer commented on SPARK-15861: - So the documentation and use case really suggests that the

[jira] [Updated] (SPARK-15861) pyspark mapPartitions with none generator functions / functors

2016-06-10 Thread Greg Bowyer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Bowyer updated SPARK-15861: Description: Hi all, it appears that the method `rdd.mapPartitions` does odd things if it is fed

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324907#comment-15324907 ] Davies Liu commented on SPARK-15822: SortMergeJoin assume that the keys do not have null in them (we

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.10 Consumer API

2016-06-10 Thread Luciano Resende (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324652#comment-15324652 ] Luciano Resende commented on SPARK-12177: - At Apache Bahir we are working on a release based on

[jira] [Created] (SPARK-15877) DataSource executed twice when using ORDER BY

2016-06-10 Thread Matthew Livesey (JIRA)
Matthew Livesey created SPARK-15877: --- Summary: DataSource executed twice when using ORDER BY Key: SPARK-15877 URL: https://issues.apache.org/jira/browse/SPARK-15877 Project: Spark Issue

[jira] [Updated] (SPARK-15723) SimpleDateParamSuite test is locale-fragile and relies on deprecated short TZ name

2016-06-10 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-15723: --- Issue Type: Sub-task (was: Bug) Parent: SPARK-15834 > SimpleDateParamSuite test is

[jira] [Commented] (SPARK-15849) FileNotFoundException on _temporary while doing saveAsTable to S3

2016-06-10 Thread Sandeep (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324887#comment-15324887 ] Sandeep commented on SPARK-15849: - I had a discussion on spark-user/dev mailing list which indicated that

[jira] [Commented] (SPARK-15613) Incorrect days to millis conversion

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324648#comment-15324648 ] Sean Owen commented on SPARK-15613: --- Possibly related https://issues.apache.org/jira/browse/SPARK-15723

[jira] [Commented] (SPARK-15878) Fix test cleanup in EventLoggingListenerSuite and ReplayListenerSuite

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324753#comment-15324753 ] Apache Spark commented on SPARK-15878: -- User 'squito' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15878) Fix test cleanup in EventLoggingListenerSuite and ReplayListenerSuite

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15878: Assignee: Apache Spark (was: Imran Rashid) > Fix test cleanup in

[jira] [Commented] (SPARK-15829) spark master webpage links to application UI broke when running in cluster mode

2016-06-10 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324789#comment-15324789 ] Xin Ren commented on SPARK-15829: - Hi Andy, maybe you want to check your port configuration to make sure

[jira] [Commented] (SPARK-15861) pyspark mapPartitions with none generator functions / functors

2016-06-10 Thread Greg Bowyer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324917#comment-15324917 ] Greg Bowyer commented on SPARK-15861: - Minor patch for usability here (its not a great patch)

[jira] [Updated] (SPARK-15861) pyspark mapPartitions with none generator functions / functors

2016-06-10 Thread Greg Bowyer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Greg Bowyer updated SPARK-15861: Flags: Patch > pyspark mapPartitions with none generator functions / functors >

[jira] [Assigned] (SPARK-15878) Fix test cleanup in EventLoggingListenerSuite and ReplayListenerSuite

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15878: Assignee: Imran Rashid (was: Apache Spark) > Fix test cleanup in

[jira] [Assigned] (SPARK-15865) Blacklist should not result in job hanging with less than 4 executors

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15865: Assignee: Apache Spark (was: Imran Rashid) > Blacklist should not result in job hanging

[jira] [Assigned] (SPARK-15865) Blacklist should not result in job hanging with less than 4 executors

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15865: Assignee: Imran Rashid (was: Apache Spark) > Blacklist should not result in job hanging

[jira] [Commented] (SPARK-15865) Blacklist should not result in job hanging with less than 4 executors

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324805#comment-15324805 ] Apache Spark commented on SPARK-15865: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324875#comment-15324875 ] Davies Liu commented on SPARK-15822: Could you try to disable whole-stage codegen to see whether this

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325093#comment-15325093 ] Pete Robbins commented on SPARK-15822: -- How do I disable whole-stage codegen? > segmentation

[jira] [Commented] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325100#comment-15325100 ] Herman van Hovell commented on SPARK-15822: --- Using spark session:

[jira] [Updated] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15856: --- Assignee: Wenchen Fan > Revert API breaking changes made in DataFrameReader.text and

[jira] [Comment Edited] (SPARK-15822) segmentation violation in o.a.s.unsafe.types.UTF8String

2016-06-10 Thread Pete Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15822?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325093#comment-15325093 ] Pete Robbins edited comment on SPARK-15822 at 6/10/16 7:17 PM: --- How do I

[jira] [Assigned] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15856: Assignee: Apache Spark > Revert API breaking changes made in DataFrameReader.text and

[jira] [Commented] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324976#comment-15324976 ] Apache Spark commented on SPARK-15856: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15856: Assignee: (was: Apache Spark) > Revert API breaking changes made in

[jira] [Commented] (SPARK-15849) FileNotFoundException on _temporary while doing saveAsTable to S3

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324993#comment-15324993 ] Sean Owen commented on SPARK-15849: --- If I'm right, I don't know if there is a solution except to not

[jira] [Resolved] (SPARK-15753) Move some Analyzer stuff to Analyzer from DataFrameWriter

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian resolved SPARK-15753. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13496

[jira] [Commented] (SPARK-15856) Revert API breaking changes made in DataFrameReader.text and SQLContext.range

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325056#comment-15325056 ] Apache Spark commented on SPARK-15856: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Commented] (SPARK-15086) Update Java API once the Scala one is finalized

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325084#comment-15325084 ] Sean Owen commented on SPARK-15086: --- Good news this might not be an issue: - Deprecate intAccumulator

[jira] [Resolved] (SPARK-15812) Allow sorting on aggregated streaming dataframe when the output mode is Complete

2016-06-10 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-15812. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13549

[jira] [Updated] (SPARK-15753) Move some Analyzer stuff to Analyzer from DataFrameWriter

2016-06-10 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15753?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-15753: --- Assignee: Liang-Chi Hsieh > Move some Analyzer stuff to Analyzer from DataFrameWriter >

[jira] [Commented] (SPARK-15829) spark master webpage links to application UI broke when running in cluster mode

2016-06-10 Thread Andrew Davidson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325001#comment-15325001 ] Andrew Davidson commented on SPARK-15829: - Hi Xin I ran netstat on my master. I do not think the

[jira] [Created] (SPARK-15879) Update logo in UI and docs to add "Apache"

2016-06-10 Thread Matei Zaharia (JIRA)
Matei Zaharia created SPARK-15879: - Summary: Update logo in UI and docs to add "Apache" Key: SPARK-15879 URL: https://issues.apache.org/jira/browse/SPARK-15879 Project: Spark Issue Type:

[jira] [Updated] (SPARK-15867) TABLESAMPLE BUCKET semantics don't match Hive's

2016-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-15867: -- Affects Version/s: 1.6.0 > TABLESAMPLE BUCKET semantics don't match Hive's >

[jira] [Commented] (SPARK-15867) TABLESAMPLE BUCKET semantics don't match Hive's

2016-06-10 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324960#comment-15324960 ] Andrew Or commented on SPARK-15867: --- I think we should fix it, though looks like it's been an issue for

[jira] [Resolved] (SPARK-15866) Rename listAccumulator collectionAccumulator

2016-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15866. - Resolution: Fixed Fix Version/s: 2.0.0 > Rename listAccumulator collectionAccumulator >

[jira] [Commented] (SPARK-15861) pyspark mapPartitions with none generator functions / functors

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325007#comment-15325007 ] Sean Owen commented on SPARK-15861: --- Got it. That does look odd. I doubt the explanation is that

[jira] [Commented] (SPARK-15829) spark master webpage links to application UI broke when running in cluster mode

2016-06-10 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325073#comment-15325073 ] Xin Ren commented on SPARK-15829: - sorry Andy, my bad. I'm running on port 7077 and client mode. > spark

[jira] [Commented] (SPARK-15086) Update Java API once the Scala one is finalized

2016-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15086?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325094#comment-15325094 ] Reynold Xin commented on SPARK-15086: - Yea it should be fine now to just ask people to use the Scala

[jira] [Commented] (SPARK-15868) Executors table in Executors tab should sort Executor IDs in numerical order (not alphabetical order)

2016-06-10 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325103#comment-15325103 ] Alex Bozarth commented on SPARK-15868: -- I've run into this before and I thought I had fixed it in

[jira] [Commented] (SPARK-15861) pyspark mapPartitions with none generator functions / functors

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324131#comment-15324131 ] Sean Owen commented on SPARK-15861: --- I may be missing the punchline, but what is the issue here? the

[jira] [Commented] (SPARK-15849) FileNotFoundException on _temporary while doing saveAsTable to S3

2016-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15324132#comment-15324132 ] Sean Owen commented on SPARK-15849: --- This is a duplicate of SPARK-2984 and we should probably keep the

[jira] [Commented] (SPARK-13207) _SUCCESS should not break partition discovery

2016-06-10 Thread Simeon Simeonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325718#comment-15325718 ] Simeon Simeonov commented on SPARK-13207: - [~yhuai] The PR associated with that ticket explicitly

[jira] [Created] (SPARK-15894) Add doc to control #partition for input files

2016-06-10 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-15894: Summary: Add doc to control #partition for input files Key: SPARK-15894 URL: https://issues.apache.org/jira/browse/SPARK-15894 Project: Spark Issue

[jira] [Commented] (SPARK-15894) Add doc to control #partition for input files

2016-06-10 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325726#comment-15325726 ] Takeshi Yamamuro commented on SPARK-15894: -- The patch is like

[jira] [Commented] (SPARK-15894) Add doc to control #partition for input files

2016-06-10 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325729#comment-15325729 ] Takeshi Yamamuro commented on SPARK-15894: -- cc: [~rxin] [~davies] > Add doc to control

[jira] [Updated] (SPARK-15888) Python UDF over aggregate fails

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15888: --- Summary: Python UDF over aggregate fails (was: UDF fails in Python) > Python UDF over aggregate

[jira] [Commented] (SPARK-15888) Python UDF over aggregate fails

2016-06-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325733#comment-15325733 ] Davies Liu commented on SPARK-15888: After some investigation, it turned out to be that the Python

[jira] [Commented] (SPARK-15672) R programming guide update

2016-06-10 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325145#comment-15325145 ] Shivaram Venkataraman commented on SPARK-15672: --- Yeah we can leave out gapply from the

[jira] [Resolved] (SPARK-15738) PySpark ml.feature RFormula missing string representation displaying formula

2016-06-10 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang resolved SPARK-15738. - Resolution: Fixed Fix Version/s: 2.0.0 > PySpark ml.feature RFormula missing string

[jira] [Created] (SPARK-15881) Update microbenchmark results

2016-06-10 Thread Eric Liang (JIRA)
Eric Liang created SPARK-15881: -- Summary: Update microbenchmark results Key: SPARK-15881 URL: https://issues.apache.org/jira/browse/SPARK-15881 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-15883) Fix broken links on MLLIB documentations

2016-06-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15883?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-15883: -- Description: This issue fixes all broken links on Spark 2.0 preview MLLib documents. Also,

[jira] [Assigned] (SPARK-15881) Update microbenchmark results

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15881: Assignee: Apache Spark > Update microbenchmark results > - >

[jira] [Commented] (SPARK-15751) Add generateAssociationRules in fpm in pyspark

2016-06-10 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15751?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325307#comment-15325307 ] Joseph K. Bradley commented on SPARK-15751: --- There isn't a JIRA for this AFAIK, but I think we

[jira] [Commented] (SPARK-15879) Update logo in UI and docs to add "Apache"

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15879?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15325313#comment-15325313 ] Apache Spark commented on SPARK-15879: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-15879) Update logo in UI and docs to add "Apache"

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15879: Assignee: Apache Spark > Update logo in UI and docs to add "Apache" >

[jira] [Assigned] (SPARK-15879) Update logo in UI and docs to add "Apache"

2016-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-15879: Assignee: (was: Apache Spark) > Update logo in UI and docs to add "Apache" >

[jira] [Resolved] (SPARK-15766) R should export is.nan

2016-06-10 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15766?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-15766. --- Resolution: Fixed Assignee: Miao Wang Fix Version/s: 2.0.0

[jira] [Issue Comment Deleted] (SPARK-15880) PREGEL Based Semi-Clustering Algorithm Implementation using Spark GraphX API

2016-06-10 Thread R J (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] R J updated SPARK-15880: Comment: was deleted (was: Algorithm is explained in the file) > PREGEL Based Semi-Clustering Algorithm

[jira] [Resolved] (SPARK-15875) Avoid using Seq.length == 0 and Seq.lenth > 0. Use Seq.isEmpty and Seq.nonEmpty instead.

2016-06-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-15875. - Resolution: Fixed Assignee: Yang Wang Fix Version/s: 2.0.0 > Avoid using

[jira] [Updated] (SPARK-15738) PySpark ml.feature RFormula missing string representation displaying formula

2016-06-10 Thread Yanbo Liang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yanbo Liang updated SPARK-15738: Assignee: Bryan Cutler > PySpark ml.feature RFormula missing string representation displaying

  1   2   3   >