[jira] [Commented] (SPARK-6951) History server slow startup if the event log directory is large

[ https://issues.apache.org/jira/browse/SPARK-6951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900871#comment-15900871 ] Cui Xixin commented on SPARK-6951: -- In my case,the inprogress file is the main reason, so if the

[jira] [Comment Edited] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900821#comment-15900821 ] Saisai Shao edited comment on SPARK-19812 at 3/8/17 7:42 AM: - [~tgraves], I'm

[jira] [Comment Edited] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900821#comment-15900821 ] Saisai Shao edited comment on SPARK-19812 at 3/8/17 7:40 AM: - [~tgraves], I'm

[jira] [Commented] (SPARK-13969) Extend input format that feature hashing can handle

[ https://issues.apache.org/jira/browse/SPARK-13969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900825#comment-15900825 ] Nick Pentreath commented on SPARK-13969: I think {{HashingTF}} and {{FeatureHasher}} are

[jira] [Commented] (SPARK-15463) Support for creating a dataframe from CSV in Dataset[String]

[ https://issues.apache.org/jira/browse/SPARK-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900824#comment-15900824 ] Takeshi Yamamuro commented on SPARK-15463: -- Have you seen

[jira] [Commented] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900821#comment-15900821 ] Saisai Shao commented on SPARK-19812: - [~tgraves], I'm not quite sure what you mean here? bq. The

[jira] [Commented] (SPARK-19843) UTF8String => (int / long) conversion expensive for invalid inputs

[ https://issues.apache.org/jira/browse/SPARK-19843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900773#comment-15900773 ] Apache Spark commented on SPARK-19843: -- User 'tejasapatil' has created a pull request for this

[jira] [Comment Edited] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900741#comment-15900741 ] jin xing edited comment on SPARK-19659 at 3/8/17 5:47 AM: -- [~irashid] [~rxin] I

[jira] [Commented] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900741#comment-15900741 ] jin xing commented on SPARK-19659: -- [~irashid] [~rxin] I uploaded SPARK-19659-design-v2.pdf, please take

[jira] [Updated] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated SPARK-19659: - Attachment: SPARK-19659-design-v2.pdf > Fetch big blocks to disk when shuffle-read >

[jira] [Updated] (SPARK-19860) DataFrame join get conflict error if two frames has a same name column.

'20170222', in_amount1=11032303), Row(fdate=u'20170216', in_amount1=11986702), Row(fdate=u'20170209', in_amount1=9082380), Row(fdate=u'20170214', in_amount1=8142569), Row(fdate=u'20170307', in_amount1=11092829), Row(fdate=u'20170213', in_amount1=12341887), Row(fdate=u'20170228', in_amount1=13966203), Ro

[jira] [Resolved] (SPARK-19348) pyspark.ml.Pipeline gets corrupted under multi threaded use

[ https://issues.apache.org/jira/browse/SPARK-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-19348. --- Resolution: Fixed Fix Version/s: 2.0.3 2.1.1 Issue

[jira] [Updated] (SPARK-19866) Add local version of Word2Vec findSynonyms for spark.ml: Python API

[ https://issues.apache.org/jira/browse/SPARK-19866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-19866: -- Shepherd: Joseph K. Bradley > Add local version of Word2Vec findSynonyms for spark.ml:

[jira] [Created] (SPARK-19866) Add local version of Word2Vec findSynonyms for spark.ml: Python API

Joseph K. Bradley created SPARK-19866: - Summary: Add local version of Word2Vec findSynonyms for spark.ml: Python API Key: SPARK-19866 URL: https://issues.apache.org/jira/browse/SPARK-19866

[jira] [Resolved] (SPARK-17629) Add local version of Word2Vec findSynonyms for spark.ml

[ https://issues.apache.org/jira/browse/SPARK-17629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-17629. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16811

[jira] [Resolved] (SPARK-19859) The new watermark should override the old one

[ https://issues.apache.org/jira/browse/SPARK-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19859. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > The new watermark

[jira] [Resolved] (SPARK-19841) StreamingDeduplicateExec.watermarkPredicate should filter rows based on keys

[ https://issues.apache.org/jira/browse/SPARK-19841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19841. -- Resolution: Fixed Fix Version/s: 2.2.0 > StreamingDeduplicateExec.watermarkPredicate

[jira] [Created] (SPARK-19865) remove the view identifier in SubqueryAlias

Wenchen Fan created SPARK-19865: --- Summary: remove the view identifier in SubqueryAlias Key: SPARK-19865 URL: https://issues.apache.org/jira/browse/SPARK-19865 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-18389) Disallow cyclic view reference

[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-18389: --- Assignee: Jiang Xingbo > Disallow cyclic view reference > -- >

[jira] [Resolved] (SPARK-18389) Disallow cyclic view reference

[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18389. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17152

[jira] [Assigned] (SPARK-19864) add makeQualifiedPath in CatalogUtils to optimize some code

[ https://issues.apache.org/jira/browse/SPARK-19864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19864: Assignee: (was: Apache Spark) > add makeQualifiedPath in CatalogUtils to optimize

[jira] [Commented] (SPARK-19864) add makeQualifiedPath in CatalogUtils to optimize some code

[ https://issues.apache.org/jira/browse/SPARK-19864?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900675#comment-15900675 ] Apache Spark commented on SPARK-19864: -- User 'windpiger' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19864) add makeQualifiedPath in CatalogUtils to optimize some code

[ https://issues.apache.org/jira/browse/SPARK-19864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19864: Assignee: Apache Spark > add makeQualifiedPath in CatalogUtils to optimize some code >

[jira] [Assigned] (SPARK-19843) UTF8String => (int / long) conversion expensive for invalid inputs

[ https://issues.apache.org/jira/browse/SPARK-19843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19843: --- Assignee: Tejas Patil > UTF8String => (int / long) conversion expensive for invalid inputs

[jira] [Resolved] (SPARK-19843) UTF8String => (int / long) conversion expensive for invalid inputs

[ https://issues.apache.org/jira/browse/SPARK-19843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19843. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17184

[jira] [Created] (SPARK-19864) add makeQualifiedPath in CatalogUtils to optimize some code

Song Jun created SPARK-19864: Summary: add makeQualifiedPath in CatalogUtils to optimize some code Key: SPARK-19864 URL: https://issues.apache.org/jira/browse/SPARK-19864 Project: Spark Issue

[jira] [Assigned] (SPARK-19863) Whether or not use CachedKafkaConsumer need to be configured, when you use DirectKafkaInputDStream to connect the kafka in a Spark Streaming application

[ https://issues.apache.org/jira/browse/SPARK-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19863: Assignee: Apache Spark > Whether or not use CachedKafkaConsumer need to be configured,

[jira] [Assigned] (SPARK-19863) Whether or not use CachedKafkaConsumer need to be configured, when you use DirectKafkaInputDStream to connect the kafka in a Spark Streaming application

[ https://issues.apache.org/jira/browse/SPARK-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19863: Assignee: (was: Apache Spark) > Whether or not use CachedKafkaConsumer need to be

[jira] [Commented] (SPARK-19863) Whether or not use CachedKafkaConsumer need to be configured, when you use DirectKafkaInputDStream to connect the kafka in a Spark Streaming application

[ https://issues.apache.org/jira/browse/SPARK-19863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900669#comment-15900669 ] Apache Spark commented on SPARK-19863: -- User 'lvdongr' has created a pull request for this issue:

[jira] [Created] (SPARK-19863) Whether or not use CachedKafkaConsumer need to be configured, when you use DirectKafkaInputDStream to connect the kafka in a Spark Streaming application

LvDongrong created SPARK-19863: -- Summary: Whether or not use CachedKafkaConsumer need to be configured, when you use DirectKafkaInputDStream to connect the kafka in a Spark Streaming application Key: SPARK-19863

[jira] [Commented] (SPARK-13969) Extend input format that feature hashing can handle

[ https://issues.apache.org/jira/browse/SPARK-13969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900662#comment-15900662 ] Joseph K. Bradley commented on SPARK-13969: --- Noticing this JIRA again. I feel like this is

[jira] [Created] (SPARK-19862) In SparkEnv.scala,shortShuffleMgrNames tungsten-sort can be deleted.

guoxiaolong created SPARK-19862: --- Summary: In SparkEnv.scala,shortShuffleMgrNames tungsten-sort can be deleted. Key: SPARK-19862 URL: https://issues.apache.org/jira/browse/SPARK-19862 Project: Spark

[jira] [Commented] (SPARK-19861) watermark should not be a negative time.

[ https://issues.apache.org/jira/browse/SPARK-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900631#comment-15900631 ] Apache Spark commented on SPARK-19861: -- User 'uncleGen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19861) watermark should not be a negative time.

[ https://issues.apache.org/jira/browse/SPARK-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19861: Assignee: (was: Apache Spark) > watermark should not be a negative time. >

[jira] [Assigned] (SPARK-19861) watermark should not be a negative time.

[ https://issues.apache.org/jira/browse/SPARK-19861?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19861: Assignee: Apache Spark > watermark should not be a negative time. >

[jira] [Created] (SPARK-19861) watermark should not be a negative time.

Genmao Yu created SPARK-19861: - Summary: watermark should not be a negative time. Key: SPARK-19861 URL: https://issues.apache.org/jira/browse/SPARK-19861 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-18359) Let user specify locale in CSV parsing

[ https://issues.apache.org/jira/browse/SPARK-18359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900608#comment-15900608 ] Takeshi Yamamuro commented on SPARK-18359: -- Since JDK9 use CLDR as locale by default, it seems

[jira] [Created] (SPARK-19860) DataFrame join get conflict error if two frames has a same name column.

1=9082380), Row(fdate=u'20170214', in_amount1=8142569), Row(fdate=u'20170307', in_amount1=11092829), Row(fdate=u'20170213', in_amount1=12341887), Row(fdate=u'20170228', in_amount1=13966203), Row(fdate=u'20170220', in_amount1=9397558), Row(fdate=u'20170210', in_amount1=8205431), Row(fdate=u

[jira] [Commented] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900568#comment-15900568 ] Apache Spark commented on SPARK-18055: -- User 'marmbrus' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18055: Assignee: Michael Armbrust (was: Apache Spark) > Dataset.flatMap can't work with types

[jira] [Assigned] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18055: Assignee: Apache Spark (was: Michael Armbrust) > Dataset.flatMap can't work with types

[jira] [Updated] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18055: - Target Version/s: 2.2.0 > Dataset.flatMap can't work with types from customized jar >

[jira] [Commented] (SPARK-19810) Remove support for Scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-19810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900548#comment-15900548 ] Min Shen commented on SPARK-19810: -- [~srowen], Want to get an idea regarding the timeline for removing

[jira] [Assigned] (SPARK-18055) Dataset.flatMap can't work with types from customized jar

[ https://issues.apache.org/jira/browse/SPARK-18055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust reassigned SPARK-18055: Assignee: Michael Armbrust > Dataset.flatMap can't work with types from

[jira] [Commented] (SPARK-16333) Excessive Spark history event/json data size (5GB each)

[ https://issues.apache.org/jira/browse/SPARK-16333?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900533#comment-15900533 ] Jim Kleckner commented on SPARK-16333: -- I ended up here when looking into why an upgrade of our

[jira] [Commented] (SPARK-19561) Pyspark Dataframes don't allow timestamps near epoch

[ https://issues.apache.org/jira/browse/SPARK-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900524#comment-15900524 ] Apache Spark commented on SPARK-19561: -- User 'JasonMWhite' has created a pull request for this

[jira] [Commented] (SPARK-19852) StringIndexer.setHandleInvalid should have another option 'new': Python API and docs

[ https://issues.apache.org/jira/browse/SPARK-19852?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900463#comment-15900463 ] Vincent commented on SPARK-19852: - I can work on this issue, since it is related to SPARK-17498 >

[jira] [Resolved] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

[ https://issues.apache.org/jira/browse/SPARK-19857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-19857. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-19859) The new watermark should override the old one

[ https://issues.apache.org/jira/browse/SPARK-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900410#comment-15900410 ] Apache Spark commented on SPARK-19859: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19859) The new watermark should override the old one

[ https://issues.apache.org/jira/browse/SPARK-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19859: Assignee: Shixiong Zhu (was: Apache Spark) > The new watermark should override the old

[jira] [Assigned] (SPARK-19859) The new watermark should override the old one

[ https://issues.apache.org/jira/browse/SPARK-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19859: Assignee: Apache Spark (was: Shixiong Zhu) > The new watermark should override the old

[jira] [Created] (SPARK-19859) The new watermark should override the old one

Shixiong Zhu created SPARK-19859: Summary: The new watermark should override the old one Key: SPARK-19859 URL: https://issues.apache.org/jira/browse/SPARK-19859 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19858: Assignee: Shixiong Zhu (was: Apache Spark) > Add output mode to flatMapGroupsWithState

[jira] [Commented] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900377#comment-15900377 ] Apache Spark commented on SPARK-19858: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19858: Assignee: Apache Spark (was: Shixiong Zhu) > Add output mode to flatMapGroupsWithState

[jira] [Created] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

Shixiong Zhu created SPARK-19858: Summary: Add output mode to flatMapGroupsWithState and disallow invalid cases Key: SPARK-19858 URL: https://issues.apache.org/jira/browse/SPARK-19858 Project: Spark

[jira] [Commented] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

[ https://issues.apache.org/jira/browse/SPARK-19857?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900363#comment-15900363 ] Apache Spark commented on SPARK-19857: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

[ https://issues.apache.org/jira/browse/SPARK-19857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19857: Assignee: (was: Apache Spark) > CredentialUpdater calculates the wrong time for next

[jira] [Assigned] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

[ https://issues.apache.org/jira/browse/SPARK-19857?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19857: Assignee: Apache Spark > CredentialUpdater calculates the wrong time for next update >

[jira] [Created] (SPARK-19857) CredentialUpdater calculates the wrong time for next update

Marcelo Vanzin created SPARK-19857: -- Summary: CredentialUpdater calculates the wrong time for next update Key: SPARK-19857 URL: https://issues.apache.org/jira/browse/SPARK-19857 Project: Spark

[jira] [Assigned] (SPARK-19855) Create an internal FilePartitionStrategy interface

[ https://issues.apache.org/jira/browse/SPARK-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19855: Assignee: Reynold Xin (was: Apache Spark) > Create an internal FilePartitionStrategy

[jira] [Commented] (SPARK-19855) Create an internal FilePartitionStrategy interface

[ https://issues.apache.org/jira/browse/SPARK-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900315#comment-15900315 ] Apache Spark commented on SPARK-19855: -- User 'rxin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19855) Create an internal FilePartitionStrategy interface

[ https://issues.apache.org/jira/browse/SPARK-19855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19855: Assignee: Apache Spark (was: Reynold Xin) > Create an internal FilePartitionStrategy

[jira] [Updated] (SPARK-19856) Turn partitioning related test cases in FileSourceStrategySuite from integration tests into unit tests

[ https://issues.apache.org/jira/browse/SPARK-19856?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-19856: Summary: Turn partitioning related test cases in FileSourceStrategySuite from integration tests

[jira] [Created] (SPARK-19855) Create an internal FilePartitionStrategy interface

Reynold Xin created SPARK-19855: --- Summary: Create an internal FilePartitionStrategy interface Key: SPARK-19855 URL: https://issues.apache.org/jira/browse/SPARK-19855 Project: Spark Issue Type:

[jira] [Created] (SPARK-19856) Turn partitioning related test cases in FileSourceStrategySuite into unit tests

Reynold Xin created SPARK-19856: --- Summary: Turn partitioning related test cases in FileSourceStrategySuite into unit tests Key: SPARK-19856 URL: https://issues.apache.org/jira/browse/SPARK-19856

[jira] [Created] (SPARK-19854) Refactor file partitioning strategy to make it easier to extend / unit test

Reynold Xin created SPARK-19854: --- Summary: Refactor file partitioning strategy to make it easier to extend / unit test Key: SPARK-19854 URL: https://issues.apache.org/jira/browse/SPARK-19854 Project:

[jira] [Updated] (SPARK-18138) More officially deprecate support for Python 2.6, Java 7, and Scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-18138?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-18138: Labels: releasenotes (was: ) > More officially deprecate support for Python 2.6, Java 7, and

[jira] [Updated] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ari Gesher updated SPARK-19764: --- We're driving everything from Python. It may be a bug that we're not getting the error to propagate up

[jira] [Updated] (SPARK-19853) Uppercase Kafka topics fail when startingOffsets are SpecificOffsets

[ https://issues.apache.org/jira/browse/SPARK-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19853: - Target Version/s: 2.2.0 > Uppercase Kafka topics fail when startingOffsets are SpecificOffsets >

[jira] [Updated] (SPARK-19853) Uppercase Kafka topics fail when startingOffsets are SpecificOffsets

[ https://issues.apache.org/jira/browse/SPARK-19853?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chris Bowden updated SPARK-19853: - Description: When using the KafkaSource with Structured Streaming, consumer assignments are not

[jira] [Created] (SPARK-19853) Uppercase Kafka topics fail when startingOffsets are SpecificOffsets

Chris Bowden created SPARK-19853: Summary: Uppercase Kafka topics fail when startingOffsets are SpecificOffsets Key: SPARK-19853 URL: https://issues.apache.org/jira/browse/SPARK-19853 Project: Spark

[jira] [Comment Edited] (SPARK-16207) order guarantees for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900220#comment-15900220 ] Chris Rogers edited comment on SPARK-16207 at 3/7/17 9:52 PM: -- [~srowen]

[jira] [Commented] (SPARK-16207) order guarantees for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900220#comment-15900220 ] Chris Rogers commented on SPARK-16207: -- [~srowen] since there is no documentation yet, I don't know

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900216#comment-15900216 ] Nick Afshartous commented on SPARK-19767: - Missed that one, thanks. > API Doc pages for

[jira] [Assigned] (SPARK-19702) Increasse refuse_seconds timeout in the Mesos Spark Dispatcher

[ https://issues.apache.org/jira/browse/SPARK-19702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-19702: - Assignee: Michael Gummelt Priority: Minor (was: Major) Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900207#comment-15900207 ] Sean Owen commented on SPARK-19767: --- Oh, are you not running from the {{docs/}} directory? > API Doc

[jira] [Commented] (SPARK-19767) API Doc pages for Streaming with Kafka 0.10 not current

[ https://issues.apache.org/jira/browse/SPARK-19767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900184#comment-15900184 ] Nick Afshartous commented on SPARK-19767: - Yes, I completed the steps in the Prerequisites

[jira] [Resolved] (SPARK-19561) Pyspark Dataframes don't allow timestamps near epoch

[ https://issues.apache.org/jira/browse/SPARK-19561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-19561. Resolution: Fixed Assignee: Jason White Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-16207) order guarantees for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900162#comment-15900162 ] Sean Owen commented on SPARK-16207: --- [~rcrogers] where would you document this? we could add a document

[jira] [Commented] (SPARK-16207) order guarantees for DataFrames

[ https://issues.apache.org/jira/browse/SPARK-16207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900152#comment-15900152 ] Chris Rogers commented on SPARK-16207: -- The lack of documentation on this is immensely confusing. >

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15900127#comment-15900127 ] Shixiong Zhu commented on SPARK-19764: -- So you don't set an UncaughtExceptionHandler and this OOM

[jira] [Updated] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19851: - Component/s: (was: Spark Core) > Add support for EVERY and ANY (SOME) aggregates >

[jira] [Updated] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-19803: --- Labels: flaky-test (was: ) > Flaky BlockManagerProactiveReplicationSuite tests >

[jira] [Updated] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-19803: --- Affects Version/s: (was: 2.3.0) 2.2.0 > Flaky

[jira] [Updated] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout updated SPARK-19803: --- Component/s: Tests > Flaky BlockManagerProactiveReplicationSuite tests >

[jira] [Resolved] (SPARK-19803) Flaky BlockManagerProactiveReplicationSuite tests

[ https://issues.apache.org/jira/browse/SPARK-19803?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kay Ousterhout resolved SPARK-19803. Resolution: Fixed Assignee: Genmao Yu Fix Version/s: 2.2.0 Thanks for

[jira] [Resolved] (SPARK-19516) update public doc to use SparkSession instead of SparkContext

[ https://issues.apache.org/jira/browse/SPARK-19516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19516. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16856

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1599#comment-1599 ] Ari Gesher commented on SPARK-19764: We were collecting more data than we had heap for. Still useful?

[jira] [Created] (SPARK-19852) StringIndexer.setHandleInvalid should have another option 'new': Python API and docs

Joseph K. Bradley created SPARK-19852: - Summary: StringIndexer.setHandleInvalid should have another option 'new': Python API and docs Key: SPARK-19852 URL: https://issues.apache.org/jira/browse/SPARK-19852

[jira] [Resolved] (SPARK-17498) StringIndexer.setHandleInvalid should have another option 'new'

[ https://issues.apache.org/jira/browse/SPARK-17498?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-17498. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16883

[jira] [Updated] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Styles updated SPARK-19851: --- Description: Add support for EVERY and ANY (SOME) aggregates. - EVERY returns true if all

[jira] [Commented] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

[ https://issues.apache.org/jira/browse/SPARK-19851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15899988#comment-15899988 ] Michael Styles commented on SPARK-19851: https://github.com/apache/spark/pull/17194 > Add

[jira] [Commented] (SPARK-19348) pyspark.ml.Pipeline gets corrupted under multi threaded use

[ https://issues.apache.org/jira/browse/SPARK-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15899979#comment-15899979 ] Apache Spark commented on SPARK-19348: -- User 'BryanCutler' has created a pull request for this

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15899980#comment-15899980 ] Shixiong Zhu commented on SPARK-19764: -- [~agesher] Do you have the OOM stack trace? So that we can

[jira] [Created] (SPARK-19851) Add support for EVERY and ANY (SOME) aggregates

Michael Styles created SPARK-19851: -- Summary: Add support for EVERY and ANY (SOME) aggregates Key: SPARK-19851 URL: https://issues.apache.org/jira/browse/SPARK-19851 Project: Spark Issue

[jira] [Resolved] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ari Gesher resolved SPARK-19764. Resolution: Not A Bug > Executors hang with supposedly running task that are really finished. >

[jira] [Commented] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15899964#comment-15899964 ] Ari Gesher commented on SPARK-19764: We narrowed this down to driver OOM that wasn't being properly

[jira] [Resolved] (SPARK-18549) Failed to Uncache a View that References a Dropped Table.

[ https://issues.apache.org/jira/browse/SPARK-18549?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-18549. - Resolution: Fixed Assignee: Wenchen Fan Fix Version/s: 2.2.0 > Failed to Uncache a View

[jira] [Resolved] (SPARK-19765) UNCACHE TABLE should also un-cache all cached plans that refer to this table

[ https://issues.apache.org/jira/browse/SPARK-19765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19765. - Resolution: Fixed Fix Version/s: 2.2.0 > UNCACHE TABLE should also un-cache all cached plans that

  1   2   >