[jira] [Updated] (SPARK-19881) Support Dynamic Partition Inserts params with SET command

[ https://issues.apache.org/jira/browse/SPARK-19881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19881: -- Description: Since Spark 2.0.0, `SET` commands do not pass the values to HiveClient. In most

[jira] [Updated] (SPARK-19881) Support Dynamic Partition Inserts params with SET command

[ https://issues.apache.org/jira/browse/SPARK-19881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19881: -- Description: ## What changes were proposed in this pull request? Since Spark 2.0.0, `SET`

[jira] [Assigned] (SPARK-19881) Support Dynamic Partition Inserts params with SET command

[ https://issues.apache.org/jira/browse/SPARK-19881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19881: Assignee: (was: Apache Spark) > Support Dynamic Partition Inserts params with SET

[jira] [Commented] (SPARK-19881) Support Dynamic Partition Inserts params with SET command

[ https://issues.apache.org/jira/browse/SPARK-19881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902614#comment-15902614 ] Apache Spark commented on SPARK-19881: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Assigned] (SPARK-19881) Support Dynamic Partition Inserts params with SET command

[ https://issues.apache.org/jira/browse/SPARK-19881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19881: Assignee: Apache Spark > Support Dynamic Partition Inserts params with SET command >

[jira] [Created] (SPARK-19881) Support Dynamic Partition Inserts params with SET command

Dongjoon Hyun created SPARK-19881: - Summary: Support Dynamic Partition Inserts params with SET command Key: SPARK-19881 URL: https://issues.apache.org/jira/browse/SPARK-19881 Project: Spark

[jira] [Commented] (SPARK-19439) PySpark's registerJavaFunction Should Support UDAFs

[ https://issues.apache.org/jira/browse/SPARK-19439?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902601#comment-15902601 ] Apache Spark commented on SPARK-19439: -- User 'zjffdu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19439) PySpark's registerJavaFunction Should Support UDAFs

[ https://issues.apache.org/jira/browse/SPARK-19439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19439: Assignee: Apache Spark > PySpark's registerJavaFunction Should Support UDAFs >

[jira] [Assigned] (SPARK-19439) PySpark's registerJavaFunction Should Support UDAFs

[ https://issues.apache.org/jira/browse/SPARK-19439?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19439: Assignee: (was: Apache Spark) > PySpark's registerJavaFunction Should Support UDAFs >

[jira] [Resolved] (SPARK-19874) Hide API docs for "org.apache.spark.sql.internal"

[ https://issues.apache.org/jira/browse/SPARK-19874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19874. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > Hide API docs for

[jira] [Updated] (SPARK-19874) Hide API docs for "org.apache.spark.sql.internal"

[ https://issues.apache.org/jira/browse/SPARK-19874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19874: - Priority: Minor (was: Major) > Hide API docs for "org.apache.spark.sql.internal" >

[jira] [Resolved] (SPARK-19235) Enable Test Cases in DDLSuite with Hive Metastore

[ https://issues.apache.org/jira/browse/SPARK-19235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19235. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16592

[jira] [Comment Edited] (SPARK-11141) Batching of ReceivedBlockTrackerLogEvents for efficient WAL writes

[ https://issues.apache.org/jira/browse/SPARK-11141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902537#comment-15902537 ] Jim Kleckner edited comment on SPARK-11141 at 3/9/17 5:54 AM: -- FYI, this can

[jira] [Commented] (SPARK-11141) Batching of ReceivedBlockTrackerLogEvents for efficient WAL writes

[ https://issues.apache.org/jira/browse/SPARK-11141?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902537#comment-15902537 ] Jim Kleckner commented on SPARK-11141: -- FYI, this can cause problems when not using S3 during

[jira] [Commented] (SPARK-19866) Add local version of Word2Vec findSynonyms for spark.ml: Python API

[ https://issues.apache.org/jira/browse/SPARK-19866?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902522#comment-15902522 ] Xin Ren commented on SPARK-19866: - I can try this one :) > Add local version of Word2Vec findSynonyms

[jira] [Updated] (SPARK-19880) About spark2.0.2 and spark1.4.1 beeline to show the database, use the default operation such as dealing with different

[ https://issues.apache.org/jira/browse/SPARK-19880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolong updated SPARK-19880: Description: About spark2.0.2 and spark1.4.1 beeline to show the database, use the default

[jira] [Created] (SPARK-19880) About spark2.0.2 and spark1.4.1 beeline to show the database, use the default operation such as dealing with different

guoxiaolong created SPARK-19880: --- Summary: About spark2.0.2 and spark1.4.1 beeline to show the database, use the default operation such as dealing with different Key: SPARK-19880 URL:

[jira] [Assigned] (SPARK-19862) In SparkEnv.scala,shortShuffleMgrNames tungsten-sort can be deleted.

[ https://issues.apache.org/jira/browse/SPARK-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19862: Assignee: (was: Apache Spark) > In SparkEnv.scala,shortShuffleMgrNames tungsten-sort

[jira] [Assigned] (SPARK-19862) In SparkEnv.scala,shortShuffleMgrNames tungsten-sort can be deleted.

[ https://issues.apache.org/jira/browse/SPARK-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19862: Assignee: Apache Spark > In SparkEnv.scala,shortShuffleMgrNames tungsten-sort can be

[jira] [Commented] (SPARK-19862) In SparkEnv.scala,shortShuffleMgrNames tungsten-sort can be deleted.

[ https://issues.apache.org/jira/browse/SPARK-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902407#comment-15902407 ] Apache Spark commented on SPARK-19862: -- User 'guoxiaolongzte' has created a pull request for this

[jira] [Updated] (SPARK-19878) Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala

[ https://issues.apache.org/jira/browse/SPARK-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kavn qin updated SPARK-19878: - External issue URL: (was: https://issues.apache.org/jira/browse/SPARK-17920) External issue ID:

[jira] [Updated] (SPARK-19878) Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala

[ https://issues.apache.org/jira/browse/SPARK-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kavn qin updated SPARK-19878: - Attachment: SPARK-19878.patch > Add hive configuration when initialize hive serde in

[jira] [Updated] (SPARK-19878) Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala

[ https://issues.apache.org/jira/browse/SPARK-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kavn qin updated SPARK-19878: - Attachment: (was: SPARK-19878.patch) > Add hive configuration when initialize hive serde in

[jira] [Updated] (SPARK-19878) Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala

[ https://issues.apache.org/jira/browse/SPARK-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kavn qin updated SPARK-19878: - Description: When case class InsertIntoHiveTable intializes a serde it explicitly passes null for the

[jira] [Issue Comment Deleted] (SPARK-12180) DataFrame.join() in PySpark gives misleading exception when column name exists on both side

[ https://issues.apache.org/jira/browse/SPARK-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Abhishek Kumar updated SPARK-12180: --- Comment: was deleted (was: Is there any concrete solution or reason explaining the issue ? I

[jira] [Updated] (SPARK-19878) Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala

[ https://issues.apache.org/jira/browse/SPARK-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] kavn qin updated SPARK-19878: - Attachment: SPARK-19878.patch > Add hive configuration when initialize hive serde in

[jira] [Created] (SPARK-19879) Spark UI table sort breaks event timeline

Sebastian Estevez created SPARK-19879: - Summary: Spark UI table sort breaks event timeline Key: SPARK-19879 URL: https://issues.apache.org/jira/browse/SPARK-19879 Project: Spark Issue

[jira] [Created] (SPARK-19878) Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala

kavn qin created SPARK-19878: Summary: Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala Key: SPARK-19878 URL: https://issues.apache.org/jira/browse/SPARK-19878 Project:

[jira] [Comment Edited] (SPARK-12180) DataFrame.join() in PySpark gives misleading exception when column name exists on both side

[ https://issues.apache.org/jira/browse/SPARK-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902389#comment-15902389 ] Abhishek Kumar edited comment on SPARK-12180 at 3/9/17 2:52 AM: Is there

[jira] [Commented] (SPARK-12180) DataFrame.join() in PySpark gives misleading exception when column name exists on both side

[ https://issues.apache.org/jira/browse/SPARK-12180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902389#comment-15902389 ] Abhishek Kumar commented on SPARK-12180: Is there any concrete solution or reason explaining the

[jira] [Commented] (SPARK-19859) The new watermark should override the old one

[ https://issues.apache.org/jira/browse/SPARK-19859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902368#comment-15902368 ] Apache Spark commented on SPARK-19859: -- User 'uncleGen' has created a pull request for this issue:

[jira] [Created] (SPARK-19877) Restrict the depth of view reference chains

Jiang Xingbo created SPARK-19877: Summary: Restrict the depth of view reference chains Key: SPARK-19877 URL: https://issues.apache.org/jira/browse/SPARK-19877 Project: Spark Issue Type:

[jira] [Commented] (SPARK-19877) Restrict the depth of view reference chains

[ https://issues.apache.org/jira/browse/SPARK-19877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902324#comment-15902324 ] Jiang Xingbo commented on SPARK-19877: -- I'm working on this. > Restrict the depth of view reference

[jira] [Commented] (SPARK-19808) About the default blocking arg in unpersist

[ https://issues.apache.org/jira/browse/SPARK-19808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902317#comment-15902317 ] zhengruifeng commented on SPARK-19808: -- [~srowen] Agreed. Changing the default may cause latent

[jira] [Closed] (SPARK-19808) About the default blocking arg in unpersist

[ https://issues.apache.org/jira/browse/SPARK-19808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng closed SPARK-19808. Resolution: Not A Problem > About the default blocking arg in unpersist >

[jira] [Comment Edited] (SPARK-16283) Implement percentile_approx SQL function

[ https://issues.apache.org/jira/browse/SPARK-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901132#comment-15901132 ] chenerlu edited comment on SPARK-16283 at 3/9/17 1:55 AM: -- Hi, I am little

[jira] [Comment Edited] (SPARK-16283) Implement percentile_approx SQL function

[ https://issues.apache.org/jira/browse/SPARK-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901132#comment-15901132 ] chenerlu edited comment on SPARK-16283 at 3/9/17 1:55 AM: -- Hi, I am little

[jira] [Commented] (SPARK-19862) In SparkEnv.scala,shortShuffleMgrNames tungsten-sort can be deleted.

[ https://issues.apache.org/jira/browse/SPARK-19862?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902263#comment-15902263 ] guoxiaolong commented on SPARK-19862: - remove tungsten-sort.Because it is not represent

[jira] [Resolved] (SPARK-19507) pyspark.sql.types._verify_type() exceptions too broad to debug collections or nested data

[ https://issues.apache.org/jira/browse/SPARK-19507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-19507. -- Resolution: Duplicate Actually, it seems a duplicate of SPARK-19871. Let me resolve this one

[jira] [Assigned] (SPARK-19876) Add OneTime trigger executor

[ https://issues.apache.org/jira/browse/SPARK-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19876: Assignee: (was: Apache Spark) > Add OneTime trigger executor >

[jira] [Commented] (SPARK-19876) Add OneTime trigger executor

[ https://issues.apache.org/jira/browse/SPARK-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902221#comment-15902221 ] Apache Spark commented on SPARK-19876: -- User 'tcondie' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19876) Add OneTime trigger executor

[ https://issues.apache.org/jira/browse/SPARK-19876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19876: Assignee: Apache Spark > Add OneTime trigger executor > > >

[jira] [Updated] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Bruggeman updated SPARK-19872: Priority: Blocker (was: Major) > UnicodeDecodeError in Pyspark on sc.textFile read with

[jira] [Created] (SPARK-19876) Add OneTime trigger executor

Tyson Condie created SPARK-19876: Summary: Add OneTime trigger executor Key: SPARK-19876 URL: https://issues.apache.org/jira/browse/SPARK-19876 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-19281) spark.ml Python API for FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-19281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19281: Assignee: Apache Spark > spark.ml Python API for FPGrowth >

[jira] [Assigned] (SPARK-19281) spark.ml Python API for FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-19281?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19281: Assignee: (was: Apache Spark) > spark.ml Python API for FPGrowth >

[jira] [Commented] (SPARK-19281) spark.ml Python API for FPGrowth

[ https://issues.apache.org/jira/browse/SPARK-19281?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902198#comment-15902198 ] Apache Spark commented on SPARK-19281: -- User 'zero323' has created a pull request for this issue:

[jira] [Updated] (SPARK-19875) Map->filter on many columns gets stuck in constraint inference optimization code

[ https://issues.apache.org/jira/browse/SPARK-19875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Pranavamurthi updated SPARK-19875: -- Description: The attached code (TestFilter.scala) works with a 10-column csv dataset,

[jira] [Updated] (SPARK-19875) Map->filter on many columns gets stuck in constraint inference optimization code

[ https://issues.apache.org/jira/browse/SPARK-19875?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jay Pranavamurthi updated SPARK-19875: -- Attachment: TestFilter.scala test50cols.csv

[jira] [Created] (SPARK-19875) Map->filter on many columns gets stuck in constraint inference optimization code

Jay Pranavamurthi created SPARK-19875: - Summary: Map->filter on many columns gets stuck in constraint inference optimization code Key: SPARK-19875 URL: https://issues.apache.org/jira/browse/SPARK-19875

[jira] [Commented] (SPARK-6936) SQLContext.sql() caused deadlock in multi-thread env

[ https://issues.apache.org/jira/browse/SPARK-6936?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902167#comment-15902167 ] Henry Min commented on SPARK-6936: -- This issue seems has been fixed on the version 1.5.0. The information

[jira] [Assigned] (SPARK-19540) Add ability to clone SparkSession with an identical copy of the SessionState

[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-19540: Assignee: Kunal Khamar > Add ability to clone SparkSession with an identical copy of the

[jira] [Resolved] (SPARK-19540) Add ability to clone SparkSession with an identical copy of the SessionState

[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19540. -- Resolution: Fixed Fix Version/s: 2.2.0 > Add ability to clone SparkSession with an

[jira] [Commented] (SPARK-19874) Hide API docs for "org.apache.spark.sql.internal"

[ https://issues.apache.org/jira/browse/SPARK-19874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902161#comment-15902161 ] Apache Spark commented on SPARK-19874: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19874) Hide API docs for "org.apache.spark.sql.internal"

[ https://issues.apache.org/jira/browse/SPARK-19874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19874: Assignee: Apache Spark (was: Shixiong Zhu) > Hide API docs for

[jira] [Assigned] (SPARK-19874) Hide API docs for "org.apache.spark.sql.internal"

[ https://issues.apache.org/jira/browse/SPARK-19874?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19874: Assignee: Shixiong Zhu (was: Apache Spark) > Hide API docs for

[jira] [Created] (SPARK-19874) Hide API docs for "org.apache.spark.sql.internal"

Shixiong Zhu created SPARK-19874: Summary: Hide API docs for "org.apache.spark.sql.internal" Key: SPARK-19874 URL: https://issues.apache.org/jira/browse/SPARK-19874 Project: Spark Issue

[jira] [Assigned] (SPARK-19873) If the user changes the number of shuffle partitions between batches, Streaming aggregation will fail.

[ https://issues.apache.org/jira/browse/SPARK-19873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19873: Assignee: Apache Spark > If the user changes the number of shuffle partitions between

[jira] [Commented] (SPARK-19873) If the user changes the number of shuffle partitions between batches, Streaming aggregation will fail.

[ https://issues.apache.org/jira/browse/SPARK-19873?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902122#comment-15902122 ] Apache Spark commented on SPARK-19873: -- User 'kunalkhamar' has created a pull request for this

[jira] [Assigned] (SPARK-19873) If the user changes the number of shuffle partitions between batches, Streaming aggregation will fail.

[ https://issues.apache.org/jira/browse/SPARK-19873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19873: Assignee: (was: Apache Spark) > If the user changes the number of shuffle partitions

[jira] [Updated] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19858: - Affects Version/s: (was: 2.1.1) > Add output mode to flatMapGroupsWithState and disallow

[jira] [Resolved] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19858. -- Resolution: Fixed Fix Version/s: 2.2.0 > Add output mode to flatMapGroupsWithState and

[jira] [Updated] (SPARK-19413) Basic mapGroupsWithState API

[ https://issues.apache.org/jira/browse/SPARK-19413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19413: - Fix Version/s: (was: 2.1.1) > Basic mapGroupsWithState API > >

[jira] [Commented] (SPARK-19413) Basic mapGroupsWithState API

[ https://issues.apache.org/jira/browse/SPARK-19413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902105#comment-15902105 ] Shixiong Zhu commented on SPARK-19413: -- Reverted the patch from branch 2.1. This feature will not go

[jira] [Updated] (SPARK-19413) Basic mapGroupsWithState API

[ https://issues.apache.org/jira/browse/SPARK-19413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19413: - Target Version/s: 2.2.0 (was: 2.1.1, 2.2.0) > Basic mapGroupsWithState API >

[jira] [Resolved] (SPARK-19813) maxFilesPerTrigger combo latestFirst may miss old files in combination with maxFileAge in FileStreamSource

[ https://issues.apache.org/jira/browse/SPARK-19813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-19813. - Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > maxFilesPerTrigger

[jira] [Comment Edited] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902023#comment-15902023 ] Brian Bruggeman edited comment on SPARK-19872 at 3/8/17 10:33 PM: -- Using

[jira] [Updated] (SPARK-19540) Add ability to clone SparkSession with an identical copy of the SessionState

[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khamar updated SPARK-19540: - Summary: Add ability to clone SparkSession with an identical copy of the SessionState (was: Add

[jira] [Updated] (SPARK-19540) Add ability to clone SparkSession wherein cloned session has an identical copy of the SessionState

[ https://issues.apache.org/jira/browse/SPARK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khamar updated SPARK-19540: - Summary: Add ability to clone SparkSession wherein cloned session has an identical copy of the

[jira] [Closed] (SPARK-19814) Spark History Server Out Of Memory / Extreme GC

[ https://issues.apache.org/jira/browse/SPARK-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon King closed SPARK-19814. -- Resolution: Duplicate Looks like it's wrong to characterize this as a bug -- couldn't identify an

[jira] [Comment Edited] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902023#comment-15902023 ] Brian Bruggeman edited comment on SPARK-19872 at 3/8/17 9:48 PM: - Using

[jira] [Comment Edited] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902023#comment-15902023 ] Brian Bruggeman edited comment on SPARK-19872 at 3/8/17 9:46 PM: - Using

[jira] [Commented] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902023#comment-15902023 ] Brian Bruggeman commented on SPARK-19872: - Using the Spark 2.1.0 serializers.py and the Spark

[jira] [Comment Edited] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15902023#comment-15902023 ] Brian Bruggeman edited comment on SPARK-19872 at 3/8/17 9:46 PM: - Using

[jira] [Assigned] (SPARK-15463) Support for creating a dataframe from CSV in Dataset[String]

[ https://issues.apache.org/jira/browse/SPARK-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-15463: --- Assignee: Hyukjin Kwon > Support for creating a dataframe from CSV in Dataset[String] >

[jira] [Resolved] (SPARK-15463) Support for creating a dataframe from CSV in Dataset[String]

[ https://issues.apache.org/jira/browse/SPARK-15463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-15463. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16854

[jira] [Updated] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19858: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-19067 > Add output mode to

[jira] [Updated] (SPARK-19858) Add output mode to flatMapGroupsWithState and disallow invalid cases

[ https://issues.apache.org/jira/browse/SPARK-19858?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19858: - Affects Version/s: 2.1.1 > Add output mode to flatMapGroupsWithState and disallow invalid cases

[jira] [Commented] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901997#comment-15901997 ] Brian Bruggeman commented on SPARK-19872: - I reverted `rdd.py` and `serializers.py` to the 2.0.2

[jira] [Updated] (SPARK-19873) If the user changes the number of shuffle partitions between batches, Streaming aggregation will fail.

[ https://issues.apache.org/jira/browse/SPARK-19873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khamar updated SPARK-19873: - Summary: If the user changes the number of shuffle partitions between batches, Streaming

[jira] [Updated] (SPARK-19873) If the user changes the shuffle partition number between batches, Streaming aggregation will fail.

[ https://issues.apache.org/jira/browse/SPARK-19873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khamar updated SPARK-19873: - Description: If the user changes the shuffle partition number between batches, Streaming

[jira] [Updated] (SPARK-19873) If the user changes the shuffle partition number between batches, Streaming aggregation will fail.

[ https://issues.apache.org/jira/browse/SPARK-19873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khamar updated SPARK-19873: - Description: If the user changes the shuffle partition number between batches, Streaming

[jira] [Updated] (SPARK-19873) If the user changes the shuffle partition number between batches, Streaming aggregation will fail.

[ https://issues.apache.org/jira/browse/SPARK-19873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kunal Khamar updated SPARK-19873: - Description: It the user changes the shuffle partition number between batches, Streaming

[jira] [Created] (SPARK-19873) If the user changes the shuffle partition number between batches, Streaming aggregation will fail.

Kunal Khamar created SPARK-19873: Summary: If the user changes the shuffle partition number between batches, Streaming aggregation will fail. Key: SPARK-19873 URL:

[jira] [Commented] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901974#comment-15901974 ] Brian Bruggeman commented on SPARK-19872: - This is a regression from spark 2.0.x. >

[jira] [Updated] (SPARK-19481) Fix flaky test: o.a.s.repl.ReplSuite should clone and clean line object in ClosureCleaner

[ https://issues.apache.org/jira/browse/SPARK-19481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19481: - Fix Version/s: 2.0.3 > Fix flaky test: o.a.s.repl.ReplSuite should clone and clean line object

[jira] [Updated] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Bruggeman updated SPARK-19872: Description: I'm receiving the following traceback: {code} >>>

[jira] [Updated] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

[ https://issues.apache.org/jira/browse/SPARK-19872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Bruggeman updated SPARK-19872: Description: I'm receiving the following traceback: {{ >>>

[jira] [Created] (SPARK-19872) UnicodeDecodeError in Pyspark on sc.textFile read with repartition

Brian Bruggeman created SPARK-19872: --- Summary: UnicodeDecodeError in Pyspark on sc.textFile read with repartition Key: SPARK-19872 URL: https://issues.apache.org/jira/browse/SPARK-19872 Project:

[jira] [Assigned] (SPARK-19727) Spark SQL round function modifies original column

[ https://issues.apache.org/jira/browse/SPARK-19727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19727: --- Assignee: Wojciech Szymanski > Spark SQL round function modifies original column >

[jira] [Resolved] (SPARK-19727) Spark SQL round function modifies original column

[ https://issues.apache.org/jira/browse/SPARK-19727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19727. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17075

[jira] [Updated] (SPARK-18355) Spark SQL fails to read data from a ORC hive table that has a new column added to it

[ https://issues.apache.org/jira/browse/SPARK-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18355: -- Affects Version/s: 2.1.0 > Spark SQL fails to read data from a ORC hive table that has a new

[jira] [Commented] (SPARK-18355) Spark SQL fails to read data from a ORC hive table that has a new column added to it

[ https://issues.apache.org/jira/browse/SPARK-18355?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901914#comment-15901914 ] Dongjoon Hyun commented on SPARK-18355: --- I confirmed that this happens only with

[jira] [Commented] (SPARK-15474) ORC data source fails to write and read back empty dataframe

[ https://issues.apache.org/jira/browse/SPARK-15474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901769#comment-15901769 ] Owen O'Malley commented on SPARK-15474: --- Ok, Hive's use is fine because it gets the schema from the

[jira] [Resolved] (SPARK-19864) add makeQualifiedPath in SQLTestUtils to optimize some code

[ https://issues.apache.org/jira/browse/SPARK-19864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19864. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17204

[jira] [Assigned] (SPARK-19864) add makeQualifiedPath in SQLTestUtils to optimize some code

[ https://issues.apache.org/jira/browse/SPARK-19864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-19864: --- Assignee: Song Jun > add makeQualifiedPath in SQLTestUtils to optimize some code >

[jira] [Resolved] (SPARK-18209) More robust view canonicalization without full SQL expansion

[ https://issues.apache.org/jira/browse/SPARK-18209?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18209. - Resolution: Fixed > More robust view canonicalization without full SQL expansion >

[jira] [Commented] (SPARK-19871) Improve error message in verify_type to indicate which field the error is for

[ https://issues.apache.org/jira/browse/SPARK-19871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901668#comment-15901668 ] Len Frodgers commented on SPARK-19871: -- https://github.com/apache/spark/pull/17213 > Improve error

[jira] [Commented] (SPARK-13740) add null check for _verify_type in types.py

[ https://issues.apache.org/jira/browse/SPARK-13740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15901667#comment-15901667 ] Len Frodgers commented on SPARK-13740: -- https://github.com/apache/spark/pull/17213 > add null check

[jira] [Assigned] (SPARK-19871) Improve error message in verify_type to indicate which field the error is for

[ https://issues.apache.org/jira/browse/SPARK-19871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19871: Assignee: Apache Spark > Improve error message in verify_type to indicate which field the

  1   2   >