[jira] [Commented] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

2018-10-15 Thread Gengliang Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651152#comment-16651152 ] Gengliang Wang commented on SPARK-25588: [~srowen] I tried the test case in branch-2.3, which

[jira] [Commented] (SPARK-25071) BuildSide is coming not as expected with join queries

2018-10-15 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25071?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651113#comment-16651113 ] Sujith commented on SPARK-25071: cc [~ZenWzh]  Please suggest as i want to take up this JIRA >

[jira] [Resolved] (SPARK-25629) ParquetFilterSuite: filter pushdown - decimal 16 sec

2018-10-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25629. -- Resolution: Fixed Assignee: Yuming Wang Fix Version/s: 3.0.0 Fixed in

[jira] [Commented] (SPARK-22601) Data load is getting displayed successful on providing non existing hdfs file path

2018-10-15 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651097#comment-16651097 ] Sujith commented on SPARK-22601: cc *[gatorsmile|https://github.com/gatorsmile]  please assign this JIRA 

[jira] [Resolved] (SPARK-22601) Data load is getting displayed successful on providing non existing hdfs file path

2018-10-15 Thread Sujith (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sujith resolved SPARK-22601. Resolution: Fixed Fix Version/s: 2.2.1 PR already merged. > Data load is getting displayed

[jira] [Commented] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651081#comment-16651081 ] Apache Spark commented on SPARK-25588: -- User 'heuermh' has created a pull request for this issue:

[jira] [Commented] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

2018-10-15 Thread Michael Heuer (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651080#comment-16651080 ] Michael Heuer commented on SPARK-25588: --- Created pull request

[jira] [Assigned] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25588: Assignee: (was: Apache Spark) > SchemaParseException: Can't redefine: list when

[jira] [Assigned] (SPARK-25588) SchemaParseException: Can't redefine: list when reading from Parquet

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25588: Assignee: Apache Spark > SchemaParseException: Can't redefine: list when reading from

[jira] [Updated] (SPARK-25714) Null Handling in the Optimizer rule BooleanSimplification

2018-10-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-25714: Fix Version/s: 2.3.3 2.2.3 > Null Handling in the Optimizer rule

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

2018-10-15 Thread Jackey Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16651061#comment-16651061 ] Jackey Lee commented on SPARK-24630: [~shijinkui] Without stream keyword means SQLStreaming cannot

[jira] [Updated] (SPARK-25723) spark sql External DataSource question

2018-10-15 Thread huanghuai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Description: {code:java} public class MyDatasourceRelation extends BaseRelation implements

[jira] [Updated] (SPARK-25723) spark sql External DataSource question

2018-10-15 Thread huanghuai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Description: {code:java} public class MyDatasourceRelation extends BaseRelation implements

[jira] [Resolved] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

2018-10-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-25738. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 22733

[jira] [Assigned] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

2018-10-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25738: -- Assignee: Imran Rashid > LOAD DATA INPATH doesn't work if hdfs conf includes port >

[jira] [Assigned] (SPARK-23257) Implement Kerberos Support in Kubernetes resource manager

2018-10-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23257: -- Assignee: Ilan Filonenko > Implement Kerberos Support in Kubernetes resource manager

[jira] [Resolved] (SPARK-23257) Implement Kerberos Support in Kubernetes resource manager

2018-10-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23257. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 21669

[jira] [Resolved] (SPARK-25716) Project and Aggregate generate valid constraints with unnecessary operation

2018-10-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25716?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-25716. - Resolution: Fixed Assignee: SongYadong Fix Version/s: 3.0.0 > Project and Aggregate

[jira] [Comment Edited] (SPARK-25643) Performance issues querying wide rows

2018-10-15 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650866#comment-16650866 ] Bruce Robbins edited comment on SPARK-25643 at 10/15/18 10:08 PM: --

[jira] [Commented] (SPARK-25643) Performance issues querying wide rows

2018-10-15 Thread Bruce Robbins (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650866#comment-16650866 ] Bruce Robbins commented on SPARK-25643: --- [~viirya] Yes, in the case where I said "predicate push

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-15 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code:java} val df =

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-15 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code} val df =

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-15 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code:scala} val df =

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-15 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Affects Version/s: (was: 2.3.2) 2.3.1 > Double quote coming in as

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-15 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code:java} val df =

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-15 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Environment:  Databricks - 4.2 (includes Apache Spark 2.3.1, Scala 2.11)  (was:   Databricks - 

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-15 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Description:  Example code -  {code:java} val df =

[jira] [Updated] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-15 Thread Brian Jones (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Jones updated SPARK-25739: Environment:   Databricks - 4.2 (includes Apache Spark 2.3.1, Scala 2.11)  was:  Example code

[jira] [Created] (SPARK-25739) Double quote coming in as empty value even when emptyValue set as null

2018-10-15 Thread Brian Jones (JIRA)
Brian Jones created SPARK-25739: --- Summary: Double quote coming in as empty value even when emptyValue set as null Key: SPARK-25739 URL: https://issues.apache.org/jira/browse/SPARK-25739 Project: Spark

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650817#comment-16650817 ] Daniel Darabos commented on SPARK-20144: Thanks, those are good questions. # The global option

[jira] [Assigned] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25738: Assignee: (was: Apache Spark) > LOAD DATA INPATH doesn't work if hdfs conf includes

[jira] [Assigned] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25738: Assignee: Apache Spark > LOAD DATA INPATH doesn't work if hdfs conf includes port >

[jira] [Commented] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650807#comment-16650807 ] Apache Spark commented on SPARK-25738: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

2018-10-15 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650799#comment-16650799 ] Imran Rashid commented on SPARK-25738: -- the fix is pretty trivial, I'm posting a pr now > LOAD

[jira] [Commented] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

2018-10-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650796#comment-16650796 ] Shixiong Zhu commented on SPARK-25738: -- Marked as a blocker since this is a regression > LOAD DATA

[jira] [Updated] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

2018-10-15 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-25738: - Priority: Blocker (was: Critical) > LOAD DATA INPATH doesn't work if hdfs conf includes port >

[jira] [Created] (SPARK-25738) LOAD DATA INPATH doesn't work if hdfs conf includes port

2018-10-15 Thread Imran Rashid (JIRA)
Imran Rashid created SPARK-25738: Summary: LOAD DATA INPATH doesn't work if hdfs conf includes port Key: SPARK-25738 URL: https://issues.apache.org/jira/browse/SPARK-25738 Project: Spark

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650779#comment-16650779 ] Dongjoon Hyun commented on SPARK-20144: --- [~silvermast] and [~darabos].  1. The proposed

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650722#comment-16650722 ] Daniel Darabos commented on SPARK-20144: Yeah, I'm not too happy about the alphabetical ordering

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Victor Tso (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650704#comment-16650704 ] Victor Tso commented on SPARK-20144: It should, because by convention the parquet files are 0-padded 

[jira] [Resolved] (SPARK-25424) Window duration and slide duration with negative values should fail fast

2018-10-15 Thread Raghav Kumar Gautam (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghav Kumar Gautam resolved SPARK-25424. - Resolution: Not A Bug > Window duration and slide duration with negative values

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650642#comment-16650642 ] Dongjoon Hyun commented on SPARK-20144: --- For me, I don't think that PR resolve this issue,

[jira] [Commented] (SPARK-25044) Address translation of LMF closure primitive args to Object in Scala 2.12

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25044?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650608#comment-16650608 ] Apache Spark commented on SPARK-25044: -- User 'maryannxue' has created a pull request for this

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650591#comment-16650591 ] Marcelo Vanzin commented on SPARK-25732: In fact, do you even need proxy user + keytab at all?

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-15 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650560#comment-16650560 ] Thomas Graves commented on SPARK-25732: --- I would much rather see Spark start to push tokens and

[jira] [Updated] (SPARK-25547) Pluggable jdbc connection factory

2018-10-15 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25547?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25547: Target Version/s: 3.0.0 > Pluggable jdbc connection factory > - > >

[jira] [Commented] (SPARK-25674) If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650541#comment-16650541 ] Apache Spark commented on SPARK-25674: -- User 'gatorsmile' has created a pull request for this

[jira] [Commented] (SPARK-25674) If the records are incremented by more than 1 at a time,the number of bytes might rarely ever get updated

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25674?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650539#comment-16650539 ] Apache Spark commented on SPARK-25674: -- User 'gatorsmile' has created a pull request for this

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650535#comment-16650535 ] Marcelo Vanzin commented on SPARK-25732: I'd have preferred a system where Livy handles this for

[jira] [Commented] (SPARK-18278) SPIP: Support native submission of spark jobs to a kubernetes cluster

2018-10-15 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650524#comment-16650524 ] Matt Cheah commented on SPARK-18278: The fork is no longer being maintained, because Kubernetes

[jira] [Commented] (SPARK-18278) SPIP: Support native submission of spark jobs to a kubernetes cluster

2018-10-15 Thread Oleg Frenkel (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650519#comment-16650519 ] Oleg Frenkel commented on SPARK-18278: -- When is this fork expected to make its way to the official

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Daniel Darabos (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650492#comment-16650492 ] Daniel Darabos commented on SPARK-20144: Thanks Victor! I've expanded the test with a case where

[jira] [Assigned] (SPARK-16775) Remove deprecated accumulator v1 APIs

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16775: Assignee: (was: Apache Spark) > Remove deprecated accumulator v1 APIs >

[jira] [Commented] (SPARK-16775) Remove deprecated accumulator v1 APIs

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650472#comment-16650472 ] Apache Spark commented on SPARK-16775: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-16775) Remove deprecated accumulator v1 APIs

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-16775: Assignee: Apache Spark > Remove deprecated accumulator v1 APIs >

[jira] [Commented] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

2018-10-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650451#comment-16650451 ] Marcelo Vanzin commented on SPARK-13478: You don't need a keytab to log in to kerberos... >

[jira] [Assigned] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25737: Assignee: Sean Owen (was: Apache Spark) > Remove JavaSparkContextVarargsWorkaround and

[jira] [Assigned] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25737: Assignee: Apache Spark (was: Sean Owen) > Remove JavaSparkContextVarargsWorkaround and

[jira] [Commented] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650425#comment-16650425 ] Apache Spark commented on SPARK-25737: -- User 'srowen' has created a pull request for this issue:

[jira] [Updated] (SPARK-16775) Remove deprecated accumulator v1 APIs

2018-10-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16775: -- Affects Version/s: (was: 2.1.0) 3.0.0 Target Version/s: 3.0.0

[jira] [Resolved] (SPARK-24154) AccumulatorV2 loses type information during serialization

2018-10-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-24154. --- Resolution: Won't Fix > AccumulatorV2 loses type information during serialization >

[jira] [Updated] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

2018-10-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25737: -- Environment: (was: In ancient times in 2013, JavaSparkContext got a superclass

[jira] [Updated] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

2018-10-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25737: -- Target Version/s: 3.0.0 > Remove JavaSparkContextVarargsWorkaround and standardize union() methods >

[jira] [Created] (SPARK-25737) Remove JavaSparkContextVarargsWorkaround and standardize union() methods

2018-10-15 Thread Sean Owen (JIRA)
Sean Owen created SPARK-25737: - Summary: Remove JavaSparkContextVarargsWorkaround and standardize union() methods Key: SPARK-25737 URL: https://issues.apache.org/jira/browse/SPARK-25737 Project: Spark

[jira] [Assigned] (SPARK-25736) add tests to verify the behavior of multi-column count

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25736: Assignee: Apache Spark (was: Wenchen Fan) > add tests to verify the behavior of

[jira] [Commented] (SPARK-25736) add tests to verify the behavior of multi-column count

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650375#comment-16650375 ] Apache Spark commented on SPARK-25736: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25736) add tests to verify the behavior of multi-column count

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25736: Assignee: Wenchen Fan (was: Apache Spark) > add tests to verify the behavior of

[jira] [Created] (SPARK-25736) add tests to verify the behavior of multi-column count

2018-10-15 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-25736: --- Summary: add tests to verify the behavior of multi-column count Key: SPARK-25736 URL: https://issues.apache.org/jira/browse/SPARK-25736 Project: Spark Issue

[jira] [Commented] (SPARK-25369) Replace Java shim functional interfaces like java.api.Function with Java 8 equivalents

2018-10-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650356#comment-16650356 ] Sean Owen commented on SPARK-25369: --- Example: {code:java} @FunctionalInterface public interface

[jira] [Commented] (SPARK-20144) spark.read.parquet no long maintains ordering of the data

2018-10-15 Thread Victor Tso (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650303#comment-16650303 ] Victor Tso commented on SPARK-20144: I looked at the PR and liked what I saw. I would only suggest

[jira] [Commented] (SPARK-13478) Fetching delegation tokens for Hive fails when using proxy users

2018-10-15 Thread Sunayan Saikia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650302#comment-16650302 ] Sunayan Saikia commented on SPARK-13478: [~vanzin] As we know 'spark-submit' command by itself

[jira] [Commented] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650247#comment-16650247 ] Apache Spark commented on SPARK-25735: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650244#comment-16650244 ] Apache Spark commented on SPARK-25735: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25735: Assignee: (was: Apache Spark) > Improve start-thriftserver.sh: print clean usage and

[jira] [Assigned] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25735: Assignee: Apache Spark > Improve start-thriftserver.sh: print clean usage and exit with

[jira] [Created] (SPARK-25735) Improve start-thriftserver.sh: print clean usage and exit with code 1

2018-10-15 Thread Gengliang Wang (JIRA)
Gengliang Wang created SPARK-25735: -- Summary: Improve start-thriftserver.sh: print clean usage and exit with code 1 Key: SPARK-25735 URL: https://issues.apache.org/jira/browse/SPARK-25735 Project:

[jira] [Updated] (SPARK-25723) spark sql External DataSource question

2018-10-15 Thread huanghuai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Priority: Minor (was: Major) > spark sql External DataSource question >

[jira] [Commented] (SPARK-25727) makeCopy failed in InMemoryRelation

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650153#comment-16650153 ] Apache Spark commented on SPARK-25727: -- User 'mgaido91' has created a pull request for this issue:

[jira] [Commented] (SPARK-24610) wholeTextFiles broken for small files

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650109#comment-16650109 ] Apache Spark commented on SPARK-24610: -- User '10110346' has created a pull request for this issue:

[jira] [Commented] (SPARK-24610) wholeTextFiles broken for small files

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24610?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650106#comment-16650106 ] Apache Spark commented on SPARK-24610: -- User '10110346' has created a pull request for this issue:

[jira] [Commented] (SPARK-25734) Literal should have a value corresponding to dataType

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16650059#comment-16650059 ] Apache Spark commented on SPARK-25734: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25734) Literal should have a value corresponding to dataType

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25734: Assignee: Apache Spark > Literal should have a value corresponding to dataType >

[jira] [Assigned] (SPARK-25734) Literal should have a value corresponding to dataType

2018-10-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25734: Assignee: (was: Apache Spark) > Literal should have a value corresponding to

[jira] [Created] (SPARK-25734) Literal should have a value corresponding to dataType

2018-10-15 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-25734: Summary: Literal should have a value corresponding to dataType Key: SPARK-25734 URL: https://issues.apache.org/jira/browse/SPARK-25734 Project: Spark

[jira] [Reopened] (SPARK-25723) spark sql External DataSource question

2018-10-15 Thread huanghuai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai reopened SPARK-25723: --- reproduce > spark sql External DataSource question > -- > >

[jira] [Updated] (SPARK-25723) spark sql External DataSource question

2018-10-15 Thread huanghuai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Description: trait PrunedFilteredScan { def buildScan(requiredColumns: Array[String], filters:

[jira] [Updated] (SPARK-25723) spark sql External DataSource

2018-10-15 Thread huanghuai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Attachment: QQ图片20181015182502.jpg > spark sql External DataSource > - >

[jira] [Updated] (SPARK-25723) spark sql External DataSource

2018-10-15 Thread huanghuai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25723?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] huanghuai updated SPARK-25723: -- Description: (was: {color:#33}*spark.read()*{color}

[jira] [Commented] (SPARK-25527) Job stuck waiting for last stage to start

2018-10-15 Thread Ran Haim (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25527?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649990#comment-16649990 ] Ran Haim commented on SPARK-25527: -- Any update? > Job stuck waiting for last stage to start >

[jira] [Updated] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

2018-10-15 Thread Bihui Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bihui Jin updated SPARK-25733: -- Environment: Spark in standalone mode, and 48 cores are available. spark-defaults.conf as blew:

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

2018-10-15 Thread shijinkui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649972#comment-16649972 ] shijinkui commented on SPARK-24630: --- I prefer without stream keyword. Because in the future bath and

[jira] [Updated] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

2018-10-15 Thread Bihui Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bihui Jin updated SPARK-25733: -- Attachment: report_dataset.zip.002 > The method toLocalIterator() with dataframe doesn't work >

[jira] [Updated] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

2018-10-15 Thread Bihui Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bihui Jin updated SPARK-25733: -- Attachment: report_dataset.zip.001 > The method toLocalIterator() with dataframe doesn't work >

[jira] [Updated] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

2018-10-15 Thread Bihui Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bihui Jin updated SPARK-25733: -- Description: {color:#FF}The dataset which I used attached.{color}   First I loaded a dataframe

[jira] [Created] (SPARK-25733) The method toLocalIterator() with dataframe doesn't work

2018-10-15 Thread Bihui Jin (JIRA)
Bihui Jin created SPARK-25733: - Summary: The method toLocalIterator() with dataframe doesn't work Key: SPARK-25733 URL: https://issues.apache.org/jira/browse/SPARK-25733 Project: Spark Issue

[jira] [Updated] (SPARK-25731) Spark Structured Streaming Support for Kafka 2.0

2018-10-15 Thread Chandan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-25731: Labels: beginner features (was: ) > Spark Structured Streaming Support for Kafka 2.0 >

[jira] [Commented] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-15 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25732?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16649917#comment-16649917 ] Marco Gaido commented on SPARK-25732: - cc [~vanzin] [~tgraves] [~jerryshao] [~mridul]. Sorry for

[jira] [Updated] (SPARK-25731) Spark Structured Streaming Support for Kafka 2.0

2018-10-15 Thread Chandan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-25731: Description: [https://github.com/apache/spark/tree/master/external] As far as I can see, This doesn't

[jira] [Created] (SPARK-25732) Allow specifying a keytab/principal for proxy user for token renewal

2018-10-15 Thread Marco Gaido (JIRA)
Marco Gaido created SPARK-25732: --- Summary: Allow specifying a keytab/principal for proxy user for token renewal Key: SPARK-25732 URL: https://issues.apache.org/jira/browse/SPARK-25732 Project: Spark

[jira] [Updated] (SPARK-25731) Spark Structured Streaming Support for Kafka 2.0

2018-10-15 Thread Chandan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chandan updated SPARK-25731: Description: [https://github.com/apache/spark/tree/master/external] As far as I can see, This doesn't

  1   2   >