[jira] [Updated] (SPARK-23191) Workers registration failes in case of network drop

2018-03-19 Thread Neeraj Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neeraj Gupta updated SPARK-23191: - Priority: Critical (was: Major) > Workers registration failes in case of network drop >

[jira] [Updated] (SPARK-23191) Workers registration failes in case of network drop

2018-03-19 Thread Neeraj Gupta (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23191?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Neeraj Gupta updated SPARK-23191: - Affects Version/s: 2.3.0 > Workers registration failes in case of network drop >

[jira] [Commented] (SPARK-23691) Use sql_conf util in PySpark tests where possible

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405833#comment-16405833 ] Apache Spark commented on SPARK-23691: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Commented] (SPARK-23691) Use sql_conf util in PySpark tests where possible

2018-03-19 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405806#comment-16405806 ] Bryan Cutler commented on SPARK-23691: -- Thanks [~hyukjin.kwon]! > Use sql_conf util in PySpark

[jira] [Commented] (SPARK-23691) Use sql_conf util in PySpark tests where possible

2018-03-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405799#comment-16405799 ] Hyukjin Kwon commented on SPARK-23691: -- (y) > Use sql_conf util in PySpark tests where possible >

[jira] [Resolved] (SPARK-23691) Use sql_conf util in PySpark tests where possible

2018-03-19 Thread Bryan Cutler (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Cutler resolved SPARK-23691. -- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.4.0 Issue resolved

[jira] [Commented] (SPARK-23691) Use sql_conf util in PySpark tests where possible

2018-03-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405795#comment-16405795 ] Hyukjin Kwon commented on SPARK-23691: -- [~bryanc], I just gave you a committer role in JIRA. You

[jira] [Commented] (SPARK-23737) Scala API documentation leads to nonexistent pages for sources

2018-03-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405794#comment-16405794 ] Hyukjin Kwon commented on SPARK-23737: -- Is it a duplicate of SPARK-23732? > Scala API documentation

[jira] [Commented] (SPARK-23744) Memory leak in ReadableChannelFileRegion

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23744?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405780#comment-16405780 ] Apache Spark commented on SPARK-23744: -- User '10110346' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23744) Memory leak in ReadableChannelFileRegion

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23744: Assignee: (was: Apache Spark) > Memory leak in ReadableChannelFileRegion >

[jira] [Assigned] (SPARK-23744) Memory leak in ReadableChannelFileRegion

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23744: Assignee: Apache Spark > Memory leak in ReadableChannelFileRegion >

[jira] [Updated] (SPARK-23744) Memory leak in ReadableChannelFileRegion

2018-03-19 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-23744: Description: In the class _ReadableChannelFileRegion_,  the _buffer_ is direct memory, we should  modify 

[jira] [Updated] (SPARK-23744) Memory leak in ReadableChannelFileRegion

2018-03-19 Thread liuxian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] liuxian updated SPARK-23744: Description: In the class _ReadableChannelFileRegion_,  the _buffer_ is direct memory, we should  modify 

[jira] [Created] (SPARK-23744) Memory leak in ReadableChannelFileRegion

2018-03-19 Thread liuxian (JIRA)
liuxian created SPARK-23744: --- Summary: Memory leak in ReadableChannelFileRegion Key: SPARK-23744 URL: https://issues.apache.org/jira/browse/SPARK-23744 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23599) The UUID() expression is too non-deterministic

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405764#comment-16405764 ] Apache Spark commented on SPARK-23599: -- User 'viirya' has created a pull request for this issue:

[jira] [Commented] (SPARK-23743) IsolatedClientLoader.isSharedClass returns an unindented result against `slf4j` keyword

2018-03-19 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405759#comment-16405759 ] Jongyoul Lee commented on SPARK-23743: -- Understood. Thanks for fixing it. >

[jira] [Commented] (SPARK-23743) IsolatedClientLoader.isSharedClass returns an unindented result against `slf4j` keyword

2018-03-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405756#comment-16405756 ] Hyukjin Kwon commented on SPARK-23743: -- (let's avoid to set a target version which is usually set by

[jira] [Updated] (SPARK-23743) IsolatedClientLoader.isSharedClass returns an unindented result against `slf4j` keyword

2018-03-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23743: - Target Version/s: (was: 2.4.0) > IsolatedClientLoader.isSharedClass returns an unindented

[jira] [Updated] (SPARK-23743) IsolatedClientLoader.isSharedClass returns an unindented result against `slf4j` keyword

2018-03-19 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-23743: - Summary: IsolatedClientLoader.isSharedClass returns an unindented result against `slf4j` keyword

[jira] [Updated] (SPARK-23743) IsolatedClientLoader.isSharedClass returns unindented results of `slf4j` keyword

2018-03-19 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jongyoul Lee updated SPARK-23743: - Summary: IsolatedClientLoader.isSharedClass returns unindented results of `slf4j` keyword (was:

[jira] [Updated] (SPARK-23644) SHS with proxy doesn't show applications

2018-03-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23644: Fix Version/s: 2.3.1 > SHS with proxy doesn't show applications >

[jira] [Assigned] (SPARK-23743) IsolatedClientLoader.isSharedClass return unindented results of `slf4j` keyword

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23743: Assignee: (was: Apache Spark) > IsolatedClientLoader.isSharedClass return unindented

[jira] [Assigned] (SPARK-23743) IsolatedClientLoader.isSharedClass return unindented results of `slf4j` keyword

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23743: Assignee: Apache Spark > IsolatedClientLoader.isSharedClass return unindented results of

[jira] [Commented] (SPARK-23743) IsolatedClientLoader.isSharedClass return unindented results of `slf4j` keyword

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405690#comment-16405690 ] Apache Spark commented on SPARK-23743: -- User 'jongyoul' has created a pull request for this issue:

[jira] [Commented] (SPARK-23743) IsolatedClientLoader.isSharedClass return unindented results of `slf4j` keyword

2018-03-19 Thread Jongyoul Lee (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23743?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405684#comment-16405684 ] Jongyoul Lee commented on SPARK-23743: -- Please assign this issue to me. >

[jira] [Created] (SPARK-23743) IsolatedClientLoader.isSharedClass return unindented results of `slf4j` keyword

2018-03-19 Thread Jongyoul Lee (JIRA)
Jongyoul Lee created SPARK-23743: Summary: IsolatedClientLoader.isSharedClass return unindented results of `slf4j` keyword Key: SPARK-23743 URL: https://issues.apache.org/jira/browse/SPARK-23743

[jira] [Assigned] (SPARK-23660) Yarn throws exception in cluster mode when the application is small

2018-03-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23660: -- Assignee: Gabor Somogyi > Yarn throws exception in cluster mode when the application

[jira] [Resolved] (SPARK-23660) Yarn throws exception in cluster mode when the application is small

2018-03-19 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23660?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23660. Resolution: Fixed Fix Version/s: 2.3.1 2.4.0 Issue resolved by

[jira] [Issue Comment Deleted] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-23534: Comment: was deleted (was: I'm curious why there's a library which depends on a such new version

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405659#comment-16405659 ] Saisai Shao commented on SPARK-23534: - I'm curious why there's a library which depends on a such new

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405658#comment-16405658 ] Saisai Shao commented on SPARK-23534: - I'm curious why there's a library which depends on a such new

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-19 Thread Darek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405653#comment-16405653 ] Darek commented on SPARK-23534: --- It's not about pure Hadoop 3.0, it's about the rest of the libraries that

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-19 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405649#comment-16405649 ] Saisai Shao commented on SPARK-23534: - I'm trying to move forward this in the community. But due to

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-19 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405615#comment-16405615 ] Nicholas Chammas commented on SPARK-23534: -- I don't know what it takes to add a Hadoop 3.0 build

[jira] [Commented] (SPARK-18673) Dataframes doesn't work on Hadoop 3.x; Hive rejects Hadoop version

2018-03-19 Thread Darek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18673?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405604#comment-16405604 ] Darek commented on SPARK-18673: --- [~joshrosen] Would you know who can help adding the current version of

[jira] [Commented] (SPARK-22513) Provide build profile for hadoop 2.8

2018-03-19 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405598#comment-16405598 ] Sean Owen commented on SPARK-22513: --- I say this because Hadoop ought to be backwards-compatible in

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-19 Thread Darek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405592#comment-16405592 ] Darek commented on SPARK-23534: --- There has been no progress on this ticket, do you know anyone that can

[jira] [Commented] (SPARK-23499) Mesos Cluster Dispatcher should support priority queues to submit drivers

2018-03-19 Thread Susan X. Huynh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405586#comment-16405586 ] Susan X. Huynh commented on SPARK-23499: [~pgillet] Here's a different issue: in the proposed

[jira] [Commented] (SPARK-23672) Support returning lists in Arrow UDFs

2018-03-19 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23672?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405574#comment-16405574 ] holdenk commented on SPARK-23672: - cc [~bryanc] for thoughts. > Support returning lists in Arrow UDFs >

[jira] [Commented] (SPARK-23534) Spark run on Hadoop 3.0.0

2018-03-19 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405563#comment-16405563 ] Nicholas Chammas commented on SPARK-23534: -- I believe this ticket is a duplicate of SPARK-23151,

[jira] [Commented] (SPARK-22513) Provide build profile for hadoop 2.8

2018-03-19 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405561#comment-16405561 ] Nicholas Chammas commented on SPARK-22513: -- [~srowen] - Just curious: How do you know that Spark

[jira] [Created] (SPARK-23742) Filter out redundant AssociationRules

2018-03-19 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-23742: - Summary: Filter out redundant AssociationRules Key: SPARK-23742 URL: https://issues.apache.org/jira/browse/SPARK-23742 Project: Spark Issue Type:

[jira] [Updated] (SPARK-19602) Unable to query using the fully qualified column name of the form ( ..)

2018-03-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19602: Target Version/s: 2.4.0 > Unable to query using the fully qualified column name of the form ( > ..) >

[jira] [Assigned] (SPARK-19602) Unable to query using the fully qualified column name of the form ( ..)

2018-03-19 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-19602: --- Assignee: Sunitha Kambhampati > Unable to query using the fully qualified column name of the form (

[jira] [Created] (SPARK-23741) UTF8String must not return chars disallowed in UTF-8

2018-03-19 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-23741: -- Summary: UTF8String must not return chars disallowed in UTF-8 Key: SPARK-23741 URL: https://issues.apache.org/jira/browse/SPARK-23741 Project: Spark Issue Type:

[jira] [Created] (SPARK-23740) Add FPGrowth Param for filtering out very common items

2018-03-19 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-23740: - Summary: Add FPGrowth Param for filtering out very common items Key: SPARK-23740 URL: https://issues.apache.org/jira/browse/SPARK-23740 Project: Spark

[jira] [Assigned] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22839: Assignee: (was: Apache Spark) > Refactor Kubernetes code for configuring

[jira] [Assigned] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22839: Assignee: Apache Spark > Refactor Kubernetes code for configuring driver/executor pods to

[jira] [Comment Edited] (SPARK-23690) VectorAssembler should have handleInvalid to handle columns with null values

2018-03-19 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405174#comment-16405174 ] yogesh garg edited comment on SPARK-23690 at 3/19/18 7:04 PM: -- In an offline

[jira] [Comment Edited] (SPARK-23690) VectorAssembler should have handleInvalid to handle columns with null values

2018-03-19 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405174#comment-16405174 ] yogesh garg edited comment on SPARK-23690 at 3/19/18 7:03 PM: -- In an offline

[jira] [Commented] (SPARK-23702) Forbid watermarks on both sides of a streaming aggregate

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405274#comment-16405274 ] Apache Spark commented on SPARK-23702: -- User 'jose-torres' has created a pull request for this

[jira] [Reopened] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

2018-03-19 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah reopened SPARK-22839: Actually we haven't done the refactor in and of itself yet,

[jira] [Resolved] (SPARK-22839) Refactor Kubernetes code for configuring driver/executor pods to use consistent and cleaner abstraction

2018-03-19 Thread Matt Cheah (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22839?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Cheah resolved SPARK-22839. Resolution: Fixed Fix Version/s: 2.4.0 Done in https://github.com/apache/spark/pull/20669

[jira] [Commented] (SPARK-23722) Add support for inserting empty map or array to table

2018-03-19 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23722?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405181#comment-16405181 ] Sital Kedia commented on SPARK-23722: - Actually this is a duplicate of SPARK-21281 and the issue has

[jira] [Resolved] (SPARK-23722) Add support for inserting empty map or array to table

2018-03-19 Thread Sital Kedia (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23722?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sital Kedia resolved SPARK-23722. - Resolution: Duplicate > Add support for inserting empty map or array to table >

[jira] [Resolved] (SPARK-23728) ML test with expected exceptions testing streaming fails on 2.3

2018-03-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley resolved SPARK-23728. --- Resolution: Fixed Fix Version/s: 2.3.1 Issue resolved by pull request 20852

[jira] [Assigned] (SPARK-23728) ML test with expected exceptions testing streaming fails on 2.3

2018-03-19 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley reassigned SPARK-23728: - Assignee: Attila Zsolt Piros > ML test with expected exceptions testing

[jira] [Commented] (SPARK-23690) VectorAssembler should have handleInvalid to handle columns with null values

2018-03-19 Thread yogesh garg (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23690?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16405174#comment-16405174 ] yogesh garg commented on SPARK-23690: - In an offline discussion with [~mrbago], we discussed the

[jira] [Updated] (SPARK-23739) Spark structured streaming long running problem

2018-03-19 Thread Florencio (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Florencio updated SPARK-23739: -- Summary: Spark structured streaming long running problem (was: Spark structured streaming long

[jira] [Created] (SPARK-23739) Spark structured streaming long running

2018-03-19 Thread Florencio (JIRA)
Florencio created SPARK-23739: - Summary: Spark structured streaming long running Key: SPARK-23739 URL: https://issues.apache.org/jira/browse/SPARK-23739 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-23738) Memory usage for executors

2018-03-19 Thread assia ydroudj (JIRA)
assia ydroudj created SPARK-23738: - Summary: Memory usage for executors Key: SPARK-23738 URL: https://issues.apache.org/jira/browse/SPARK-23738 Project: Spark Issue Type: Question

[jira] [Updated] (SPARK-23737) Scala API documentation leads to nonexistent pages for sources

2018-03-19 Thread Alexander Bessonov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23737?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alexander Bessonov updated SPARK-23737: --- Description: h3. Steps to reproduce: # Go to [Scala API

[jira] [Created] (SPARK-23737) Scala API documentation leads to nonexistent pages for sources

2018-03-19 Thread Alexander Bessonov (JIRA)
Alexander Bessonov created SPARK-23737: -- Summary: Scala API documentation leads to nonexistent pages for sources Key: SPARK-23737 URL: https://issues.apache.org/jira/browse/SPARK-23737 Project:

[jira] [Assigned] (SPARK-23736) collection function: concat_arrays

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23736: Assignee: Apache Spark > collection function: concat_arrays >

[jira] [Assigned] (SPARK-23736) collection function: concat_arrays

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23736: Assignee: (was: Apache Spark) > collection function: concat_arrays >

[jira] [Commented] (SPARK-23736) collection function: concat_arrays

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23736?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404937#comment-16404937 ] Apache Spark commented on SPARK-23736: -- User 'mn-mikke' has created a pull request for this issue:

[jira] [Updated] (SPARK-23736) collection function: concat_arrays

2018-03-19 Thread Marek Novotny (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23736?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marek Novotny updated SPARK-23736: -- Issue Type: New Feature (was: Story) > collection function: concat_arrays >

[jira] [Commented] (SPARK-23724) Custom record separator for jsons in charsets different from UTF-8

2018-03-19 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404815#comment-16404815 ] Maxim Gekk commented on SPARK-23724: For this issue I plan to propose this PR:

[jira] [Created] (SPARK-23736) collection function: concat_arrays

2018-03-19 Thread Marek Novotny (JIRA)
Marek Novotny created SPARK-23736: - Summary: collection function: concat_arrays Key: SPARK-23736 URL: https://issues.apache.org/jira/browse/SPARK-23736 Project: Spark Issue Type: Story

[jira] [Assigned] (SPARK-23735) Optimize the document by adding an import streaming configuration

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23735: Assignee: Apache Spark > Optimize the document by adding an import streaming

[jira] [Assigned] (SPARK-23735) Optimize the document by adding an import streaming configuration

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23735: Assignee: (was: Apache Spark) > Optimize the document by adding an import streaming

[jira] [Commented] (SPARK-23735) Optimize the document by adding an import streaming configuration

2018-03-19 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404534#comment-16404534 ] Apache Spark commented on SPARK-23735: -- User 'liuxianjiao' has created a pull request for this

[jira] [Created] (SPARK-23735) Optimize the document by adding an import streaming configuration

2018-03-19 Thread liuxianjiao (JIRA)
liuxianjiao created SPARK-23735: --- Summary: Optimize the document by adding an import streaming configuration Key: SPARK-23735 URL: https://issues.apache.org/jira/browse/SPARK-23735 Project: Spark

[jira] [Commented] (SPARK-23599) The UUID() expression is too non-deterministic

2018-03-19 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404511#comment-16404511 ] Herman van Hovell commented on SPARK-23599: --- PR 1 out of 2 has been merged. > The UUID()

[jira] [Commented] (SPARK-23244) Incorrect handling of default values when deserializing python wrappers of scala transformers

2018-03-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404448#comment-16404448 ] Hyukjin Kwon commented on SPARK-23244: -- Shall we resolve all duplicates for now? Looks a bit

[jira] [Resolved] (SPARK-23692) Print metadata of files when infer schema failed

2018-03-19 Thread zhoukang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23692?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhoukang resolved SPARK-23692. -- Resolution: Won't Fix > Print metadata of files when infer schema failed >

[jira] [Commented] (SPARK-23724) Custom record separator for jsons in charsets different from UTF-8

2018-03-19 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23724?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16404416#comment-16404416 ] Hyukjin Kwon commented on SPARK-23724: -- Just to be clear, do you plan to rewrite