[jira] [Created] (SPARK-23805) support vector-size validation and Inference

2018-03-28 Thread zhengruifeng (JIRA)
zhengruifeng created SPARK-23805: Summary: support vector-size validation and Inference Key: SPARK-23805 URL: https://issues.apache.org/jira/browse/SPARK-23805 Project: Spark Issue Type:

[jira] [Commented] (SPARK-23565) Improved error message for when the number of sources for a query changes

2018-03-28 Thread Patrick McGloin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417206#comment-16417206 ] Patrick McGloin commented on SPARK-23565: - I would like to work on this. > Improved error

[jira] [Assigned] (SPARK-23705) dataframe.groupBy() may inadvertently receive sequence of non-distinct strings

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23705: Assignee: Apache Spark > dataframe.groupBy() may inadvertently receive sequence of

[jira] [Commented] (SPARK-23705) dataframe.groupBy() may inadvertently receive sequence of non-distinct strings

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417064#comment-16417064 ] Apache Spark commented on SPARK-23705: -- User 'vinodkc' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23705) dataframe.groupBy() may inadvertently receive sequence of non-distinct strings

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23705?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23705: Assignee: (was: Apache Spark) > dataframe.groupBy() may inadvertently receive

[jira] [Commented] (SPARK-23782) SHS should not show applications to user without read permission

2018-03-28 Thread Marco Gaido (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417149#comment-16417149 ] Marco Gaido commented on SPARK-23782: - [~vanzin] sorry but I cannot see any usability issue with

[jira] [Commented] (SPARK-19076) Upgrade Hive dependence to Hive 2.x

2018-03-28 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-19076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417291#comment-16417291 ] Maciej Bryński commented on SPARK-19076: So it's not resolved but in progress > Upgrade Hive

[jira] [Resolved] (SPARK-23765) Supports line separator for json datasource

2018-03-28 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23765. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20877

[jira] [Commented] (SPARK-19076) Upgrade Hive dependence to Hive 2.x

2018-03-28 Thread Darek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417284#comment-16417284 ] Darek commented on SPARK-19076: --- [PR 20659|https://github.com/apache/spark/pull/20659] for this issue

[jira] [Assigned] (SPARK-23765) Supports line separator for json datasource

2018-03-28 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23765?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-23765: --- Assignee: Hyukjin Kwon > Supports line separator for json datasource >

[jira] [Commented] (SPARK-19076) Upgrade Hive dependence to Hive 2.x

2018-03-28 Thread Darek (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19076?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417293#comment-16417293 ] Darek commented on SPARK-19076: --- It's done, passed all the tests, just needs to be merged in, not sure who

[jira] [Assigned] (SPARK-23806) Broadcast. unpersist can cause fatal exception when used with dynamic allocation

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23806: Assignee: Apache Spark > Broadcast. unpersist can cause fatal exception when used with

[jira] [Assigned] (SPARK-23806) Broadcast. unpersist can cause fatal exception when used with dynamic allocation

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23806: Assignee: (was: Apache Spark) > Broadcast. unpersist can cause fatal exception when

[jira] [Commented] (SPARK-23806) Broadcast. unpersist can cause fatal exception when used with dynamic allocation

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417881#comment-16417881 ] Apache Spark commented on SPARK-23806: -- User 'tgravescs' has created a pull request for this issue:

[jira] [Commented] (SPARK-23782) SHS should not show applications to user without read permission

2018-03-28 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23782?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417751#comment-16417751 ] Marcelo Vanzin commented on SPARK-23782: All the information you're trying to protect can be

[jira] [Commented] (SPARK-23096) Migrate rate source to v2

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417787#comment-16417787 ] Apache Spark commented on SPARK-23096: -- User 'jose-torres' has created a pull request for this

[jira] [Updated] (SPARK-23806) Broadcast. unpersist can cause fatal exception when used with dynamic allocation

2018-03-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-23806: -- Description: Very similar to https://issues.apache.org/jira/browse/SPARK-22618 . But this

[jira] [Commented] (SPARK-23096) Migrate rate source to v2

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23096?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417848#comment-16417848 ] Apache Spark commented on SPARK-23096: -- User 'jose-torres' has created a pull request for this

[jira] [Commented] (SPARK-23807) Add Hadoop 3 profile with relevant POM fix ups, cloud-storage artifacts and binding

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417864#comment-16417864 ] Apache Spark commented on SPARK-23807: -- User 'steveloughran' has created a pull request for this

[jira] [Assigned] (SPARK-23807) Add Hadoop 3 profile with relevant POM fix ups, cloud-storage artifacts and binding

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23807: Assignee: Apache Spark > Add Hadoop 3 profile with relevant POM fix ups, cloud-storage

[jira] [Assigned] (SPARK-23807) Add Hadoop 3 profile with relevant POM fix ups, cloud-storage artifacts and binding

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23807?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23807: Assignee: (was: Apache Spark) > Add Hadoop 3 profile with relevant POM fix ups,

[jira] [Assigned] (SPARK-23811) Same tasks' FetchFailed event comes before Success will cause child stage never succeed

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23811: Assignee: (was: Apache Spark) > Same tasks' FetchFailed event comes before Success

[jira] [Commented] (SPARK-23811) Same tasks' FetchFailed event comes before Success will cause child stage never succeed

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418431#comment-16418431 ] Apache Spark commented on SPARK-23811: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-23811) Same tasks' FetchFailed event comes before Success will cause child stage never succeed

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23811: Assignee: Apache Spark > Same tasks' FetchFailed event comes before Success will cause

[jira] [Created] (SPARK-23815) Spark writer dynamic partition overwrite mode fails to write output on multi level partition

2018-03-28 Thread Fangshi Li (JIRA)
Fangshi Li created SPARK-23815: -- Summary: Spark writer dynamic partition overwrite mode fails to write output on multi level partition Key: SPARK-23815 URL: https://issues.apache.org/jira/browse/SPARK-23815

[jira] [Commented] (SPARK-23812) DFS should be removed from unsupportedHiveNativeCommands in SqlBase.g4

2018-03-28 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418430#comment-16418430 ] Yuming Wang commented on SPARK-23812: - [~wangtao93] Would you like to work on this? > DFS should be

[jira] [Commented] (SPARK-23815) Spark writer dynamic partition overwrite mode fails to write output on multi level partition

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418452#comment-16418452 ] Apache Spark commented on SPARK-23815: -- User 'fangshil' has created a pull request for this issue:

[jira] [Updated] (SPARK-23814) Couldn't read file with colon in name and new line character in one of the field.

2018-03-28 Thread bharath kumar avusherla (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bharath kumar avusherla updated SPARK-23814: Description: When the file name has colon and new line character in data,

[jira] [Updated] (SPARK-23814) Couldn't read file with colon in name and new line character in one of the field.

2018-03-28 Thread bharath kumar avusherla (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bharath kumar avusherla updated SPARK-23814: Description: When the file name has colon and new line character in data,

[jira] [Updated] (SPARK-23814) Couldn't read file with colon in name and new line character in one of the field.

2018-03-28 Thread bharath kumar avusherla (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] bharath kumar avusherla updated SPARK-23814: Description: When the file name has colon and new line character in data,

[jira] [Created] (SPARK-23814) Couldn't read file with colon in name and new line character in one of the field.

2018-03-28 Thread bharath kumar avusherla (JIRA)
bharath kumar avusherla created SPARK-23814: --- Summary: Couldn't read file with colon in name and new line character in one of the field. Key: SPARK-23814 URL:

[jira] [Commented] (SPARK-23811) Same tasks' FetchFailed event comes before Success will cause child stage never succeed

2018-03-28 Thread Li Yuanjian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418434#comment-16418434 ] Li Yuanjian commented on SPARK-23811: -   The scenario can be reproduced by below test case added in

[jira] [Assigned] (SPARK-23815) Spark writer dynamic partition overwrite mode fails to write output on multi level partition

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23815: Assignee: Apache Spark > Spark writer dynamic partition overwrite mode fails to write

[jira] [Assigned] (SPARK-23815) Spark writer dynamic partition overwrite mode fails to write output on multi level partition

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23815: Assignee: (was: Apache Spark) > Spark writer dynamic partition overwrite mode fails

[jira] [Commented] (SPARK-23772) Provide an option to ignore column of all null values or empty map/array during JSON schema inference

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418280#comment-16418280 ] Apache Spark commented on SPARK-23772: -- User 'maropu' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23772) Provide an option to ignore column of all null values or empty map/array during JSON schema inference

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23772: Assignee: Apache Spark > Provide an option to ignore column of all null values or empty

[jira] [Assigned] (SPARK-23772) Provide an option to ignore column of all null values or empty map/array during JSON schema inference

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23772: Assignee: (was: Apache Spark) > Provide an option to ignore column of all null values

[jira] [Resolved] (SPARK-23675) Title add spark logo, use spark logo image

2018-03-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-23675. --- Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20818

[jira] [Commented] (SPARK-23807) Add Hadoop 3 profile with relevant POM fix ups, cloud-storage artifacts and binding

2018-03-28 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418298#comment-16418298 ] Saisai Shao commented on SPARK-23807: - Hi [~ste...@apache.org], this is dup of SPARK-23534, I think

[jira] [Created] (SPARK-23812) DFS should be removed from unsupportedHiveNativeCommands in SqlBase.g4

2018-03-28 Thread wangtao93 (JIRA)
wangtao93 created SPARK-23812: - Summary: DFS should be removed from unsupportedHiveNativeCommands in SqlBase.g4 Key: SPARK-23812 URL: https://issues.apache.org/jira/browse/SPARK-23812 Project: Spark

[jira] [Updated] (SPARK-23811) Same tasks' FetchFailed event comes before Success will cause child stage never succeed

2018-03-28 Thread Li Yuanjian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-23811: Attachment: 1.png > Same tasks' FetchFailed event comes before Success will cause child stage >

[jira] [Updated] (SPARK-23813) [SparkSQL] the result is different between hive and spark when use PARSE_URL()

2018-03-28 Thread Cui Xixin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cui Xixin updated SPARK-23813: -- Description:  i am working on replacing hivesql with sparksql now, i found parse_url perform

[jira] [Updated] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

2018-03-28 Thread yuliang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuliang updated SPARK-5928: --- Attachment: image-2018-03-29-11-52-32-075.png > Remote Shuffle Blocks cannot be more than 2 GB >

[jira] [Commented] (SPARK-23810) Matrix Multiplication is so bad, file I/O to local python is better

2018-03-28 Thread dciborow (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418244#comment-16418244 ] dciborow commented on SPARK-23810: -- test comment to see if i get an email... > Matrix Multiplication is

[jira] [Created] (SPARK-23811) Same tasks' FetchFailed event comes before Success will cause child stage never succeed

2018-03-28 Thread Li Yuanjian (JIRA)
Li Yuanjian created SPARK-23811: --- Summary: Same tasks' FetchFailed event comes before Success will cause child stage never succeed Key: SPARK-23811 URL: https://issues.apache.org/jira/browse/SPARK-23811

[jira] [Updated] (SPARK-23811) Same tasks' FetchFailed event comes before Success will cause child stage never succeed

2018-03-28 Thread Li Yuanjian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23811?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Li Yuanjian updated SPARK-23811: Attachment: 2.png > Same tasks' FetchFailed event comes before Success will cause child stage >

[jira] [Created] (SPARK-23810) Matrix Multiplication is so bad, file I/O to local python is better

2018-03-28 Thread dciborow (JIRA)
dciborow created SPARK-23810: Summary: Matrix Multiplication is so bad, file I/O to local python is better Key: SPARK-23810 URL: https://issues.apache.org/jira/browse/SPARK-23810 Project: Spark

[jira] [Commented] (SPARK-20169) Groupby Bug with Sparksql

2018-03-28 Thread Maryann Xue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418282#comment-16418282 ] Maryann Xue commented on SPARK-20169: - This was related to the use of "withColumnRenamed", which will

[jira] [Updated] (SPARK-23810) Matrix Multiplication is so bad, file I/O to local python is better

2018-03-28 Thread dciborow (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dciborow updated SPARK-23810: - Priority: Minor (was: Major) > Matrix Multiplication is so bad, file I/O to local python is better >

[jira] [Issue Comment Deleted] (SPARK-23810) Matrix Multiplication is so bad, file I/O to local python is better

2018-03-28 Thread dciborow (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dciborow updated SPARK-23810: - Comment: was deleted (was: test comment to see if i get an email...) > Matrix Multiplication is so bad,

[jira] [Assigned] (SPARK-23675) Title add spark logo, use spark logo image

2018-03-28 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23675?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-23675: - Assignee: guoxiaolongzte > Title add spark logo, use spark logo image >

[jira] [Updated] (SPARK-23813) [SparkSQL] the result is different between hive and spark when use PARSE_URL()

2018-03-28 Thread Cui Xixin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cui Xixin updated SPARK-23813: -- Description:  i am working on replacing hivesql with sparksql now, i fount parse_url perform

[jira] [Created] (SPARK-23813) [SparkSQL] the result is different between hive and spark when use PARSE_URL()

2018-03-28 Thread Cui Xixin (JIRA)
Cui Xixin created SPARK-23813: - Summary: [SparkSQL] the result is different between hive and spark when use PARSE_URL() Key: SPARK-23813 URL: https://issues.apache.org/jira/browse/SPARK-23813 Project:

[jira] [Commented] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

2018-03-28 Thread yuliang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418410#comment-16418410 ] yuliang commented on SPARK-5928: !image-2018-03-29-11-52-32-075.png|width=741,height=189!   >From the

[jira] [Created] (SPARK-23808) Test spark sessions should set default session

2018-03-28 Thread Jose Torres (JIRA)
Jose Torres created SPARK-23808: --- Summary: Test spark sessions should set default session Key: SPARK-23808 URL: https://issues.apache.org/jira/browse/SPARK-23808 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-23809) Active SparkSession should be set by getOrCreate

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418211#comment-16418211 ] Apache Spark commented on SPARK-23809: -- User 'ericl' has created a pull request for this issue:

[jira] [Assigned] (SPARK-23809) Active SparkSession should be set by getOrCreate

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23809: Assignee: Apache Spark > Active SparkSession should be set by getOrCreate >

[jira] [Assigned] (SPARK-23809) Active SparkSession should be set by getOrCreate

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23809: Assignee: (was: Apache Spark) > Active SparkSession should be set by getOrCreate >

[jira] [Assigned] (SPARK-22941) Allow SparkSubmit to throw exceptions instead of exiting / printing errors.

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22941: Assignee: (was: Apache Spark) > Allow SparkSubmit to throw exceptions instead of

[jira] [Assigned] (SPARK-22941) Allow SparkSubmit to throw exceptions instead of exiting / printing errors.

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-22941: Assignee: Apache Spark > Allow SparkSubmit to throw exceptions instead of exiting /

[jira] [Commented] (SPARK-22941) Allow SparkSubmit to throw exceptions instead of exiting / printing errors.

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22941?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418087#comment-16418087 ] Apache Spark commented on SPARK-22941: -- User 'vanzin' has created a pull request for this issue:

[jira] [Commented] (SPARK-23808) Test spark sessions should set default session

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16418142#comment-16418142 ] Apache Spark commented on SPARK-23808: -- User 'jose-torres' has created a pull request for this

[jira] [Created] (SPARK-23809) Active SparkSession should be set by getOrCreate

2018-03-28 Thread Eric Liang (JIRA)
Eric Liang created SPARK-23809: -- Summary: Active SparkSession should be set by getOrCreate Key: SPARK-23809 URL: https://issues.apache.org/jira/browse/SPARK-23809 Project: Spark Issue Type: Bug

[jira] [Assigned] (SPARK-23808) Test spark sessions should set default session

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23808: Assignee: (was: Apache Spark) > Test spark sessions should set default session >

[jira] [Assigned] (SPARK-23808) Test spark sessions should set default session

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23808?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23808: Assignee: Apache Spark > Test spark sessions should set default session >

[jira] [Commented] (SPARK-23682) Memory issue with Spark structured streaming

2018-03-28 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417474#comment-16417474 ] Andrew Korzhuev commented on SPARK-23682: - I can confirm that _HDFSBackedStateStoreProvider_ is

[jira] [Comment Edited] (SPARK-23682) Memory issue with Spark structured streaming

2018-03-28 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417474#comment-16417474 ] Andrew Korzhuev edited comment on SPARK-23682 at 3/28/18 2:55 PM: -- I can

[jira] [Comment Edited] (SPARK-23682) Memory issue with Spark structured streaming

2018-03-28 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417474#comment-16417474 ] Andrew Korzhuev edited comment on SPARK-23682 at 3/28/18 2:56 PM: -- I can

[jira] [Commented] (SPARK-23040) BlockStoreShuffleReader's return Iterator isn't interruptible if aggregator or ordering is specified

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23040?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417429#comment-16417429 ] Apache Spark commented on SPARK-23040: -- User 'jiangxb1987' has created a pull request for this

[jira] [Updated] (SPARK-23682) Memory issue with Spark structured streaming

2018-03-28 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Korzhuev updated SPARK-23682: Attachment: Screen Shot 2018-03-28 at 16.44.20.png > Memory issue with Spark structured

[jira] [Commented] (SPARK-22618) RDD.unpersist can cause fatal exception when used with dynamic allocation

2018-03-28 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417466#comment-16417466 ] Thomas Graves commented on SPARK-22618: --- I'll file a separate Jira for it and put up a pr >

[jira] [Updated] (SPARK-23682) Memory issue with Spark structured streaming

2018-03-28 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Korzhuev updated SPARK-23682: Attachment: Screen Shot 2018-03-28 at 16.44.20.png > Memory issue with Spark structured

[jira] [Updated] (SPARK-23682) Memory issue with Spark structured streaming

2018-03-28 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Korzhuev updated SPARK-23682: Attachment: Screen Shot 2018-03-28 at 16.44.20.png > Memory issue with Spark structured

[jira] [Created] (SPARK-23807) Add Hadoop 3 profile with relevant POM fix ups, cloud-storage artifacts and binding

2018-03-28 Thread Steve Loughran (JIRA)
Steve Loughran created SPARK-23807: -- Summary: Add Hadoop 3 profile with relevant POM fix ups, cloud-storage artifacts and binding Key: SPARK-23807 URL: https://issues.apache.org/jira/browse/SPARK-23807

[jira] [Commented] (SPARK-23805) support vector-size validation and Inference

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23805?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417088#comment-16417088 ] Apache Spark commented on SPARK-23805: -- User 'zhengruifeng' has created a pull request for this

[jira] [Assigned] (SPARK-23805) support vector-size validation and Inference

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23805: Assignee: (was: Apache Spark) > support vector-size validation and Inference >

[jira] [Assigned] (SPARK-23805) support vector-size validation and Inference

2018-03-28 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23805: Assignee: Apache Spark > support vector-size validation and Inference >

[jira] [Comment Edited] (SPARK-23682) Memory issue with Spark structured streaming

2018-03-28 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417474#comment-16417474 ] Andrew Korzhuev edited comment on SPARK-23682 at 3/28/18 3:00 PM: -- I can

[jira] [Updated] (SPARK-23682) Memory issue with Spark structured streaming

2018-03-28 Thread Andrew Korzhuev (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23682?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Korzhuev updated SPARK-23682: Attachment: screen_shot_2018-03-20_at_15.23.29.png > Memory issue with Spark structured

[jira] [Commented] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-28 Thread Nathan Kleyn (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417519#comment-16417519 ] Nathan Kleyn commented on SPARK-23801: -- [~kiszk] We'll have our best go at trying to narrow down the

[jira] [Comment Edited] (SPARK-23801) Consistent SIGSEGV after upgrading to Spark v2.3.0

2018-03-28 Thread Nathan Kleyn (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16417519#comment-16417519 ] Nathan Kleyn edited comment on SPARK-23801 at 3/28/18 3:14 PM: --- [~kiszk] 

[jira] [Created] (SPARK-23806) Broadcast. unpersist can cause fatal exception when used with dynamic allocation

2018-03-28 Thread Thomas Graves (JIRA)
Thomas Graves created SPARK-23806: - Summary: Broadcast. unpersist can cause fatal exception when used with dynamic allocation Key: SPARK-23806 URL: https://issues.apache.org/jira/browse/SPARK-23806