[jira] [Assigned] (SPARK-20935) A daemon thread, "BatchedWriteAheadLog Writer", left behind after terminating StreamingContext.

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20935: Assignee: Apache Spark > A daemon thread, "BatchedWriteAheadLog Writer", left behind

[jira] [Commented] (SPARK-20935) A daemon thread, "BatchedWriteAheadLog Writer", left behind after terminating StreamingContext.

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040176#comment-16040176 ] Apache Spark commented on SPARK-20935: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-20935) A daemon thread, "BatchedWriteAheadLog Writer", left behind after terminating StreamingContext.

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20935: Assignee: (was: Apache Spark) > A daemon thread, "BatchedWriteAheadLog Writer", left

[jira] [Commented] (SPARK-19360) Spark 2.X does not support stored by cluase

2017-06-06 Thread Dayou Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040223#comment-16040223 ] Dayou Zhou commented on SPARK-19360: Same here... Since STORED BY is **essential** for using Hive

[jira] [Commented] (SPARK-21002) Syntax error regression when creating Hive storage handlers on Spark shell

2017-06-06 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040055#comment-16040055 ] Liang-Chi Hsieh commented on SPARK-21002: - Seems that we don't support {{STORED BY

[jira] [Commented] (SPARK-21002) Syntax error regression when creating Hive storage handlers on Spark shell

2017-06-06 Thread Dayou Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040226#comment-16040226 ] Dayou Zhou commented on SPARK-21002: Hi [~viirya], thanks for pointing to SPARK-19360 -- I missed it

[jira] [Updated] (SPARK-20760) Memory Leak of RDD blocks

2017-06-06 Thread Binzi Cao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20760?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Binzi Cao updated SPARK-20760: -- Attachment: Storage in spark 2.1.1.png RDD blocks in spark 2.1.1.png Spark 2.1.1 Test

[jira] [Resolved] (SPARK-21002) Syntax error regression when creating Hive storage handlers on Spark shell

2017-06-06 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21002. -- Resolution: Duplicate > Syntax error regression when creating Hive storage handlers on Spark

[jira] [Commented] (SPARK-11966) Spark API for UDTFs

2017-06-06 Thread Dayou Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040235#comment-16040235 ] Dayou Zhou commented on SPARK-11966: Hi [~hvanhovell], any examples on how this might be done using

[jira] [Resolved] (SPARK-20972) rename HintInfo.isBroadcastable to broadcast

2017-06-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-20972. - Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18189

[jira] [Updated] (SPARK-20972) rename HintInfo.isBroadcastable to broadcast

2017-06-06 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-20972: Summary: rename HintInfo.isBroadcastable to broadcast (was: rename HintInfo.isBroadcastable to

[jira] [Closed] (SPARK-19236) Add createOrReplaceGlobalTempView

2017-06-06 Thread Arman Yazdani (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19236?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arman Yazdani closed SPARK-19236. - > Add createOrReplaceGlobalTempView > - > > Key:

[jira] [Assigned] (SPARK-20994) Alleviate memory pressure in StreamManager

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20994: Assignee: (was: Apache Spark) > Alleviate memory pressure in StreamManager >

[jira] [Assigned] (SPARK-20994) Alleviate memory pressure in StreamManager

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20994?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20994: Assignee: Apache Spark > Alleviate memory pressure in StreamManager >

[jira] [Assigned] (SPARK-20993) The configuration item about 'Spark.blacklist.enabled' need to set the default value 'false'

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20993: Assignee: (was: Apache Spark) > The configuration item about

[jira] [Commented] (SPARK-20984) Reading back from ORC format gives error on big endian systems.

2017-06-06 Thread Mahesh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038267#comment-16038267 ] Mahesh commented on SPARK-20984: I checked the orc code and that is failing with near similar stack

[jira] [Created] (SPARK-20994) Alleviate memory pressure in StreamManager

2017-06-06 Thread jin xing (JIRA)
jin xing created SPARK-20994: Summary: Alleviate memory pressure in StreamManager Key: SPARK-20994 URL: https://issues.apache.org/jira/browse/SPARK-20994 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-20994) Alleviate memory pressure in StreamManager

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20994?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038280#comment-16038280 ] Apache Spark commented on SPARK-20994: -- User 'jinxing64' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20993) The configuration item about 'Spark.blacklist.enabled' need to set the default value 'false'

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20993: Assignee: Apache Spark > The configuration item about 'Spark.blacklist.enabled' need to

[jira] [Commented] (SPARK-20993) The configuration item about 'Spark.blacklist.enabled' need to set the default value 'false'

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20993?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038281#comment-16038281 ] Apache Spark commented on SPARK-20993: -- User 'guoxiaolongzte' has created a pull request for this

[jira] [Created] (SPARK-20995) 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration instructions.

2017-06-06 Thread guoxiaolongzte (JIRA)
guoxiaolongzte created SPARK-20995: -- Summary: 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration instructions. Key: SPARK-20995 URL: https://issues.apache.org/jira/browse/SPARK-20995

[jira] [Created] (SPARK-20993) The configuration item about 'Spark.blacklist.enabled' need to set the default value 'false'

2017-06-06 Thread guoxiaolongzte (JIRA)
guoxiaolongzte created SPARK-20993: -- Summary: The configuration item about 'Spark.blacklist.enabled' need to set the default value 'false' Key: SPARK-20993 URL: https://issues.apache.org/jira/browse/SPARK-20993

[jira] [Updated] (SPARK-20995) 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration instructions.

2017-06-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-20995: --- Description: Ensure that HADOOP_CONF_DIR or YARN_CONF_DIR points to the directory which

[jira] [Commented] (SPARK-20996) Better handling AM reattempt based on exit code in yarn mode

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038404#comment-16038404 ] Apache Spark commented on SPARK-20996: -- User 'jerryshao' has created a pull request for this issue:

[jira] [Commented] (SPARK-20199) GradientBoostedTreesModel doesn't have featureSubsetStrategy parameter

2017-06-06 Thread Kedarnath Reddy (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038403#comment-16038403 ] Kedarnath Reddy commented on SPARK-20199: - Please look into this feature , as I needed this for

[jira] [Assigned] (SPARK-20996) Better handling AM reattempt based on exit code in yarn mode

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20996: Assignee: (was: Apache Spark) > Better handling AM reattempt based on exit code in

[jira] [Assigned] (SPARK-20996) Better handling AM reattempt based on exit code in yarn mode

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20996: Assignee: Apache Spark > Better handling AM reattempt based on exit code in yarn mode >

[jira] [Commented] (SPARK-20995) 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration instructions.

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038361#comment-16038361 ] Apache Spark commented on SPARK-20995: -- User 'guoxiaolongzte' has created a pull request for this

[jira] [Assigned] (SPARK-20995) 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration instructions.

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20995: Assignee: (was: Apache Spark) > 'Spark-env.sh.template' should add 'YARN_CONF_DIR'

[jira] [Created] (SPARK-20996) Better handling AM reattempt based on exit code in yarn mode

2017-06-06 Thread Saisai Shao (JIRA)
Saisai Shao created SPARK-20996: --- Summary: Better handling AM reattempt based on exit code in yarn mode Key: SPARK-20996 URL: https://issues.apache.org/jira/browse/SPARK-20996 Project: Spark

[jira] [Assigned] (SPARK-20995) 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration instructions.

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20995: Assignee: Apache Spark > 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration

[jira] [Updated] (SPARK-20996) Better handling AM reattempt based on exit code in yarn mode

2017-06-06 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20996?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-20996: Description: Yarn provides max attempt configuration for applications running on it, application

[jira] [Resolved] (SPARK-20943) Correct BypassMergeSortShuffleWriter's comment

2017-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20943. --- Resolution: Not A Problem If there's no (other) support for a request to change the wording, and no

[jira] [Commented] (SPARK-20851) Drop spark table failed if a column name is a numeric string

2017-06-06 Thread Ben P (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20851?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038445#comment-16038445 ] Ben P commented on SPARK-20851: --- This issue seems to be handled/prevented in v2.0.1 as 2.0.1 throws below

[jira] [Resolved] (SPARK-20884) Spark' masters will be both standby due to the bug of curator

2017-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20884. --- Resolution: Won't Fix We can't merge this change, because Curator 2.8 won't work with Curator 2.6

[jira] [Updated] (SPARK-20995) 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration instructions.

2017-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20995: -- Priority: Trivial (was: Minor) > 'Spark-env.sh.template' should add 'YARN_CONF_DIR' configuration

[jira] [Resolved] (SPARK-20499) Spark MLlib, GraphX 2.2 QA umbrella

2017-06-06 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-20499. Resolution: Done > Spark MLlib, GraphX 2.2 QA umbrella >

[jira] [Commented] (SPARK-20162) Reading data from MySQL - Cannot up cast from decimal(30,6) to decimal(38,18)

2017-06-06 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038515#comment-16038515 ] Yuming Wang commented on SPARK-20162: - [~mspehar] How to reproduce it? read a table like

[jira] [Assigned] (SPARK-20914) Javadoc contains code that is invalid

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20914: Assignee: Apache Spark > Javadoc contains code that is invalid >

[jira] [Assigned] (SPARK-20914) Javadoc contains code that is invalid

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20914: Assignee: (was: Apache Spark) > Javadoc contains code that is invalid >

[jira] [Commented] (SPARK-20914) Javadoc contains code that is invalid

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038518#comment-16038518 ] Apache Spark commented on SPARK-20914: -- User 'srowen' has created a pull request for this issue:

[jira] [Resolved] (SPARK-20985) Improve KryoSerializerResizableOutputSuite

2017-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20985. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18204

[jira] [Assigned] (SPARK-20985) Improve KryoSerializerResizableOutputSuite

2017-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-20985: - Assignee: jin xing > Improve KryoSerializerResizableOutputSuite >

[jira] [Resolved] (SPARK-20507) Update MLlib, GraphX websites for 2.2

2017-06-06 Thread Nick Pentreath (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20507?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nick Pentreath resolved SPARK-20507. Resolution: Done Assignee: Nick Pentreath No updates to MLlib project website

[jira] [Created] (SPARK-20997) spark-submit's --driver-cores marked as "YARN-only" but listed under "Spark standalone with cluster deploy mode only"

2017-06-06 Thread Jacek Laskowski (JIRA)
Jacek Laskowski created SPARK-20997: --- Summary: spark-submit's --driver-cores marked as "YARN-only" but listed under "Spark standalone with cluster deploy mode only" Key: SPARK-20997 URL:

[jira] [Updated] (SPARK-20997) spark-submit's --driver-cores marked as "YARN-only" but listed under "Spark standalone with cluster deploy mode only"

2017-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20997: -- Priority: Trivial (was: Minor) Component/s: Documentation I suppose this should be removed

[jira] [Commented] (SPARK-19878) Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala

2017-06-06 Thread lyc (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038648#comment-16038648 ] lyc commented on SPARK-19878: - See

[jira] [Comment Edited] (SPARK-20969) last() aggregate function fails returning the right answer with ordered windows

2017-06-06 Thread Perrine Letellier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038825#comment-16038825 ] Perrine Letellier edited comment on SPARK-20969 at 6/6/17 12:59 PM:

[jira] [Comment Edited] (SPARK-20969) last() aggregate function fails returning the right answer with ordered windows

2017-06-06 Thread Perrine Letellier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038825#comment-16038825 ] Perrine Letellier edited comment on SPARK-20969 at 6/6/17 1:00 PM: ---

[jira] [Comment Edited] (SPARK-20969) last() aggregate function fails returning the right answer with ordered windows

2017-06-06 Thread Perrine Letellier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038825#comment-16038825 ] Perrine Letellier edited comment on SPARK-20969 at 6/6/17 1:00 PM: ---

[jira] [Resolved] (SPARK-20969) last() aggregate function fails returning the right answer with ordered windows

2017-06-06 Thread Perrine Letellier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Perrine Letellier resolved SPARK-20969. --- Resolution: Not A Problem > last() aggregate function fails returning the right

[jira] [Updated] (SPARK-20712) [SQL] Spark can't read Hive table when column type has length greater than 4000 bytes

2017-06-06 Thread JIRA
[ https://issues.apache.org/jira/browse/SPARK-20712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maciej BryƄski updated SPARK-20712: --- Affects Version/s: 2.1.2 > [SQL] Spark can't read Hive table when column type has length

[jira] [Created] (SPARK-21001) Staging folders from Hive table are not being cleared.

2017-06-06 Thread Ajay Cherukuri (JIRA)
Ajay Cherukuri created SPARK-21001: -- Summary: Staging folders from Hive table are not being cleared. Key: SPARK-21001 URL: https://issues.apache.org/jira/browse/SPARK-21001 Project: Spark

[jira] [Updated] (SPARK-20973) insert table fail caused by unable to fetch data definition file from remote hdfs

2017-06-06 Thread Yunjian Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yunjian Zhang updated SPARK-20973: -- Attachment: spark-sql-insert.patch > insert table fail caused by unable to fetch data

[jira] [Issue Comment Deleted] (SPARK-20973) insert table fail caused by unable to fetch data definition file from remote hdfs

2017-06-06 Thread Yunjian Zhang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yunjian Zhang updated SPARK-20973: -- Comment: was deleted (was: I did check the source code and add a patch to fix the insert issue

[jira] [Commented] (SPARK-21002) Syntax error regression when creating Hive storage handlers on Spark shell

2017-06-06 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040057#comment-16040057 ] Liang-Chi Hsieh commented on SPARK-21002: - This is duplicate to SPARK-19360. > Syntax error

[jira] [Commented] (SPARK-19878) Add hive configuration when initialize hive serde in InsertIntoHiveTable.scala

2017-06-06 Thread Dayou Zhou (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19878?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039762#comment-16039762 ] Dayou Zhou commented on SPARK-19878: Hello, just checking will this be shipped to Spark 2.2? Thank

[jira] [Created] (SPARK-21002) Syntax error regression when creating Hive storage handlers on Spark shell

2017-06-06 Thread Dayou Zhou (JIRA)
Dayou Zhou created SPARK-21002: -- Summary: Syntax error regression when creating Hive storage handlers on Spark shell Key: SPARK-21002 URL: https://issues.apache.org/jira/browse/SPARK-21002 Project:

[jira] [Commented] (SPARK-20969) last() aggregate function fails returning the right answer with ordered windows

2017-06-06 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039862#comment-16039862 ] Liang-Chi Hsieh commented on SPARK-20969: - [~pletelli] I don't find an api doc for that. Maybe we

[jira] [Commented] (SPARK-20977) NPE in CollectionAccumulator

2017-06-06 Thread Boris Capitanu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039870#comment-16039870 ] Boris Capitanu commented on SPARK-20977: I am seeing this problem as well. I have a scala SBT

[jira] [Comment Edited] (SPARK-20977) NPE in CollectionAccumulator

2017-06-06 Thread Boris Capitanu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20977?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039870#comment-16039870 ] Boris Capitanu edited comment on SPARK-20977 at 6/6/17 11:35 PM: - I am

[jira] [Comment Edited] (SPARK-20760) Memory Leak of RDD blocks

2017-06-06 Thread Binzi Cao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040112#comment-16040112 ] Binzi Cao edited comment on SPARK-20760 at 6/7/17 3:47 AM: --- Hi David, Thanks

[jira] [Commented] (SPARK-20997) spark-submit's --driver-cores marked as "YARN-only" but listed under "Spark standalone with cluster deploy mode only"

2017-06-06 Thread guoxiaolongzte (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20997?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040210#comment-16040210 ] guoxiaolongzte commented on SPARK-20997: Can i fix this jira? Because i found a similar problem

[jira] [Commented] (SPARK-20760) Memory Leak of RDD blocks

2017-06-06 Thread Binzi Cao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16040112#comment-16040112 ] Binzi Cao commented on SPARK-20760: --- Hi David, Thanks very much for the message, I did a test with

[jira] [Assigned] (SPARK-20999) No failure Stages, no log 'DAGScheduler: failed: Set()' output.

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20999: Assignee: Apache Spark > No failure Stages, no log 'DAGScheduler: failed: Set()' output.

[jira] [Created] (SPARK-20999) No failure Stages, no log 'DAGScheduler: failed: Set()' output.

2017-06-06 Thread caoxuewen (JIRA)
caoxuewen created SPARK-20999: - Summary: No failure Stages, no log 'DAGScheduler: failed: Set()' output. Key: SPARK-20999 URL: https://issues.apache.org/jira/browse/SPARK-20999 Project: Spark

[jira] [Assigned] (SPARK-20999) No failure Stages, no log 'DAGScheduler: failed: Set()' output.

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20999: Assignee: (was: Apache Spark) > No failure Stages, no log 'DAGScheduler: failed:

[jira] [Commented] (SPARK-20999) No failure Stages, no log 'DAGScheduler: failed: Set()' output.

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038789#comment-16038789 ] Apache Spark commented on SPARK-20999: -- User 'heary-cao' has created a pull request for this issue:

[jira] [Commented] (SPARK-20999) No failure Stages, no log 'DAGScheduler: failed: Set()' output.

2017-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20999?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038793#comment-16038793 ] Sean Owen commented on SPARK-20999: --- As I've already commented, there doesn't seem to be value in

[jira] [Resolved] (SPARK-20999) No failure Stages, no log 'DAGScheduler: failed: Set()' output.

2017-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20999. --- Resolution: Not A Problem > No failure Stages, no log 'DAGScheduler: failed: Set()' output. >

[jira] [Closed] (SPARK-20999) No failure Stages, no log 'DAGScheduler: failed: Set()' output.

2017-06-06 Thread caoxuewen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20999?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] caoxuewen closed SPARK-20999. - Not A Problem > No failure Stages, no log 'DAGScheduler: failed: Set()' output. >

[jira] [Commented] (SPARK-20969) last() aggregate function fails returning the right answer with ordered windows

2017-06-06 Thread Perrine Letellier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038825#comment-16038825 ] Perrine Letellier commented on SPARK-20969: --- [~viirya] Thanks for your answer ! I could get the

[jira] [Comment Edited] (SPARK-20969) last() aggregate function fails returning the right answer with ordered windows

2017-06-06 Thread Perrine Letellier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20969?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038825#comment-16038825 ] Perrine Letellier edited comment on SPARK-20969 at 6/6/17 12:59 PM:

[jira] [Commented] (SPARK-20854) extend hint syntax to support any expression, not just identifiers or strings

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038671#comment-16038671 ] Apache Spark commented on SPARK-20854: -- User 'bogdanrdc' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-18791) Stream-Stream Joins

2017-06-06 Thread xianyao jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038690#comment-16038690 ] xianyao jiang edited comment on SPARK-18791 at 6/6/17 12:09 PM: We have

[jira] [Created] (SPARK-20998) BroadcastHashJoin producing different results

2017-06-06 Thread Mohit (JIRA)
Mohit created SPARK-20998: - Summary: BroadcastHashJoin producing different results Key: SPARK-20998 URL: https://issues.apache.org/jira/browse/SPARK-20998 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-20998) BroadcastHashJoin producing wrong results

2017-06-06 Thread Mohit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit updated SPARK-20998: -- Affects Version/s: (was: 2.0.1) 2.0.0 > BroadcastHashJoin producing wrong results >

[jira] [Updated] (SPARK-20998) BroadcastHashJoin producing wrong results

2017-06-06 Thread Mohit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit updated SPARK-20998: -- Summary: BroadcastHashJoin producing wrong results (was: BroadcastHashJoin producing different results) >

[jira] [Commented] (SPARK-18791) Stream-Stream Joins

2017-06-06 Thread xianyao jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038690#comment-16038690 ] xianyao jiang commented on SPARK-18791: --- We have the draft design for the stream-stream inner

[jira] [Updated] (SPARK-20969) last() aggregate function fails returning the right answer with ordered windows

2017-06-06 Thread Perrine Letellier (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Perrine Letellier updated SPARK-20969: -- Description: The column on which `orderBy` is performed is considered as another

[jira] [Updated] (SPARK-20998) BroadcastHashJoin producing wrong results

2017-06-06 Thread Mohit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit updated SPARK-20998: -- Description: I have a hive table : _eagle_edw_batch.DistributionAttributes_, with schema: root |--

[jira] [Comment Edited] (SPARK-18791) Stream-Stream Joins

2017-06-06 Thread xianyao jiang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038690#comment-16038690 ] xianyao jiang edited comment on SPARK-18791 at 6/6/17 12:15 PM: We have

[jira] [Updated] (SPARK-20998) BroadcastHashJoin producing wrong results

2017-06-06 Thread Mohit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit updated SPARK-20998: -- Description: I have a hive table : _eagle_edw_batch.DistributionAttributes_, with *Schema*: root |--

[jira] [Updated] (SPARK-20998) BroadcastHashJoin producing wrong results

2017-06-06 Thread Mohit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20998?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mohit updated SPARK-20998: -- Description: I have a hive table : _eagle_edw_batch.DistributionAttributes_, with *Schema*: root |--

[jira] [Updated] (SPARK-13669) Job will always fail in the external shuffle service unavailable situation

2017-06-06 Thread Saisai Shao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13669?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao updated SPARK-13669: Description: Currently we are running into an issue with Yarn work preserving enabled + external

[jira] [Commented] (SPARK-20935) A daemon thread, "BatchedWriteAheadLog Writer", left behind after terminating StreamingContext.

2017-06-06 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038549#comment-16038549 ] Sean Owen commented on SPARK-20935: --- Sounds OK to me [~hyukjin.kwon] if you're up for it > A daemon

[jira] [Assigned] (SPARK-20920) ForkJoinPool pools are leaked when writing hive tables with many partitions

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20920: Assignee: (was: Apache Spark) > ForkJoinPool pools are leaked when writing hive

[jira] [Commented] (SPARK-20920) ForkJoinPool pools are leaked when writing hive tables with many partitions

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16038583#comment-16038583 ] Apache Spark commented on SPARK-20920: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20920) ForkJoinPool pools are leaked when writing hive tables with many partitions

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20920: Assignee: Apache Spark > ForkJoinPool pools are leaked when writing hive tables with many

[jira] [Created] (SPARK-21000) Add labels support to the Spark Dispatcher

2017-06-06 Thread Michael Gummelt (JIRA)
Michael Gummelt created SPARK-21000: --- Summary: Add labels support to the Spark Dispatcher Key: SPARK-21000 URL: https://issues.apache.org/jira/browse/SPARK-21000 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-21000) Add labels support to the Spark Dispatcher

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21000: Assignee: Apache Spark > Add labels support to the Spark Dispatcher >

[jira] [Commented] (SPARK-21000) Add labels support to the Spark Dispatcher

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039308#comment-16039308 ] Apache Spark commented on SPARK-21000: -- User 'mgummelt' has created a pull request for this issue:

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

2017-06-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039223#comment-16039223 ] Marcelo Vanzin commented on SPARK-18085: Yes, https://github.com/vanzin/spark/commits/shs-ng/HEAD

[jira] [Updated] (SPARK-20926) Exposure to Guava libraries by directly accessing tableRelationCache in SessionCatalog caused failures

2017-06-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-20926: --- Fix Version/s: 2.2.1 > Exposure to Guava libraries by directly accessing tableRelationCache

[jira] [Commented] (SPARK-20926) Exposure to Guava libraries by directly accessing tableRelationCache in SessionCatalog caused failures

2017-06-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039249#comment-16039249 ] Marcelo Vanzin commented on SPARK-20926: Turns out I didn't notice the PR was against 2.2 so it's

[jira] [Commented] (SPARK-18372) .Hive-staging folders created from Spark hiveContext are not getting cleaned up

2017-06-06 Thread Ajay Cherukuri (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18372?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039330#comment-16039330 ] Ajay Cherukuri commented on SPARK-18372: I have this issue in Spark 2.0.2 > .Hive-staging

[jira] [Commented] (SPARK-20760) Memory Leak of RDD blocks

2017-06-06 Thread David Lewis (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20760?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16039375#comment-16039375 ] David Lewis commented on SPARK-20760: - I believe this is fixed by this issue:

[jira] [Issue Comment Deleted] (SPARK-18085) Better History Server scalability for many / large applications

2017-06-06 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin updated SPARK-18085: --- Comment: was deleted (was: Your contact with NYSE has changed, please visit

[jira] [Assigned] (SPARK-21000) Add labels support to the Spark Dispatcher

2017-06-06 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21000?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21000: Assignee: (was: Apache Spark) > Add labels support to the Spark Dispatcher >

[jira] [Resolved] (SPARK-20641) Key-value store abstraction and implementation for storing application data

2017-06-06 Thread Imran Rashid (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20641?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid resolved SPARK-20641. -- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 17902

  1   2   >