[jira] [Created] (SPARK-21053) Number overflow on agg function of Dataframe

2017-06-10 Thread DUC LIEM NGUYEN (JIRA)
DUC LIEM NGUYEN created SPARK-21053: --- Summary: Number overflow on agg function of Dataframe Key: SPARK-21053 URL: https://issues.apache.org/jira/browse/SPARK-21053 Project: Spark Issue

[jira] [Commented] (SPARK-20427) Issue with Spark interpreting Oracle datatype NUMBER

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045783#comment-16045783 ] Apache Spark commented on SPARK-20427: -- User 'wangyum' has created a pull request for this issue:

[jira] [Assigned] (SPARK-20427) Issue with Spark interpreting Oracle datatype NUMBER

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20427: Assignee: Apache Spark > Issue with Spark interpreting Oracle datatype NUMBER >

[jira] [Assigned] (SPARK-20427) Issue with Spark interpreting Oracle datatype NUMBER

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20427: Assignee: (was: Apache Spark) > Issue with Spark interpreting Oracle datatype NUMBER

[jira] [Commented] (SPARK-21043) Add unionByName API to Dataset

2017-06-10 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045773#comment-16045773 ] Takeshi Yamamuro commented on SPARK-21043: -- Thank you for ping me! Yea, I'll try > Add

[jira] [Commented] (SPARK-21052) Add hash map metrics to join

2017-06-10 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045769#comment-16045769 ] Liang-Chi Hsieh commented on SPARK-21052: - I'll submit a PR for this soon. > Add hash map

[jira] [Assigned] (SPARK-21051) Add hash map metrics to aggregate

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21051: Assignee: Apache Spark > Add hash map metrics to aggregate >

[jira] [Updated] (SPARK-21052) Add hash map metrics to join

2017-06-10 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21052?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-21052: Description: We should add avg hash map probe metric to join operator and report it on UI.

[jira] [Assigned] (SPARK-21051) Add hash map metrics to aggregate

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21051: Assignee: (was: Apache Spark) > Add hash map metrics to aggregate >

[jira] [Commented] (SPARK-21051) Add hash map metrics to aggregate

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21051?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045768#comment-16045768 ] Apache Spark commented on SPARK-21051: -- User 'viirya' has created a pull request for this issue:

[jira] [Updated] (SPARK-21051) Add hash map metrics to aggregate

2017-06-10 Thread Liang-Chi Hsieh (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21051?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Liang-Chi Hsieh updated SPARK-21051: Description: We should add avg hash map probe metric to aggregate operator and report it

[jira] [Created] (SPARK-21052) Add hash map metrics to join

2017-06-10 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-21052: --- Summary: Add hash map metrics to join Key: SPARK-21052 URL: https://issues.apache.org/jira/browse/SPARK-21052 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-21051) Add hash map metrics to aggregate

2017-06-10 Thread Liang-Chi Hsieh (JIRA)
Liang-Chi Hsieh created SPARK-21051: --- Summary: Add hash map metrics to aggregate Key: SPARK-21051 URL: https://issues.apache.org/jira/browse/SPARK-21051 Project: Spark Issue Type: Sub-task

[jira] [Assigned] (SPARK-21050) ml word2vec write has overflow issue in calculating numPartitions

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21050: Assignee: Apache Spark (was: Joseph K. Bradley) > ml word2vec write has overflow issue

[jira] [Assigned] (SPARK-21050) ml word2vec write has overflow issue in calculating numPartitions

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21050: Assignee: Joseph K. Bradley (was: Apache Spark) > ml word2vec write has overflow issue

[jira] [Commented] (SPARK-21050) ml word2vec write has overflow issue in calculating numPartitions

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045763#comment-16045763 ] Apache Spark commented on SPARK-21050: -- User 'jkbradley' has created a pull request for this issue:

[jira] [Created] (SPARK-21050) ml word2vec write has overflow issue in calculating numPartitions

2017-06-10 Thread Joseph K. Bradley (JIRA)
Joseph K. Bradley created SPARK-21050: - Summary: ml word2vec write has overflow issue in calculating numPartitions Key: SPARK-21050 URL: https://issues.apache.org/jira/browse/SPARK-21050 Project:

[jira] [Commented] (SPARK-20877) Shorten test sets to run on CRAN

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045732#comment-16045732 ] Apache Spark commented on SPARK-20877: -- User 'felixcheung' has created a pull request for this

[jira] [Updated] (SPARK-20877) Shorten test sets to run on CRAN

2017-06-10 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-20877: - Summary: Shorten test sets to run on CRAN (was: Investigate if tests will time out on CRAN) >

[jira] [Closed] (SPARK-21044) Add `RemoveInvalidRange` optimizer

2017-06-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21044?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-21044. - Resolution: Invalid > Add `RemoveInvalidRange` optimizer > -- >

[jira] [Updated] (SPARK-17642) Support DESC FORMATTED TABLE COLUMN command to show column-level statistics

2017-06-10 Thread Zhenhua Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-17642: - Summary: Support DESC FORMATTED TABLE COLUMN command to show column-level statistics (was:

[jira] [Assigned] (SPARK-21039) Use treeAggregate instead of aggregate in DataFrame.stat.bloomFilter

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21039: Assignee: (was: Apache Spark) > Use treeAggregate instead of aggregate in

[jira] [Assigned] (SPARK-21039) Use treeAggregate instead of aggregate in DataFrame.stat.bloomFilter

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21039?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21039: Assignee: Apache Spark > Use treeAggregate instead of aggregate in

[jira] [Commented] (SPARK-21039) Use treeAggregate instead of aggregate in DataFrame.stat.bloomFilter

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045650#comment-16045650 ] Apache Spark commented on SPARK-21039: -- User 'rishabhbhardwaj' has created a pull request for this

[jira] [Commented] (SPARK-21043) Add unionByName API to Dataset

2017-06-10 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045630#comment-16045630 ] Xiao Li commented on SPARK-21043: - [~maropu] Do you want to make a try? > Add unionByName API to Dataset

[jira] [Commented] (SPARK-21045) Spark executor blocked instead of throwing exception because exception occur when python worker send exception info to Java Gateway

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045604#comment-16045604 ] Apache Spark commented on SPARK-21045: -- User 'dataknocker' has created a pull request for this

[jira] [Commented] (SPARK-21045) Spark executor blocked instead of throwing exception because exception occur when python worker send exception info to Java Gateway

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045592#comment-16045592 ] Apache Spark commented on SPARK-21045: -- User 'dataknocker' has created a pull request for this

[jira] [Assigned] (SPARK-21045) Spark executor blocked instead of throwing exception because exception occur when python worker send exception info to Java Gateway

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21045: Assignee: Apache Spark > Spark executor blocked instead of throwing exception because

[jira] [Assigned] (SPARK-21045) Spark executor blocked instead of throwing exception because exception occur when python worker send exception info to Java Gateway

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21045: Assignee: (was: Apache Spark) > Spark executor blocked instead of throwing exception

[jira] [Updated] (SPARK-21045) Spark executor blocked instead of throwing exception because exception occur when python worker send exception info to Java Gateway

2017-06-10 Thread Joshuawangzj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-21045: - Summary: Spark executor blocked instead of throwing exception because exception occur when

[jira] [Closed] (SPARK-20684) expose createGlobalTempView and dropGlobalTempView in SparkR

2017-06-10 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20684?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-20684. - Resolution: Later > expose createGlobalTempView and dropGlobalTempView in SparkR >

[jira] [Commented] (SPARK-21048) Add an option --merged-properties-file to distinguish the configuration loading behavior

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045539#comment-16045539 ] Lantao Jin commented on SPARK-21048: Thank you for pointing out about JIRA. Confusing name can be

[jira] [Commented] (SPARK-21049) why do we need computeGramianMatrix when computing SVD

2017-06-10 Thread Vincent (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21049?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045536#comment-16045536 ] Vincent commented on SPARK-21049: - [~srowen]thanks. that's right. But we found it quite often that, the

[jira] [Commented] (SPARK-21048) Add an option --merged-properties-file to distinguish the configuration loading behavior

2017-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045533#comment-16045533 ] Sean Owen commented on SPARK-21048: --- I think this is confusing relative to any value it adds, and don't

[jira] [Resolved] (SPARK-21049) why do we need computeGramianMatrix when computing SVD

2017-06-10 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21049?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-21049. --- Resolution: Invalid Questions should go to the mailing list. Consider what "just" computing the SVD

[jira] [Commented] (SPARK-21001) Staging folders from Hive table are not being cleared.

2017-06-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045531#comment-16045531 ] Hyukjin Kwon commented on SPARK-21001: -- Does this still exist in 2.1.0? > Staging folders from Hive

[jira] [Created] (SPARK-21049) why do we need computeGramianMatrix when computing SVD

2017-06-10 Thread Vincent (JIRA)
Vincent created SPARK-21049: --- Summary: why do we need computeGramianMatrix when computing SVD Key: SPARK-21049 URL: https://issues.apache.org/jira/browse/SPARK-21049 Project: Spark Issue Type:

[jira] [Commented] (SPARK-21048) Add an option --merged-properties-file to distinguish the configuration loading behavior

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045502#comment-16045502 ] Lantao Jin commented on SPARK-21048: Will push a PR soon after a short discussing. > Add an option

[jira] [Closed] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin closed SPARK-21023. -- Resolution: Not A Problem Close as NotAProblem and new solution see

[jira] [Created] (SPARK-21048) Add an option --merged-properties-file to distinguish the configuration loading behavior

2017-06-10 Thread Lantao Jin (JIRA)
Lantao Jin created SPARK-21048: -- Summary: Add an option --merged-properties-file to distinguish the configuration loading behavior Key: SPARK-21048 URL: https://issues.apache.org/jira/browse/SPARK-21048

[jira] [Resolved] (SPARK-21015) Check field name is not null and empty in GenericRowWithSchema

2017-06-10 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21015?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-21015. -- Resolution: Invalid I am resolving this per

[jira] [Commented] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045495#comment-16045495 ] Lantao Jin commented on SPARK-21023: [~cloud_fan], I come to see what you mean. Maybe add a

[jira] [Updated] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-21023: --- Description: The default properties file {{spark-defaults.conf}} shouldn't be ignore to load even

[jira] [Comment Edited] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045483#comment-16045483 ] Lantao Jin edited comment on SPARK-21023 at 6/10/17 10:15 AM: -- [~cloud_fan]

[jira] [Commented] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045483#comment-16045483 ] Lantao Jin commented on SPARK-21023: [~cloud_fan] {{--properties-file}} and

[jira] [Updated] (SPARK-21047) Add test cases for nested array

2017-06-10 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21047?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kazuaki Ishizaki updated SPARK-21047: - Summary: Add test cases for nested array (was: Add a test case for nested array) > Add

[jira] [Created] (SPARK-21047) Add a test case for nested array

2017-06-10 Thread Kazuaki Ishizaki (JIRA)
Kazuaki Ishizaki created SPARK-21047: Summary: Add a test case for nested array Key: SPARK-21047 URL: https://issues.apache.org/jira/browse/SPARK-21047 Project: Spark Issue Type:

[jira] [Updated] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-21023: --- Description: The default properties file {{spark-defaults.conf}} shouldn't be ignore to load even

[jira] [Commented] (SPARK-21037) ignoreNulls does not working properly with window functions

2017-06-10 Thread Stanislav Chernichkin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045479#comment-16045479 ] Stanislav Chernichkin commented on SPARK-21037: --- To be more precise the problem not related

[jira] [Updated] (SPARK-21046) simplify the array offset and length in ColumnVector

2017-06-10 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21046: Issue Type: Sub-task (was: Improvement) Parent: SPARK-20960 > simplify the array offset

[jira] [Assigned] (SPARK-21046) simplify the array offset and length in ColumnVector

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21046: Assignee: Apache Spark (was: Wenchen Fan) > simplify the array offset and length in

[jira] [Assigned] (SPARK-21046) simplify the array offset and length in ColumnVector

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21046?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-21046: Assignee: Wenchen Fan (was: Apache Spark) > simplify the array offset and length in

[jira] [Commented] (SPARK-21046) simplify the array offset and length in ColumnVector

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21046?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045477#comment-16045477 ] Apache Spark commented on SPARK-21046: -- User 'cloud-fan' has created a pull request for this issue:

[jira] [Updated] (SPARK-21037) ignoreNulls does not working properly with window functions

2017-06-10 Thread Stanislav Chernichkin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Chernichkin updated SPARK-21037: -- Description: Following code reproduces issue: spark .sql("select 0 as

[jira] [Created] (SPARK-21046) simplify the array offset and length in ColumnVector

2017-06-10 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-21046: --- Summary: simplify the array offset and length in ColumnVector Key: SPARK-21046 URL: https://issues.apache.org/jira/browse/SPARK-21046 Project: Spark Issue

[jira] [Updated] (SPARK-21037) ignoreNulls does not working properly with window functions

2017-06-10 Thread Stanislav Chernichkin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21037?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stanislav Chernichkin updated SPARK-21037: -- Description: Following code reproduces issue: spark .sql("select 0 as

[jira] [Commented] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045472#comment-16045472 ] Wenchen Fan commented on SPARK-21023: - can't we just introduce something like

[jira] [Commented] (SPARK-21006) Create rpcEnv and run later needs shutdown and awaitTermination

2017-06-10 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045468#comment-16045468 ] Apache Spark commented on SPARK-21006: -- User '10110346' has created a pull request for this issue:

[jira] [Commented] (SPARK-20752) Build-in SQL Function Support - SQRT

2017-06-10 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045466#comment-16045466 ] Kazuaki Ishizaki commented on SPARK-20752: -- ping [~smilegator] > Build-in SQL Function Support

[jira] [Updated] (SPARK-21045) Spark executor is blocked instead of throwing exception because exception occur when python worker send exception trace stack info to Java Gateway

2017-06-10 Thread Joshuawangzj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-21045: - Description: My pyspark program is always blocking in product yarn cluster. Then I jstack and

[jira] [Updated] (SPARK-21045) Spark executor is blocked instead of throwing exception because exception occur when python worker send exception trace stack info to Java Gateway

2017-06-10 Thread Joshuawangzj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-21045: - Description: My pyspark program is always blocking in product yarn cluster. Then I jstack and

[jira] [Updated] (SPARK-21045) Spark executor is blocked instead of throwing exception because exception occur when python worker send exception trace stack info to Java Gateway

2017-06-10 Thread Joshuawangzj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-21045: - Description: My pyspark program is always blocking in product yarn cluster. Then I jstack and

[jira] [Updated] (SPARK-21045) Spark executor is blocked instead of throwing exception because exception occur when python worker send exception trace stack info to Java Gateway

2017-06-10 Thread Joshuawangzj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-21045: - Description: My pyspark program is always blocking in product yarn cluster. Then I jstack and

[jira] [Updated] (SPARK-21045) Spark executor is blocked instead of throwing exception because exception occur when python worker send exception trace stack info to Java Gateway

2017-06-10 Thread Joshuawangzj (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshuawangzj updated SPARK-21045: - Description: My pyspark program is always blocking in product yarn cluster. Then I jstack and

[jira] [Created] (SPARK-21045) Spark executor is blocked instead of throwing exception because exception occur when python worker send exception trace stack info to Java Gateway

2017-06-10 Thread Joshuawangzj (JIRA)
Joshuawangzj created SPARK-21045: Summary: Spark executor is blocked instead of throwing exception because exception occur when python worker send exception trace stack info to Java Gateway Key: SPARK-21045 URL:

[jira] [Comment Edited] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045452#comment-16045452 ] Lantao Jin edited comment on SPARK-21023 at 6/10/17 8:18 AM: - If current

[jira] [Comment Edited] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045452#comment-16045452 ] Lantao Jin edited comment on SPARK-21023 at 6/10/17 8:16 AM: - If current

[jira] [Commented] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045452#comment-16045452 ] Lantao Jin commented on SPARK-21023: If current behavior should be keep. I can add an environment

[jira] [Commented] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16045441#comment-16045441 ] Lantao Jin commented on SPARK-21023: I modify the description with adding two tables to illustrate

[jira] [Updated] (SPARK-21023) Ignore to load default properties file is not a good choice from the perspective of system

2017-06-10 Thread Lantao Jin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21023?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-21023: --- Description: The default properties file {{spark-defaults.conf}} shouldn't be ignore to load even