[jira] [Commented] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680970#comment-16680970 ] Hyukjin Kwon commented on SPARK-25976: -- Yea true in that way. I'm not underestimating the proposed

[jira] [Commented] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680954#comment-16680954 ] Yuval Yaari commented on SPARK-25976: - Thanks! Isnt take quite expensive? I had in mind that for

[jira] [Commented] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680946#comment-16680946 ] Hyukjin Kwon commented on SPARK-25976: -- BTW, {{isEmpty}} is arguably cheap: {code}

[jira] [Commented] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680944#comment-16680944 ] Hyukjin Kwon commented on SPARK-25976: -- Okay, then, it's more minor then I thought. I would just

[jira] [Commented] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680938#comment-16680938 ] Yuval Yaari commented on SPARK-25976: - correct, however in scala there is not much performance

[jira] [Commented] (SPARK-25961) Random numbers are not supported when handling data skew

[ https://issues.apache.org/jira/browse/SPARK-25961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680925#comment-16680925 ] Hyukjin Kwon commented on SPARK-25961: -- Questions should go to mailing list. That seems to be

[jira] [Commented] (SPARK-25958) error: [Errno 97] Address family not supported by protocol in dataframe.take()

[ https://issues.apache.org/jira/browse/SPARK-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680920#comment-16680920 ] Yuanjian Li commented on SPARK-25958: - [~Tagar] Yep, you should also comment out IPv6 hosts in

[jira] [Commented] (SPARK-25961) Random numbers are not supported when handling data skew

[ https://issues.apache.org/jira/browse/SPARK-25961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680908#comment-16680908 ] zengxl commented on SPARK-25961: I'm not asking for an investigation. My question is why sparksql does

[jira] [Updated] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Tsukanov updated SPARK-25987: -- Description: When I execute {code:java} val columnsCount = 100 val columns = (1 to

[jira] [Assigned] (SPARK-25988) Keep names unchanged when deduplicating the column names in Analyzer

[ https://issues.apache.org/jira/browse/SPARK-25988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25988: Assignee: Apache Spark (was: Xiao Li) > Keep names unchanged when deduplicating the

[jira] [Commented] (SPARK-25988) Keep names unchanged when deduplicating the column names in Analyzer

[ https://issues.apache.org/jira/browse/SPARK-25988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680903#comment-16680903 ] Apache Spark commented on SPARK-25988: -- User 'gatorsmile' has created a pull request for this

[jira] [Assigned] (SPARK-25988) Keep names unchanged when deduplicating the column names in Analyzer

[ https://issues.apache.org/jira/browse/SPARK-25988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25988: Assignee: Xiao Li (was: Apache Spark) > Keep names unchanged when deduplicating the

[jira] [Created] (SPARK-25988) Keep names unchanged when deduplicating the column names in Analyzer

Xiao Li created SPARK-25988: --- Summary: Keep names unchanged when deduplicating the column names in Analyzer Key: SPARK-25988 URL: https://issues.apache.org/jira/browse/SPARK-25988 Project: Spark

[jira] [Commented] (SPARK-25966) "EOF Reached the end of stream with bytes left to read" while reading/writing to Parquets

[ https://issues.apache.org/jira/browse/SPARK-25966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680901#comment-16680901 ] Xiao Li commented on SPARK-25966: - Thank you for reporting this. I think this is not an issue. Please

[jira] [Updated] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

[ https://issues.apache.org/jira/browse/SPARK-25987?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ivan Tsukanov updated SPARK-25987: -- Description: When I execute {code:java} val columnsCount = 100 val columns = (1 to

[jira] [Created] (SPARK-25987) StackOverflowError when executing many operations on a table with many columns

Ivan Tsukanov created SPARK-25987: - Summary: StackOverflowError when executing many operations on a table with many columns Key: SPARK-25987 URL: https://issues.apache.org/jira/browse/SPARK-25987

[jira] [Commented] (SPARK-25986) Banning throw new OutOfMemoryErrors

[ https://issues.apache.org/jira/browse/SPARK-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680899#comment-16680899 ] Apache Spark commented on SPARK-25986: -- User 'xuanyuanking' has created a pull request for this

[jira] [Comment Edited] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680889#comment-16680889 ] Hyukjin Kwon edited comment on SPARK-25976 at 11/9/18 5:29 AM: --- Can you

[jira] [Commented] (SPARK-25986) Banning throw new OutOfMemoryErrors

[ https://issues.apache.org/jira/browse/SPARK-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680898#comment-16680898 ] Apache Spark commented on SPARK-25986: -- User 'xuanyuanking' has created a pull request for this

[jira] [Assigned] (SPARK-25986) Banning throw new OutOfMemoryErrors

[ https://issues.apache.org/jira/browse/SPARK-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25986: Assignee: (was: Apache Spark) > Banning throw new OutOfMemoryErrors >

[jira] [Assigned] (SPARK-25986) Banning throw new OutOfMemoryErrors

[ https://issues.apache.org/jira/browse/SPARK-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25986: Assignee: Apache Spark > Banning throw new OutOfMemoryErrors >

[jira] [Commented] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680889#comment-16680889 ] Hyukjin Kwon commented on SPARK-25976: -- Can you describe expected input and output? Scala itself

[jira] [Commented] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680870#comment-16680870 ] Yuval Yaari commented on SPARK-25976: - Hi. Thanks for your answer.  Fold with an initial value is

[jira] [Commented] (SPARK-25982) Dataframe write is non blocking in fair scheduling mode

[ https://issues.apache.org/jira/browse/SPARK-25982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680827#comment-16680827 ] Hyukjin Kwon commented on SPARK-25982: -- Can you post reproducible codes to describe your idea, and

[jira] [Commented] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680824#comment-16680824 ] Hyukjin Kwon commented on SPARK-25976: -- Use {{fold}}. > Allow rdd.reduce on empty rdd by returning

[jira] [Resolved] (SPARK-25976) Allow rdd.reduce on empty rdd by returning an Option[T]

[ https://issues.apache.org/jira/browse/SPARK-25976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25976. -- Resolution: Won't Fix > Allow rdd.reduce on empty rdd by returning an Option[T] >

[jira] [Commented] (SPARK-25966) "EOF Reached the end of stream with bytes left to read" while reading/writing to Parquets

[ https://issues.apache.org/jira/browse/SPARK-25966?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680816#comment-16680816 ] Hyukjin Kwon commented on SPARK-25966: -- No one can reproduce this or makes sure if it's really

[jira] [Resolved] (SPARK-25961) Random numbers are not supported when handling data skew

[ https://issues.apache.org/jira/browse/SPARK-25961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25961. -- Resolution: Incomplete > Random numbers are not supported when handling data skew >

[jira] [Commented] (SPARK-25961) Random numbers are not supported when handling data skew

[ https://issues.apache.org/jira/browse/SPARK-25961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680813#comment-16680813 ] Hyukjin Kwon commented on SPARK-25961: -- It looks super difficult to read. Is it a question? or an

[jira] [Updated] (SPARK-25961) Random numbers are not supported when handling data skew

[ https://issues.apache.org/jira/browse/SPARK-25961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25961: - Description: My SQL query uses two tables to join. One table join key has null value. I use

[jira] [Commented] (SPARK-25958) error: [Errno 97] Address family not supported by protocol in dataframe.take()

[ https://issues.apache.org/jira/browse/SPARK-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680811#comment-16680811 ] Hyukjin Kwon commented on SPARK-25958: -- [~XuanYuan], FWIW, I used that way to fix a similar problem

[jira] [Commented] (SPARK-25948) Spark load floating point number is automatically rounded to an integer

[ https://issues.apache.org/jira/browse/SPARK-25948?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680810#comment-16680810 ] Hyukjin Kwon commented on SPARK-25948: -- Please avoid to set {{Critical}}+ in priority which is

[jira] [Updated] (SPARK-25948) Spark load floating point number is automatically rounded to an integer

[ https://issues.apache.org/jira/browse/SPARK-25948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-25948: - Priority: Major (was: Critical) > Spark load floating point number is automatically rounded to

[jira] [Updated] (SPARK-25986) Banning throw new OutOfMemoryErrors

[ https://issues.apache.org/jira/browse/SPARK-25986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-25986: Description: Adding a linter rule to ban the construction of new OutOfMemoryErrors and then make sure

[jira] [Resolved] (SPARK-25975) Spark History does not display necessarily the incomplete applications when requested

[ https://issues.apache.org/jira/browse/SPARK-25975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang resolved SPARK-25975. - Resolution: Duplicate > Spark History does not display necessarily the incomplete applications

[jira] [Resolved] (SPARK-25945) Support locale while parsing date/timestamp from CSV/JSON

[ https://issues.apache.org/jira/browse/SPARK-25945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25945. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22951

[jira] [Assigned] (SPARK-25945) Support locale while parsing date/timestamp from CSV/JSON

[ https://issues.apache.org/jira/browse/SPARK-25945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-25945: Assignee: Maxim Gekk > Support locale while parsing date/timestamp from CSV/JSON >

[jira] [Created] (SPARK-25986) Banning throw new OutOfMemoryErrors

Xiao Li created SPARK-25986: --- Summary: Banning throw new OutOfMemoryErrors Key: SPARK-25986 URL: https://issues.apache.org/jira/browse/SPARK-25986 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-25985) Verify the SPARK-24613 Cache with UDF could not be matched with subsequent dependent caches

Xiao Li created SPARK-25985: --- Summary: Verify the SPARK-24613 Cache with UDF could not be matched with subsequent dependent caches Key: SPARK-25985 URL: https://issues.apache.org/jira/browse/SPARK-25985

[jira] [Commented] (SPARK-25960) Support subpath mounting with Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-25960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680644#comment-16680644 ] Nihar Sheth commented on SPARK-25960: - Spoke with Tim, I'll be looking into this > Support subpath

[jira] [Updated] (SPARK-25984) Remove deprecated .newInstance(), primitive wrapper class constructor calls

[ https://issues.apache.org/jira/browse/SPARK-25984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-25984: -- Description: While working on JDK 11 support, I noticed a lot of new deprecation warnings. 80% of

[jira] [Assigned] (SPARK-25984) Remove deprecated .newInstance(), primitive box class constructor calls

[ https://issues.apache.org/jira/browse/SPARK-25984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25984: Assignee: Sean Owen (was: Apache Spark) > Remove deprecated .newInstance(), primitive

[jira] [Commented] (SPARK-25984) Remove deprecated .newInstance(), primitive box class constructor calls

[ https://issues.apache.org/jira/browse/SPARK-25984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680572#comment-16680572 ] Apache Spark commented on SPARK-25984: -- User 'srowen' has created a pull request for this issue:

[jira] [Assigned] (SPARK-25984) Remove deprecated .newInstance(), primitive box class constructor calls

[ https://issues.apache.org/jira/browse/SPARK-25984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25984: Assignee: Apache Spark (was: Sean Owen) > Remove deprecated .newInstance(), primitive

[jira] [Created] (SPARK-25984) Remove deprecated .newInstance(), primitive box class constructor calls

Sean Owen created SPARK-25984: - Summary: Remove deprecated .newInstance(), primitive box class constructor calls Key: SPARK-25984 URL: https://issues.apache.org/jira/browse/SPARK-25984 Project: Spark

[jira] [Commented] (SPARK-25904) Avoid allocating arrays too large for JVMs

[ https://issues.apache.org/jira/browse/SPARK-25904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679861#comment-16679861 ] Apache Spark commented on SPARK-25904: -- User 'squito' has created a pull request for this issue:

[jira] [Updated] (SPARK-24421) Accessing sun.misc.Cleaner in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24421: -- Summary: Accessing sun.misc.Cleaner in JDK11 (was: sun.misc.Unsafe in JDK11) > Accessing

[jira] [Created] (SPARK-25979) Window function: allow parentheses around window reference

Gengliang Wang created SPARK-25979: -- Summary: Window function: allow parentheses around window reference Key: SPARK-25979 URL: https://issues.apache.org/jira/browse/SPARK-25979 Project: Spark

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679840#comment-16679840 ] Eyal Farago commented on SPARK-24437: - [~dvogelbacher], I think what you actually want is somewhat

[jira] [Comment Edited] (SPARK-24421) sun.misc.Unsafe in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680169#comment-16680169 ] Sean Owen edited comment on SPARK-24421 at 11/8/18 9:36 PM: EDIT: most of

[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680436#comment-16680436 ] Sean Owen commented on SPARK-24421: --- EDIT: I replaced most of this comment because Kris got me past

[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680439#comment-16680439 ] Sean Owen commented on SPARK-24421: --- Hey [~Bateman], thank you, yeah your comments and my edits just

[jira] [Updated] (SPARK-24421) Accessing sun.misc.Cleaner in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-24421: -- Affects Version/s: (was: 2.3.0) 3.0.0 > Accessing sun.misc.Cleaner in

[jira] [Commented] (SPARK-25975) Spark History does not display necessarily the incomplete applications when requested

[ https://issues.apache.org/jira/browse/SPARK-25975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679788#comment-16679788 ] William Montaz commented on SPARK-25975: Associated pull request 

[jira] [Assigned] (SPARK-25975) Spark History does not display necessarily the incomplete applications when requested

[ https://issues.apache.org/jira/browse/SPARK-25975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25975: Assignee: (was: Apache Spark) > Spark History does not display necessarily the

[jira] [Commented] (SPARK-25975) Spark History does not display necessarily the incomplete applications when requested

[ https://issues.apache.org/jira/browse/SPARK-25975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679781#comment-16679781 ] Apache Spark commented on SPARK-25975: -- User 'Willymontaz' has created a pull request for this

[jira] [Assigned] (SPARK-25959) Difference in featureImportances results on computed vs saved models

[ https://issues.apache.org/jira/browse/SPARK-25959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25959: Assignee: (was: Apache Spark) > Difference in featureImportances results on computed

[jira] [Assigned] (SPARK-25970) Add Instrumentation to PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25970: Assignee: Apache Spark > Add Instrumentation to PrefixSpan >

[jira] [Resolved] (SPARK-25962) Specify minimum versions for both pydocstyle and flake8 in 'lint-python' script

[ https://issues.apache.org/jira/browse/SPARK-25962?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-25962. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 22963

[jira] [Updated] (SPARK-25975) Spark History does not display necessarily the incomplete applications when requested

[ https://issues.apache.org/jira/browse/SPARK-25975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Montaz updated SPARK-25975: --- Attachment: fix.patch > Spark History does not display necessarily the incomplete

[jira] [Updated] (SPARK-25960) Support subpath mounting with Kubernetes

[ https://issues.apache.org/jira/browse/SPARK-25960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25960: -- Affects Version/s: (was: 2.5.0) 3.0.0 > Support subpath mounting

[jira] [Commented] (SPARK-25973) Spark History Main page performance improvement

[ https://issues.apache.org/jira/browse/SPARK-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679772#comment-16679772 ] William Montaz commented on SPARK-25973: Ok created https://github.com/apache/spark/pull/22980

[jira] [Commented] (SPARK-16759) Spark expose an API to pass in Caller Context into it

[ https://issues.apache.org/jira/browse/SPARK-16759?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679135#comment-16679135 ] Aihua Xu commented on SPARK-16759: -- Seems we should implement this callerContext in SparkContext rather

[jira] [Resolved] (SPARK-25971) Ignore partition byte-size statistics in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-25971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-25971. --- Resolution: Fixed Assignee: Dongjoon Hyun Fix Version/s: 3.0.0 This is

[jira] [Updated] (SPARK-25961) 处理数据倾斜时使用随机数不支持

[ https://issues.apache.org/jira/browse/SPARK-25961?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zengxl updated SPARK-25961: --- Description: my query sql use two table join,one table join key has null value,i use rand value instead of

[jira] [Commented] (SPARK-25904) Avoid allocating arrays too large for JVMs

[ https://issues.apache.org/jira/browse/SPARK-25904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679862#comment-16679862 ] Apache Spark commented on SPARK-25904: -- User 'squito' has created a pull request for this issue:

[jira] [Commented] (SPARK-25971) Ignore partition byte-size statistics in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-25971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679400#comment-16679400 ] Apache Spark commented on SPARK-25971: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Commented] (SPARK-25979) Window function: allow parentheses around window reference

[ https://issues.apache.org/jira/browse/SPARK-25979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679977#comment-16679977 ] Apache Spark commented on SPARK-25979: -- User 'gengliangwang' has created a pull request for this

[jira] [Assigned] (SPARK-25972) Missed JSON options in streaming.py

[ https://issues.apache.org/jira/browse/SPARK-25972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25972: Assignee: (was: Apache Spark) > Missed JSON options in streaming.py >

[jira] [Assigned] (SPARK-25979) Window function: allow parentheses around window reference

[ https://issues.apache.org/jira/browse/SPARK-25979?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25979: Assignee: (was: Apache Spark) > Window function: allow parentheses around window

[jira] [Commented] (SPARK-24540) Support for multiple delimiter in Spark CSV read

[ https://issues.apache.org/jira/browse/SPARK-24540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679870#comment-16679870 ] Maxim Gekk commented on SPARK-24540: The restriction has been fixed already at least in uniVocity

[jira] [Commented] (SPARK-20156) Java String toLowerCase "Turkish locale bug" causes Spark problems

[ https://issues.apache.org/jira/browse/SPARK-20156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679542#comment-16679542 ] Apache Spark commented on SPARK-20156: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-25974) Optimizes Generates bytecode for ordering based on the given order

[ https://issues.apache.org/jira/browse/SPARK-25974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679581#comment-16679581 ] Apache Spark commented on SPARK-25974: -- User 'heary-cao' has created a pull request for this issue:

[jira] [Commented] (SPARK-25965) Add read benchmark for Avro

[ https://issues.apache.org/jira/browse/SPARK-25965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679331#comment-16679331 ] Apache Spark commented on SPARK-25965: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-24834) Utils#nanSafeCompare{Double,Float} functions do not differ from normal java double/float comparison

[ https://issues.apache.org/jira/browse/SPARK-24834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680290#comment-16680290 ] Sean Owen commented on SPARK-24834: --- The goal is to match Hive semantics, if anything. And of course

[jira] [Assigned] (SPARK-25977) Parsing decimals from CSV using locale

[ https://issues.apache.org/jira/browse/SPARK-25977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25977: Assignee: Apache Spark > Parsing decimals from CSV using locale >

[jira] [Created] (SPARK-25972) Missed JSON options in streaming.py

Maxim Gekk created SPARK-25972: -- Summary: Missed JSON options in streaming.py Key: SPARK-25972 URL: https://issues.apache.org/jira/browse/SPARK-25972 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-25904) Avoid allocating arrays too large for JVMs

[ https://issues.apache.org/jira/browse/SPARK-25904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Imran Rashid updated SPARK-25904: - Fix Version/s: 2.4.1 > Avoid allocating arrays too large for JVMs >

[jira] [Commented] (SPARK-25970) Add Instrumentation to PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25970?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679367#comment-16679367 ] Apache Spark commented on SPARK-25970: -- User 'zhengruifeng' has created a pull request for this

[jira] [Commented] (SPARK-23831) Add org.apache.derby to IsolatedClientLoader

[ https://issues.apache.org/jira/browse/SPARK-23831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679444#comment-16679444 ] Hyukjin Kwon commented on SPARK-23831: -- This is reverted at 2.4.1 and 3.0.0 > Add org.apache.derby

[jira] [Assigned] (SPARK-25970) Add Instrumentation to PrefixSpan

[ https://issues.apache.org/jira/browse/SPARK-25970?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25970: Assignee: (was: Apache Spark) > Add Instrumentation to PrefixSpan >

[jira] [Commented] (SPARK-25973) Spark History Main page performance improvement

[ https://issues.apache.org/jira/browse/SPARK-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679853#comment-16679853 ] William Montaz commented on SPARK-25973: New pull request on master branch 

[jira] [Issue Comment Deleted] (SPARK-25971) Ignore partition byte-size statistics in SQLQueryTestSuite

[ https://issues.apache.org/jira/browse/SPARK-25971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-25971: -- Comment: was deleted (was: User 'dongjoon-hyun' has created a pull request for this issue:

[jira] [Commented] (SPARK-25958) error: [Errno 97] Address family not supported by protocol in dataframe.take()

[ https://issues.apache.org/jira/browse/SPARK-25958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679261#comment-16679261 ] Ruslan Dautkhanov commented on SPARK-25958: --- [~XuanYuan] interesting.. here's our /etc/hosts:

[jira] [Commented] (SPARK-25973) Spark History Main page performance improvement

[ https://issues.apache.org/jira/browse/SPARK-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679639#comment-16679639 ] Yuming Wang commented on SPARK-25973: - Please create a pull request: 

[jira] [Commented] (SPARK-25961) 处理数据倾斜时使用随机数不支持

[ https://issues.apache.org/jira/browse/SPARK-25961?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679237#comment-16679237 ] Dongjoon Hyun commented on SPARK-25961: --- [~zengxl]. Please use English in Apache Spark JIRA. >

[jira] [Assigned] (SPARK-25975) Spark History does not display necessarily the incomplete applications when requested

[ https://issues.apache.org/jira/browse/SPARK-25975?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25975: Assignee: Apache Spark > Spark History does not display necessarily the incomplete

[jira] [Assigned] (SPARK-25897) Cannot run k8s integration tests in sbt

[ https://issues.apache.org/jira/browse/SPARK-25897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-25897: -- Assignee: Marcelo Vanzin > Cannot run k8s integration tests in sbt >

[jira] [Commented] (SPARK-25975) Spark History does not display necessarily the incomplete applications when requested

[ https://issues.apache.org/jira/browse/SPARK-25975?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679782#comment-16679782 ] Apache Spark commented on SPARK-25975: -- User 'Willymontaz' has created a pull request for this

[jira] [Comment Edited] (SPARK-24421) sun.misc.Unsafe in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680169#comment-16680169 ] Sean Owen edited comment on SPARK-24421 at 11/8/18 7:56 PM: I've found that,

[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK11

[ https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16680355#comment-16680355 ] Alan commented on SPARK-24421: -- The comment that sun.misc.Unsafe is private and not accessible in JDK 9 or

[jira] [Created] (SPARK-25983) spark-sql-kafka-0-10 no longer works with Kafka 0.10.0

Alexander Bessonov created SPARK-25983: -- Summary: spark-sql-kafka-0-10 no longer works with Kafka 0.10.0 Key: SPARK-25983 URL: https://issues.apache.org/jira/browse/SPARK-25983 Project: Spark

[jira] [Created] (SPARK-25978) Pyspark can only be used in spark-submit in spark-py docker image for kubernetes

Maxime Nannan created SPARK-25978: - Summary: Pyspark can only be used in spark-submit in spark-py docker image for kubernetes Key: SPARK-25978 URL: https://issues.apache.org/jira/browse/SPARK-25978

[jira] [Assigned] (SPARK-25977) Parsing decimals from CSV using locale

[ https://issues.apache.org/jira/browse/SPARK-25977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25977: Assignee: (was: Apache Spark) > Parsing decimals from CSV using locale >

[jira] [Resolved] (SPARK-25980) dev list mail server is down

[ https://issues.apache.org/jira/browse/SPARK-25980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-25980. - Resolution: Invalid > dev list mail server is down > > >

[jira] [Updated] (SPARK-25973) Spark History Main page performance improvement

[ https://issues.apache.org/jira/browse/SPARK-25973?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] William Montaz updated SPARK-25973: --- Priority: Minor (was: Major) > Spark History Main page performance improvement >

[jira] [Commented] (SPARK-25972) Missed JSON options in streaming.py

[ https://issues.apache.org/jira/browse/SPARK-25972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679434#comment-16679434 ] Apache Spark commented on SPARK-25972: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Commented] (SPARK-24437) Memory leak in UnsafeHashedRelation

[ https://issues.apache.org/jira/browse/SPARK-24437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679745#comment-16679745 ] Marco Gaido commented on SPARK-24437: - [~dvogelbacher] the point is: a broadcast is never

[jira] [Assigned] (SPARK-25965) Add read benchmark for Avro

[ https://issues.apache.org/jira/browse/SPARK-25965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-25965: Assignee: (was: Apache Spark) > Add read benchmark for Avro >

[jira] [Commented] (SPARK-22827) Avoid throwing OutOfMemoryError in case of exception in spill

[ https://issues.apache.org/jira/browse/SPARK-22827?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679306#comment-16679306 ] Apache Spark commented on SPARK-22827: -- User 'ueshin' has created a pull request for this issue:

  1   2   >