[jira] [Commented] (SPARK-19498) Discussion: Making MLlib APIs extensible for 3rd party libraries

[ https://issues.apache.org/jira/browse/SPARK-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553942#comment-16553942 ] Paul Vanhaesebrouck commented on SPARK-19498: - I have been working on an extension to the

[jira] [Resolved] (SPARK-24870) Cache can't work normally if there are case letters in SQL

[ https://issues.apache.org/jira/browse/SPARK-24870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24870. - Resolution: Fixed Fix Version/s: 2.4.0 > Cache can't work normally if there are case letters in

[jira] [Assigned] (SPARK-24870) Cache can't work normally if there are case letters in SQL

[ https://issues.apache.org/jira/browse/SPARK-24870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reassigned SPARK-24870: --- Assignee: eaton > Cache can't work normally if there are case letters in SQL >

[jira] [Created] (SPARK-24897) DAGScheduler should not unregisterMapOutput and increaseEpoch repeatedly for stage fetchFailed

liupengcheng created SPARK-24897: Summary: DAGScheduler should not unregisterMapOutput and increaseEpoch repeatedly for stage fetchFailed Key: SPARK-24897 URL: https://issues.apache.org/jira/browse/SPARK-24897

[jira] [Assigned] (SPARK-24738) [HistoryServer] FsHistoryProvider clean outdated event logs at start

[ https://issues.apache.org/jira/browse/SPARK-24738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24738: Assignee: (was: Apache Spark) > [HistoryServer] FsHistoryProvider clean outdated

[jira] [Commented] (SPARK-24738) [HistoryServer] FsHistoryProvider clean outdated event logs at start

[ https://issues.apache.org/jira/browse/SPARK-24738?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553903#comment-16553903 ] Apache Spark commented on SPARK-24738: -- User 'LiShuMing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24738) [HistoryServer] FsHistoryProvider clean outdated event logs at start

[ https://issues.apache.org/jira/browse/SPARK-24738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24738: Assignee: Apache Spark > [HistoryServer] FsHistoryProvider clean outdated event logs at

[jira] [Commented] (SPARK-22499) 64KB JVM bytecode limit problem with least and greatest

[ https://issues.apache.org/jira/browse/SPARK-22499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553879#comment-16553879 ] Apache Spark commented on SPARK-22499: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Resolved] (SPARK-24886) Increase Jenkins build time

[ https://issues.apache.org/jira/browse/SPARK-24886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-24886. -- Resolution: Won't Fix It will be reopened when it's needed next time > Increase Jenkins

[jira] [Commented] (SPARK-24897) DAGScheduler should not unregisterMapOutput and increaseEpoch repeatedly for stage fetchFailed

[ https://issues.apache.org/jira/browse/SPARK-24897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553907#comment-16553907 ] liupengcheng commented on SPARK-24897: -- Can some one verify the issue? I checked the 2.3.1 branch,

[jira] [Commented] (SPARK-21274) Implement EXCEPT ALL and INTERSECT ALL

[ https://issues.apache.org/jira/browse/SPARK-21274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553924#comment-16553924 ] Apache Spark commented on SPARK-21274: -- User 'dilipbiswal' has created a pull request for this

[jira] [Commented] (SPARK-19498) Discussion: Making MLlib APIs extensible for 3rd party libraries

[ https://issues.apache.org/jira/browse/SPARK-19498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16553945#comment-16553945 ] Peter Knight commented on SPARK-19498: -- Thank you for your e-mail. I am on holiday until Wednesday

[jira] [Updated] (SPARK-24909) Spark scheduler can hang with fetch failures and executor lost and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-24909: -- Description: The DAGScheduler can hang if the executor was lost (due to fetch failure) and

[jira] [Updated] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-24895: --- Description: Spark 2.4.0 has Maven build errors because artifacts uploaded to apache maven repo

[jira] [Assigned] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-24895: -- Assignee: Eric Chang > Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched

[jira] [Resolved] (SPARK-24812) Last Access Time in the table description is not valid

[ https://issues.apache.org/jira/browse/SPARK-24812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24812. - Resolution: Fixed Assignee: Sujith Fix Version/s: 2.4.0 > Last Access Time in the table

[jira] [Commented] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554748#comment-16554748 ] Imran Rashid commented on SPARK-24909: -- ugh, yeah I think you're right. do you have a fix in mind?

[jira] [Created] (SPARK-24909) Spark scheduler can hang with fetch failures and executor lost and multiple stage attempts

Thomas Graves created SPARK-24909: - Summary: Spark scheduler can hang with fetch failures and executor lost and multiple stage attempts Key: SPARK-24909 URL: https://issues.apache.org/jira/browse/SPARK-24909

[jira] [Updated] (SPARK-24909) Spark scheduler can hang with fetch failures and executor lost and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-24909: -- Priority: Critical (was: Major) > Spark scheduler can hang with fetch failures and executor

[jira] [Commented] (SPARK-24909) Spark scheduler can hang with fetch failures and executor lost and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554701#comment-16554701 ] Thomas Graves commented on SPARK-24909: --- Note this may have been introduced as part of SPARK-23433

[jira] [Updated] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-24909: -- Description: The DAGScheduler can hang if the executor was lost (due to fetch failure) and

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554742#comment-16554742 ] Imran Rashid commented on SPARK-24615: -- hi, just catching up here -- agree with a lot of Tom's

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554595#comment-16554595 ] Apache Spark commented on SPARK-24895: -- User 'ericfchang' has created a pull request for this

[jira] [Commented] (SPARK-24908) [R] remove spaces to make lintr happy

[ https://issues.apache.org/jira/browse/SPARK-24908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554597#comment-16554597 ] shane knapp commented on SPARK-24908: - i noticed lintr complaining in my PRB build

[jira] [Assigned] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24895: Assignee: Apache Spark > Spark 2.4.0 Snapshot artifacts has broken metadata due to

[jira] [Commented] (SPARK-24894) Invalid DNS name due to hostname truncation

[ https://issues.apache.org/jira/browse/SPARK-24894?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554602#comment-16554602 ] Yinan Li commented on SPARK-24894: -- [~mcheah]. We need to make sure the truncation leads to a valid

[jira] [Assigned] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24895: Assignee: (was: Apache Spark) > Spark 2.4.0 Snapshot artifacts has broken metadata

[jira] [Commented] (SPARK-24768) Have a built-in AVRO data source implementation

[ https://issues.apache.org/jira/browse/SPARK-24768?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554715#comment-16554715 ] Apache Spark commented on SPARK-24768: -- User 'gengliangwang' has created a pull request for this

[jira] [Commented] (SPARK-24906) Enlarge split size for columnar file to ensure the task read enough data

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554716#comment-16554716 ] Liang-Chi Hsieh commented on SPARK-24906: - A {{maxPartitionBytes}} value adapted improperly

[jira] [Resolved] (SPARK-23325) DataSourceV2 readers should always produce InternalRow.

[ https://issues.apache.org/jira/browse/SPARK-23325?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23325. - Resolution: Fixed Assignee: Ryan Blue Fix Version/s: 2.4.0 > DataSourceV2 readers

[jira] [Updated] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-24909: -- Summary: Spark scheduler can hang when fetch failures, executor lost, task running on lost

[jira] [Commented] (SPARK-24899) Add example of monotonically_increasing_id standard function to scaladoc

[ https://issues.apache.org/jira/browse/SPARK-24899?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554022#comment-16554022 ] Apache Spark commented on SPARK-24899: -- User 'jaceklaskowski' has created a pull request for this

[jira] [Assigned] (SPARK-24899) Add example of monotonically_increasing_id standard function to scaladoc

[ https://issues.apache.org/jira/browse/SPARK-24899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24899: Assignee: (was: Apache Spark) > Add example of monotonically_increasing_id standard

[jira] [Assigned] (SPARK-24899) Add example of monotonically_increasing_id standard function to scaladoc

[ https://issues.apache.org/jira/browse/SPARK-24899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24899: Assignee: Apache Spark > Add example of monotonically_increasing_id standard function to

[jira] [Created] (SPARK-24900) speed up sort when the dataset is small

SongXun created SPARK-24900: --- Summary: speed up sort when the dataset is small Key: SPARK-24900 URL: https://issues.apache.org/jira/browse/SPARK-24900 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-24903) Allow driver container name to be configurable in k8 spark

Yifei Huang created SPARK-24903: --- Summary: Allow driver container name to be configurable in k8 spark Key: SPARK-24903 URL: https://issues.apache.org/jira/browse/SPARK-24903 Project: Spark

[jira] [Created] (SPARK-24898) Include configuration parameter in doc

Riccardo Corbella created SPARK-24898: - Summary: Include configuration parameter in doc Key: SPARK-24898 URL: https://issues.apache.org/jira/browse/SPARK-24898 Project: Spark Issue Type:

[jira] [Created] (SPARK-24899) Add example of monotonically_increasing_id standard function to scaladoc

Jacek Laskowski created SPARK-24899: --- Summary: Add example of monotonically_increasing_id standard function to scaladoc Key: SPARK-24899 URL: https://issues.apache.org/jira/browse/SPARK-24899

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554043#comment-16554043 ] Genmao Yu commented on SPARK-24630: --- {{Structured Streaming supports standard SQL as the batch

[jira] [Comment Edited] (SPARK-24630) SPIP: Support SQLStreaming in Spark

[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554043#comment-16554043 ] Genmao Yu edited comment on SPARK-24630 at 7/24/18 10:07 AM: - {{Structured

[jira] [Assigned] (SPARK-24900) speed up sort when the dataset is small

[ https://issues.apache.org/jira/browse/SPARK-24900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24900: Assignee: Apache Spark > speed up sort when the dataset is small >

[jira] [Commented] (SPARK-24900) speed up sort when the dataset is small

[ https://issues.apache.org/jira/browse/SPARK-24900?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554094#comment-16554094 ] Apache Spark commented on SPARK-24900: -- User 'sddyljsx' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24900) speed up sort when the dataset is small

[ https://issues.apache.org/jira/browse/SPARK-24900?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24900: Assignee: (was: Apache Spark) > speed up sort when the dataset is small >

[jira] [Created] (SPARK-24902) Add integration tests for PVs

Stavros Kontopoulos created SPARK-24902: --- Summary: Add integration tests for PVs Key: SPARK-24902 URL: https://issues.apache.org/jira/browse/SPARK-24902 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24288) Enable preventing predicate pushdown

[ https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554109#comment-16554109 ] Tomasz Gawęda commented on SPARK-24288: --- [~smilegator] Yes, you are right. If we don't want to use

[jira] [Created] (SPARK-24901) Merge the codegen of RegularHashMap and fastHashMap to reduce compiler maxCodesize when VectorizedHashMap is false

caoxuewen created SPARK-24901: - Summary: Merge the codegen of RegularHashMap and fastHashMap to reduce compiler maxCodesize when VectorizedHashMap is false Key: SPARK-24901 URL:

[jira] [Updated] (SPARK-24903) Allow driver container name to be configurable in k8 spark

[ https://issues.apache.org/jira/browse/SPARK-24903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yifei Huang updated SPARK-24903: Description: We'd like to expose the container name as a configurable value. In case it changes

[jira] [Updated] (SPARK-24902) Add integration tests for PVs

[ https://issues.apache.org/jira/browse/SPARK-24902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Stavros Kontopoulos updated SPARK-24902: Description: PVs and hostpath support has been added recently

[jira] [Created] (SPARK-24904) Join with broadcasted dataframe causes shuffle of redundant data

Shay Elbaz created SPARK-24904: -- Summary: Join with broadcasted dataframe causes shuffle of redundant data Key: SPARK-24904 URL: https://issues.apache.org/jira/browse/SPARK-24904 Project: Spark

[jira] [Commented] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554099#comment-16554099 ] Stavros Kontopoulos commented on SPARK-24434: - [~liyinan926] Should we move with the

[jira] [Comment Edited] (SPARK-24434) Support user-specified driver and executor pod templates

[ https://issues.apache.org/jira/browse/SPARK-24434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554099#comment-16554099 ] Stavros Kontopoulos edited comment on SPARK-24434 at 7/24/18 11:06 AM:

[jira] [Commented] (SPARK-24901) Merge the codegen of RegularHashMap and fastHashMap to reduce compiler maxCodesize when VectorizedHashMap is false

[ https://issues.apache.org/jira/browse/SPARK-24901?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554106#comment-16554106 ] Apache Spark commented on SPARK-24901: -- User 'heary-cao' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24901) Merge the codegen of RegularHashMap and fastHashMap to reduce compiler maxCodesize when VectorizedHashMap is false

[ https://issues.apache.org/jira/browse/SPARK-24901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24901: Assignee: (was: Apache Spark) > Merge the codegen of RegularHashMap and fastHashMap

[jira] [Assigned] (SPARK-24901) Merge the codegen of RegularHashMap and fastHashMap to reduce compiler maxCodesize when VectorizedHashMap is false

[ https://issues.apache.org/jira/browse/SPARK-24901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24901: Assignee: Apache Spark > Merge the codegen of RegularHashMap and fastHashMap to reduce

[jira] [Commented] (SPARK-24909) Spark scheduler can hang when fetch failures, executor lost, task running on lost executor, and multiple stage attempts

[ https://issues.apache.org/jira/browse/SPARK-24909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554803#comment-16554803 ] Thomas Graves commented on SPARK-24909: --- I haven't come up with a fix yet but have been looking at

[jira] [Created] (SPARK-24910) Spark Bloom Filter Closure Serialization improvement for very high volume of Data

Himangshu Ranjan Borah created SPARK-24910: -- Summary: Spark Bloom Filter Closure Serialization improvement for very high volume of Data Key: SPARK-24910 URL:

[jira] [Created] (SPARK-24911) SHOW CREATE TABLE drops escaping of nested column names

Maxim Gekk created SPARK-24911: -- Summary: SHOW CREATE TABLE drops escaping of nested column names Key: SPARK-24911 URL: https://issues.apache.org/jira/browse/SPARK-24911 Project: Spark Issue

[jira] [Created] (SPARK-24912) Broadcast join OutOfMemory stack trace obscures actual cause of OOM

Bruce Robbins created SPARK-24912: - Summary: Broadcast join OutOfMemory stack trace obscures actual cause of OOM Key: SPARK-24912 URL: https://issues.apache.org/jira/browse/SPARK-24912 Project: Spark

[jira] [Commented] (SPARK-24911) SHOW CREATE TABLE drops escaping of nested column names

[ https://issues.apache.org/jira/browse/SPARK-24911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554839#comment-16554839 ] Apache Spark commented on SPARK-24911: -- User 'MaxGekk' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24911) SHOW CREATE TABLE drops escaping of nested column names

[ https://issues.apache.org/jira/browse/SPARK-24911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24911: Assignee: Apache Spark > SHOW CREATE TABLE drops escaping of nested column names >

[jira] [Assigned] (SPARK-24911) SHOW CREATE TABLE drops escaping of nested column names

[ https://issues.apache.org/jira/browse/SPARK-24911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24911: Assignee: (was: Apache Spark) > SHOW CREATE TABLE drops escaping of nested column

[jira] [Updated] (SPARK-24912) Broadcast join OutOfMemory stack trace obscures actual cause of OOM

[ https://issues.apache.org/jira/browse/SPARK-24912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24912: -- Priority: Minor (was: Major) > Broadcast join OutOfMemory stack trace obscures actual cause

[jira] [Resolved] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-24895. -- Resolution: Fixed Fix Version/s: 2.4.0 [https://github.com/apache/spark/pull/21865] has been

[jira] [Commented] (SPARK-24307) Support sending messages over 2GB from memory

[ https://issues.apache.org/jira/browse/SPARK-24307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554784#comment-16554784 ] Apache Spark commented on SPARK-24307: -- User 'squito' has created a pull request for this issue:

[jira] [Resolved] (SPARK-24908) [R] remove spaces to make lintr happy

[ https://issues.apache.org/jira/browse/SPARK-24908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai resolved SPARK-24908. - Resolution: Fixed Target Version/s: 2.4.0 > [R] remove spaces to make lintr happy >

[jira] [Created] (SPARK-24913) Make `AssertTrue` and `AssertNotNull` non-deterministic

DB Tsai created SPARK-24913: --- Summary: Make `AssertTrue` and `AssertNotNull` non-deterministic Key: SPARK-24913 URL: https://issues.apache.org/jira/browse/SPARK-24913 Project: Spark Issue Type:

[jira] [Created] (SPARK-24914) totalSize is not a good estimate for broadcast joins

Bruce Robbins created SPARK-24914: - Summary: totalSize is not a good estimate for broadcast joins Key: SPARK-24914 URL: https://issues.apache.org/jira/browse/SPARK-24914 Project: Spark Issue

[jira] [Comment Edited] (SPARK-24906) Enlarge split size for columnar file to ensure the task read enough data

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554972#comment-16554972 ] Jason Guo edited comment on SPARK-24906 at 7/25/18 1:03 AM: Thanks [~maropu]

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555027#comment-16555027 ] Hyukjin Kwon commented on SPARK-24895: -- Thank you [~yhuai]. I couldn't foresee this problem. >

[jira] [Commented] (SPARK-24663) Flaky test: StreamingContextSuite "stop slow receiver gracefully"

[ https://issues.apache.org/jira/browse/SPARK-24663?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555028#comment-16555028 ] Yuming Wang commented on SPARK-24663: -

[jira] [Updated] (SPARK-24529) Add spotbugs into maven build process

[ https://issues.apache.org/jira/browse/SPARK-24529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-24529: - Fix Version/s: (was: 2.4.0) > Add spotbugs into maven build process >

[jira] [Reopened] (SPARK-24529) Add spotbugs into maven build process

[ https://issues.apache.org/jira/browse/SPARK-24529?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reopened SPARK-24529: -- This was reverted in favour of https://github.com/apache/spark/pull/21865 and SPARK-24895 for

[jira] [Issue Comment Deleted] (SPARK-24663) Flaky test: StreamingContextSuite "stop slow receiver gracefully"

[ https://issues.apache.org/jira/browse/SPARK-24663?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-24663: Comment: was deleted (was:

[jira] [Commented] (SPARK-23622) Flaky Test: HiveClientSuites

[ https://issues.apache.org/jira/browse/SPARK-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555041#comment-16555041 ] Yuming Wang commented on SPARK-23622: -

[jira] [Resolved] (SPARK-24891) Fix HandleNullInputsForUDF rule

[ https://issues.apache.org/jira/browse/SPARK-24891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24891. - Resolution: Fixed Assignee: Maryann Xue Fix Version/s: 2.4.0 2.3.2 >

[jira] [Comment Edited] (SPARK-23622) Flaky Test: HiveClientSuites

[ https://issues.apache.org/jira/browse/SPARK-23622?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555041#comment-16555041 ] Yuming Wang edited comment on SPARK-23622 at 7/25/18 2:44 AM: --

[jira] [Commented] (SPARK-24615) Accelerator-aware task scheduling for Spark

[ https://issues.apache.org/jira/browse/SPARK-24615?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555080#comment-16555080 ] Saisai Shao commented on SPARK-24615: - Hi [~tgraves] [~irashid] thanks a lot for your comments.

[jira] [Updated] (SPARK-24914) totalSize is not a good estimate for broadcast joins

[ https://issues.apache.org/jira/browse/SPARK-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bruce Robbins updated SPARK-24914: -- Description: When determining whether to do a broadcast join, Spark estimates the size of

[jira] [Commented] (SPARK-24906) Enlarge split size for columnar file to ensure the task read enough data

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554972#comment-16554972 ] Jason Guo commented on SPARK-24906: --- Thanks [~maropu] and [~viirya] for your comments. Here is my

[jira] [Commented] (SPARK-24911) SHOW CREATE TABLE drops escaping of nested column names

[ https://issues.apache.org/jira/browse/SPARK-24911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554990#comment-16554990 ] Yuming Wang commented on SPARK-24911: - Can we show non printable field delimiter when {{SHOW CREATE

[jira] [Commented] (SPARK-24778) DateTimeUtils.getTimeZone method returns GMT time if timezone cannot be parsed

[ https://issues.apache.org/jira/browse/SPARK-24778?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554907#comment-16554907 ] Vinitha Reddy Gankidi commented on SPARK-24778: --- [~maropu] Sorry, I missed seeing this

[jira] [Commented] (SPARK-24895) Spark 2.4.0 Snapshot artifacts has broken metadata due to mismatched filenames

[ https://issues.apache.org/jira/browse/SPARK-24895?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554932#comment-16554932 ] Yin Huai commented on SPARK-24895: -- [~hyukjin.kwon] [~kiszk] seems this revert indeed fixed the problem

[jira] [Assigned] (SPARK-24297) Change default value for spark.maxRemoteBlockSizeFetchToMem to be < 2GB

[ https://issues.apache.org/jira/browse/SPARK-24297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao reassigned SPARK-24297: --- Assignee: Imran Rashid > Change default value for spark.maxRemoteBlockSizeFetchToMem to be

[jira] [Resolved] (SPARK-24297) Change default value for spark.maxRemoteBlockSizeFetchToMem to be < 2GB

[ https://issues.apache.org/jira/browse/SPARK-24297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Saisai Shao resolved SPARK-24297. - Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 21474

[jira] [Commented] (SPARK-24867) Add AnalysisBarrier to DataFrameWriter

[ https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554986#comment-16554986 ] Saisai Shao commented on SPARK-24867: - [~smilegator] what's the ETA of this issue? > Add

[jira] [Commented] (SPARK-24897) DAGScheduler should not unregisterMapOutput and increaseEpoch repeatedly for stage fetchFailed

[ https://issues.apache.org/jira/browse/SPARK-24897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554988#comment-16554988 ] liupengcheng commented on SPARK-24897: -- [~srowen] anybody can verify this issue? Thanks a lot >

[jira] [Commented] (SPARK-24897) DAGScheduler should not unregisterMapOutput and increaseEpoch repeatedly for stage fetchFailed

[ https://issues.apache.org/jira/browse/SPARK-24897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555002#comment-16555002 ] Sean Owen commented on SPARK-24897: --- I don't understand what you're reporting. I think you should

[jira] [Commented] (SPARK-24906) Enlarge split size for columnar file to ensure the task read enough data

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555023#comment-16555023 ] Apache Spark commented on SPARK-24906: -- User 'habren' has created a pull request for this issue:

[jira] [Assigned] (SPARK-24906) Enlarge split size for columnar file to ensure the task read enough data

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24906: Assignee: (was: Apache Spark) > Enlarge split size for columnar file to ensure the

[jira] [Assigned] (SPARK-24906) Enlarge split size for columnar file to ensure the task read enough data

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24906: Assignee: Apache Spark > Enlarge split size for columnar file to ensure the task read

[jira] [Resolved] (SPARK-24890) Short circuiting the `if` condition when `trueValue` and `falseValue` are the same

[ https://issues.apache.org/jira/browse/SPARK-24890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-24890. - Resolution: Fixed Fix Version/s: 2.4.0 > Short circuiting the `if` condition when `trueValue`

[jira] [Resolved] (SPARK-23957) Sorts in subqueries are redundant and can be removed

[ https://issues.apache.org/jira/browse/SPARK-23957?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23957. - Resolution: Fixed Assignee: Henry Robinson Fix Version/s: 2.4.0 > Sorts in subqueries

[jira] [Commented] (SPARK-24891) Fix HandleNullInputsForUDF rule

[ https://issues.apache.org/jira/browse/SPARK-24891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16555111#comment-16555111 ] Apache Spark commented on SPARK-24891: -- User 'gatorsmile' has created a pull request for this

[jira] [Assigned] (SPARK-24907) Migrate JDBC data source to DataSource API v2

[ https://issues.apache.org/jira/browse/SPARK-24907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-24907: Assignee: Apache Spark > Migrate JDBC data source to DataSource API v2 >

[jira] [Updated] (SPARK-24906) Enlarge split size for columnar file to ensure the task read enough data

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-24906: -- Description: For columnar file, such as, when spark sql read the table, each split will be 128 MB by

[jira] [Created] (SPARK-24907) Migrate JDBC data source to DataSource API v2

Teng Peng created SPARK-24907: - Summary: Migrate JDBC data source to DataSource API v2 Key: SPARK-24907 URL: https://issues.apache.org/jira/browse/SPARK-24907 Project: Spark Issue Type: New

[jira] [Commented] (SPARK-24630) SPIP: Support SQLStreaming in Spark

[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554210#comment-16554210 ] Jackey Lee commented on SPARK-24630: In SQLStreaming, we also support standard SQL as batch queries,

[jira] [Comment Edited] (SPARK-24630) SPIP: Support SQLStreaming in Spark

[ https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16554043#comment-16554043 ] Genmao Yu edited comment on SPARK-24630 at 7/24/18 11:38 AM: - [~zsxwing]

[jira] [Updated] (SPARK-24906) Enlarge split size for columnar file to ensure the task read enough data

[ https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Guo updated SPARK-24906: -- Attachment: image-2018-07-24-20-30-24-552.png > Enlarge split size for columnar file to ensure the

[jira] [Updated] (SPARK-24905) Spark 2.3 Internal URL env variable

[ https://issues.apache.org/jira/browse/SPARK-24905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Björn Wenzel updated SPARK-24905: - Priority: Critical (was: Major) > Spark 2.3 Internal URL env variable >

  1   2   >