[jira] [Resolved] (SPARK-23526) KafkaMicroBatchV2SourceSuite.ensure stream-stream self-join generates only one offset in offset log

[ https://issues.apache.org/jira/browse/SPARK-23526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-23526. - Resolution: Duplicate > KafkaMicroBatchV2SourceSuite.ensure stream-stream self-join generates

[jira] [Commented] (SPARK-26335) Add a property for Dataset#show not to care about wide characters when padding them

[ https://issues.apache.org/jira/browse/SPARK-26335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719895#comment-16719895 ] Keiji Yoshida commented on SPARK-26335: --- > Hm, I don't think Dataset#show is supposed to be used

[jira] [Commented] (SPARK-26335) Add a property for Dataset#show not to care about wide characters when padding them

[ https://issues.apache.org/jira/browse/SPARK-26335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16719870#comment-16719870 ] ASF GitHub Bot commented on SPARK-26335: kjmrknsn opened a new pull request #23307:

[jira] [Updated] (SPARK-26358) Spark deployed mode question

[ https://issues.apache.org/jira/browse/SPARK-26358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Si Chen updated SPARK-26358: Attachment: sparkbug.jpg > Spark deployed mode question > > >

[jira] [Assigned] (SPARK-26308) Large BigDecimal value is converted to null when passed into a UDF

[ https://issues.apache.org/jira/browse/SPARK-26308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26308: Assignee: Apache Spark > Large BigDecimal value is converted to null when passed into a

[jira] [Assigned] (SPARK-26308) Large BigDecimal value is converted to null when passed into a UDF

[ https://issues.apache.org/jira/browse/SPARK-26308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26308: Assignee: (was: Apache Spark) > Large BigDecimal value is converted to null when

[jira] [Commented] (SPARK-26280) Spark will read entire CSV file even when limit is used

[ https://issues.apache.org/jira/browse/SPARK-26280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1672#comment-1672 ] Marco Gaido commented on SPARK-26280: - I'd say this is most likely a duplicate of

[jira] [Created] (SPARK-26358) Spark deployed mode question

Si Chen created SPARK-26358: --- Summary: Spark deployed mode question Key: SPARK-26358 URL: https://issues.apache.org/jira/browse/SPARK-26358 Project: Spark Issue Type: Bug Components:

[jira] [Commented] (SPARK-26222) Scan: track file listing time

[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720088#comment-16720088 ] Yuanjian Li commented on SPARK-26222: - Hi Reynold, the first PR track all file listing time into a

[jira] [Commented] (SPARK-26308) Large BigDecimal value is converted to null when passed into a UDF

[ https://issues.apache.org/jira/browse/SPARK-26308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720070#comment-16720070 ] ASF GitHub Bot commented on SPARK-26308: mgaido91 opened a new pull request #23308:

[jira] [Assigned] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26360: Assignee: Apache Spark > Avoid extra validateQuery call in createStreamingWriteSupport >

[jira] [Assigned] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26360: Assignee: (was: Apache Spark) > Avoid extra validateQuery call in

[jira] [Updated] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wu Wenjie updated SPARK-26360: -- Shepherd: Wenchen Fan > Avoid extra validateQuery call in createStreamingWriteSupport >

[jira] [Resolved] (SPARK-26340) Ensure cores per executor is greater than cpu per task

[ https://issues.apache.org/jira/browse/SPARK-26340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-26340. --- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23290

[jira] [Created] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

Hyukjin Kwon created SPARK-26362: Summary: Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts Key: SPARK-26362 URL: https://issues.apache.org/jira/browse/SPARK-26362

[jira] [Commented] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720200#comment-16720200 ] ASF GitHub Bot commented on SPARK-26360: JasonWayne opened a new pull request #23309:

[jira] [Updated] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wu Wenjie updated SPARK-26360: -- Shepherd: Sean Owen (was: Wenchen Fan) > Avoid extra validateQuery call in

[jira] [Updated] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wu Wenjie updated SPARK-26360: -- Description: When I'm reading structured streaming source code, I find there is a extra

[jira] [Assigned] (SPARK-26340) Ensure cores per executor is greater than cpu per task

[ https://issues.apache.org/jira/browse/SPARK-26340?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-26340: - Assignee: Nicolas Fraison > Ensure cores per executor is greater than cpu per task >

[jira] [Created] (SPARK-26359) Spark checkpoint restore fails after query restart

Kaspar Tint created SPARK-26359: --- Summary: Spark checkpoint restore fails after query restart Key: SPARK-26359 URL: https://issues.apache.org/jira/browse/SPARK-26359 Project: Spark Issue Type:

[jira] [Updated] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaspar Tint updated SPARK-26359: Attachment: worker-redacted state-redacted redacted-offsets

[jira] [Created] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

Wu Wenjie created SPARK-26360: - Summary: Avoid extra validateQuery call in createStreamingWriteSupport Key: SPARK-26360 URL: https://issues.apache.org/jira/browse/SPARK-26360 Project: Spark

[jira] [Commented] (SPARK-26335) Add a property for Dataset#show not to care about wide characters when padding them

[ https://issues.apache.org/jira/browse/SPARK-26335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720199#comment-16720199 ] ASF GitHub Bot commented on SPARK-26335: kjmrknsn closed pull request #23307: [SPARK-26335][SQL]

[jira] [Commented] (SPARK-26335) Add a property for Dataset#show not to care about wide characters when padding them

[ https://issues.apache.org/jira/browse/SPARK-26335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720201#comment-16720201 ] Keiji Yoshida commented on SPARK-26335: ---

[jira] [Resolved] (SPARK-26335) Add a property for Dataset#show not to care about wide characters when padding them

[ https://issues.apache.org/jira/browse/SPARK-26335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Keiji Yoshida resolved SPARK-26335. --- Resolution: Won't Fix > Add a property for Dataset#show not to care about wide characters

[jira] [Updated] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kaspar Tint updated SPARK-26359: Environment: Spark 2.4.0 deployed in standalone-client mode Checkpointing is done to S3 The Spark

[jira] [Updated] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wu Wenjie updated SPARK-26360: -- Description: When I'm reading structured streaming source code, I find there is a redundant

[jira] [Commented] (SPARK-26254) Move delegation token providers into a separate project

[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720241#comment-16720241 ] Gabor Somogyi commented on SPARK-26254: --- {quote}There was concern about using ServiceLoader

[jira] [Commented] (SPARK-26363) Remove redundant field `executorLogs` in TaskData

[ https://issues.apache.org/jira/browse/SPARK-26363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720259#comment-16720259 ] ASF GitHub Bot commented on SPARK-26363: gengliangwang opened a new pull request #23310:

[jira] [Commented] (SPARK-26340) Ensure cores per executor is greater than cpu per task

[ https://issues.apache.org/jira/browse/SPARK-26340?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720239#comment-16720239 ] ASF GitHub Bot commented on SPARK-26340: srowen closed pull request #23290: [SPARK-26340][Core]

[jira] [Assigned] (SPARK-26363) Remove redundant field `executorLogs` in TaskData

[ https://issues.apache.org/jira/browse/SPARK-26363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26363: Assignee: (was: Apache Spark) > Remove redundant field `executorLogs` in TaskData >

[jira] [Updated] (SPARK-26313) move read related methods from Table to read related mix-in traits

[ https://issues.apache.org/jira/browse/SPARK-26313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26313: - Fix Version/s: 3.0.0 > move read related methods from Table to read related mix-in traits >

[jira] [Updated] (SPARK-26362) Deprecate 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26362: - Summary: Deprecate 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

[jira] [Created] (SPARK-26361) RsuseExchange is not available in case of ColumnPruning

miaojianlong created SPARK-26361: Summary: RsuseExchange is not available in case of ColumnPruning Key: SPARK-26361 URL: https://issues.apache.org/jira/browse/SPARK-26361 Project: Spark

[jira] [Comment Edited] (SPARK-26254) Move delegation token providers into a separate project

[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720241#comment-16720241 ] Gabor Somogyi edited comment on SPARK-26254 at 12/13/18 3:29 PM: -

[jira] [Assigned] (SPARK-26362) Deprecate 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26362: Assignee: (was: Apache Spark) > Deprecate 'spark.driver.allowMultipleContexts' to

[jira] [Assigned] (SPARK-26362) Deprecate 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26362: Assignee: Apache Spark > Deprecate 'spark.driver.allowMultipleContexts' to disallow

[jira] [Comment Edited] (SPARK-26222) Scan: track file listing time

[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720088#comment-16720088 ] Yuanjian Li edited comment on SPARK-26222 at 12/13/18 3:43 PM: --- Hi

[jira] [Commented] (SPARK-26255) Custom error/exception is not thrown for the SQL tab when UI filters are added in spark-sql launch

[ https://issues.apache.org/jira/browse/SPARK-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720340#comment-16720340 ] ASF GitHub Bot commented on SPARK-26255: chakravarthiT opened a new pull request #23312:

[jira] [Commented] (SPARK-26255) Custom error/exception is not thrown for the SQL tab when UI filters are added in spark-sql launch

[ https://issues.apache.org/jira/browse/SPARK-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720353#comment-16720353 ] Chakravarthi commented on SPARK-26255: -- [~hyukjin.kwon]I have attached the snapshots > Custom

[jira] [Commented] (SPARK-26305) Breakthrough the memory limitation of broadcast join

[ https://issues.apache.org/jira/browse/SPARK-26305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720257#comment-16720257 ] Lantao Jin commented on SPARK-26305: Add a design doc. Not totally completed. > Breakthrough the

[jira] [Assigned] (SPARK-26363) Remove redundant field `executorLogs` in TaskData

[ https://issues.apache.org/jira/browse/SPARK-26363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26363: Assignee: Apache Spark > Remove redundant field `executorLogs` in TaskData >

[jira] [Assigned] (SPARK-26255) Custom error/exception is not thrown for the SQL tab when UI filters are added in spark-sql launch

[ https://issues.apache.org/jira/browse/SPARK-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26255: Assignee: Apache Spark > Custom error/exception is not thrown for the SQL tab when UI

[jira] [Comment Edited] (SPARK-26255) Custom error/exception is not thrown for the SQL tab when UI filters are added in spark-sql launch

[ https://issues.apache.org/jira/browse/SPARK-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720353#comment-16720353 ] Chakravarthi edited comment on SPARK-26255 at 12/13/18 4:16 PM:

[jira] [Created] (SPARK-26363) Remove redundant field `executorLogs` in TaskData

Gengliang Wang created SPARK-26363: -- Summary: Remove redundant field `executorLogs` in TaskData Key: SPARK-26363 URL: https://issues.apache.org/jira/browse/SPARK-26363 Project: Spark Issue

[jira] [Updated] (SPARK-26363) Avoid duplicated KV store lookups for task table

[ https://issues.apache.org/jira/browse/SPARK-26363?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gengliang Wang updated SPARK-26363: --- Summary: Avoid duplicated KV store lookups for task table (was: Remove redundant field

[jira] [Resolved] (SPARK-26313) move read related methods from Table to read related mix-in traits

[ https://issues.apache.org/jira/browse/SPARK-26313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26313. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/23266 > move read related

[jira] [Commented] (SPARK-26362) Deprecate 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720318#comment-16720318 ] ASF GitHub Bot commented on SPARK-26362: HyukjinKwon opened a new pull request #23311:

[jira] [Created] (SPARK-26364) Clean up import statements in pandas udf tests

Li Jin created SPARK-26364: -- Summary: Clean up import statements in pandas udf tests Key: SPARK-26364 URL: https://issues.apache.org/jira/browse/SPARK-26364 Project: Spark Issue Type: Improvement

[jira] [Updated] (SPARK-26255) Custom error/exception is not thrown for the SQL tab when UI filters are added in spark-sql launch

[ https://issues.apache.org/jira/browse/SPARK-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chakravarthi updated SPARK-26255: - Attachment: logs_before_fix.png > Custom error/exception is not thrown for the SQL tab when UI

[jira] [Updated] (SPARK-26255) Custom error/exception is not thrown for the SQL tab when UI filters are added in spark-sql launch

[ https://issues.apache.org/jira/browse/SPARK-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chakravarthi updated SPARK-26255: - Attachment: ui_befofre_fix.png > Custom error/exception is not thrown for the SQL tab when UI

[jira] [Assigned] (SPARK-26255) Custom error/exception is not thrown for the SQL tab when UI filters are added in spark-sql launch

[ https://issues.apache.org/jira/browse/SPARK-26255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26255: Assignee: (was: Apache Spark) > Custom error/exception is not thrown for the SQL tab

[jira] [Assigned] (SPARK-26367) Remove ReplaceExceptWithFilter from nonExcludableRules

[ https://issues.apache.org/jira/browse/SPARK-26367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26367: Assignee: Apache Spark (was: Xiao Li) > Remove ReplaceExceptWithFilter from

[jira] [Updated] (SPARK-26352) join reordering should not change the order of output attributes

[ https://issues.apache.org/jira/browse/SPARK-26352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kris Mok updated SPARK-26352: - Summary: join reordering should not change the order of output attributes (was: ReorderJoin should not

[jira] [Resolved] (SPARK-23504) Flaky test: RateSourceV2Suite.basic microbatch execution

[ https://issues.apache.org/jira/browse/SPARK-23504?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23504. Resolution: Cannot Reproduce Closing this for now based on the above comment. > Flaky

[jira] [Commented] (SPARK-23886) update query.status

[ https://issues.apache.org/jira/browse/SPARK-23886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720719#comment-16720719 ] ASF GitHub Bot commented on SPARK-23886: asfgit closed pull request #23095: [SPARK-23886][SS]

[jira] [Resolved] (SPARK-23886) update query.status

[ https://issues.apache.org/jira/browse/SPARK-23886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-23886. Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23095

[jira] [Comment Edited] (SPARK-19739) SparkHadoopUtil.appendS3AndSparkHadoopConfigurations to propagate full set of AWS env vars

[ https://issues.apache.org/jira/browse/SPARK-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720650#comment-16720650 ] Imran Rashid edited comment on SPARK-19739 at 12/13/18 10:18 PM: -

[jira] [Assigned] (SPARK-26367) Remove ReplaceExceptWithFilter from nonExcludableRules

[ https://issues.apache.org/jira/browse/SPARK-26367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26367: Assignee: Xiao Li (was: Apache Spark) > Remove ReplaceExceptWithFilter from

[jira] [Commented] (SPARK-19739) SparkHadoopUtil.appendS3AndSparkHadoopConfigurations to propagate full set of AWS env vars

[ https://issues.apache.org/jira/browse/SPARK-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720656#comment-16720656 ] Steve Loughran commented on SPARK-19739: no, leave alone. * In HADOOP-14556 I'm actually adding

[jira] [Commented] (SPARK-19739) SparkHadoopUtil.appendS3AndSparkHadoopConfigurations to propagate full set of AWS env vars

[ https://issues.apache.org/jira/browse/SPARK-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720664#comment-16720664 ] Imran Rashid commented on SPARK-19739: -- ok, sounds good, and thanks for the quick response! sounds

[jira] [Commented] (SPARK-26364) Clean up import statements in pandas udf tests

[ https://issues.apache.org/jira/browse/SPARK-26364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720858#comment-16720858 ] ASF GitHub Bot commented on SPARK-26364: asfgit closed pull request #23314:

[jira] [Resolved] (SPARK-26369) How to limit Spark concurrent tasks number in one job?

[ https://issues.apache.org/jira/browse/SPARK-26369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26369. -- Resolution: Invalid Questions should go to mailing list. I think you could have a better

[jira] [Resolved] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26360. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23309

[jira] [Assigned] (SPARK-26370) Fix resolution of higher-order function for the same identifier.

[ https://issues.apache.org/jira/browse/SPARK-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26370: Assignee: (was: Apache Spark) > Fix resolution of higher-order function for the same

[jira] [Resolved] (SPARK-26364) Clean up import statements in pandas udf tests

[ https://issues.apache.org/jira/browse/SPARK-26364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-26364. -- Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23314

[jira] [Resolved] (SPARK-26337) Add benchmark for LongToUnsafeRowMap

[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-26337. - Resolution: Fixed Fix Version/s: 3.0.0 Issue resolved by pull request 23284

[jira] [Commented] (SPARK-26222) Scan: track file listing time

[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720957#comment-16720957 ] Reynold Xin commented on SPARK-26222: - So I spent some time looking at the code base to understand

[jira] [Commented] (SPARK-26353) Add typed aggregate functions:max&

[ https://issues.apache.org/jira/browse/SPARK-26353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720997#comment-16720997 ] ASF GitHub Bot commented on SPARK-26353: 10110346 closed pull request #23304:

[jira] [Created] (SPARK-26369) How to limit Spark concurrent tasks number in one job?

Fu Chen created SPARK-26369: --- Summary: How to limit Spark concurrent tasks number in one job? Key: SPARK-26369 URL: https://issues.apache.org/jira/browse/SPARK-26369 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26360: Assignee: Wu Wenjie > Avoid extra validateQuery call in createStreamingWriteSupport >

[jira] [Assigned] (SPARK-26337) Add benchmark for LongToUnsafeRowMap

[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-26337: --- Assignee: Liang-Chi Hsieh > Add benchmark for LongToUnsafeRowMap >

[jira] [Updated] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-26362: - Summary: Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts (was:

[jira] [Commented] (SPARK-26368) Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex

[ https://issues.apache.org/jira/browse/SPARK-26368?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720809#comment-16720809 ] ASF GitHub Bot commented on SPARK-26368: rxin opened a new pull request #23317:

[jira] [Assigned] (SPARK-26368) Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex

[ https://issues.apache.org/jira/browse/SPARK-26368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26368: Assignee: Apache Spark (was: Reynold Xin) > Make it clear that

[jira] [Commented] (SPARK-26337) Add benchmark for LongToUnsafeRowMap

[ https://issues.apache.org/jira/browse/SPARK-26337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720862#comment-16720862 ] ASF GitHub Bot commented on SPARK-26337: asfgit closed pull request #23284:

[jira] [Assigned] (SPARK-26364) Clean up import statements in pandas udf tests

[ https://issues.apache.org/jira/browse/SPARK-26364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-26364: Assignee: Li Jin > Clean up import statements in pandas udf tests >

[jira] [Commented] (SPARK-26316) Because of the perf degradation in TPC-DS, we currently partial revert SPARK-21052：Add hash map metrics to join,

[ https://issues.apache.org/jira/browse/SPARK-26316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720970#comment-16720970 ] Apache Spark commented on SPARK-26316: -- User 'JkSelf' has created a pull request for this issue:

[jira] [Closed] (SPARK-26369) How to limit Spark concurrent tasks number in one job?

[ https://issues.apache.org/jira/browse/SPARK-26369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Fu Chen closed SPARK-26369. --- > How to limit Spark concurrent tasks number in one job? >

[jira] [Commented] (SPARK-26369) How to limit Spark concurrent tasks number in one job?

[ https://issues.apache.org/jira/browse/SPARK-26369?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721010#comment-16721010 ] Fu Chen commented on SPARK-26369: - ok, thank your reply, I found solution in 

[jira] [Created] (SPARK-26370) Fix resolution of higher-order function for the same identifier.

Takuya Ueshin created SPARK-26370: - Summary: Fix resolution of higher-order function for the same identifier. Key: SPARK-26370 URL: https://issues.apache.org/jira/browse/SPARK-26370 Project: Spark

[jira] [Created] (SPARK-26368) Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex

Reynold Xin created SPARK-26368: --- Summary: Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex Key: SPARK-26368 URL: https://issues.apache.org/jira/browse/SPARK-26368

[jira] [Assigned] (SPARK-26368) Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex

[ https://issues.apache.org/jira/browse/SPARK-26368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26368: Assignee: Reynold Xin (was: Apache Spark) > Make it clear that

[jira] [Commented] (SPARK-26367) Remove ReplaceExceptWithFilter from nonExcludableRules

[ https://issues.apache.org/jira/browse/SPARK-26367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720850#comment-16720850 ] ASF GitHub Bot commented on SPARK-26367: gatorsmile closed pull request #23316: [SPARK-26367]

[jira] [Commented] (SPARK-26360) Avoid extra validateQuery call in createStreamingWriteSupport

[ https://issues.apache.org/jira/browse/SPARK-26360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720859#comment-16720859 ] ASF GitHub Bot commented on SPARK-26360: asfgit closed pull request #23309: [SPARK-26360]remove

[jira] [Updated] (SPARK-26350) Allow the user to override the group id of the Kafka's consumer

[ https://issues.apache.org/jira/browse/SPARK-26350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kevin Zhang updated SPARK-26350: Attachment: Permalink.url > Allow the user to override the group id of the Kafka's consumer >

[jira] [Commented] (SPARK-26222) Scan: track file listing time

[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720974#comment-16720974 ] Yuanjian Li commented on SPARK-26222: - Copy that, thanks for your explain ! > Scan: track file

[jira] [Commented] (SPARK-26316) Because of the perf degradation in TPC-DS, we currently partial revert SPARK-21052：Add hash map metrics to join,

[ https://issues.apache.org/jira/browse/SPARK-26316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720978#comment-16720978 ] ASF GitHub Bot commented on SPARK-26316: JkSelf opened a new pull request #23319:

[jira] [Commented] (SPARK-26370) Fix resolution of higher-order function for the same identifier.

[ https://issues.apache.org/jira/browse/SPARK-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16721025#comment-16721025 ] ASF GitHub Bot commented on SPARK-26370: ueshin opened a new pull request #23320:

[jira] [Assigned] (SPARK-26370) Fix resolution of higher-order function for the same identifier.

[ https://issues.apache.org/jira/browse/SPARK-26370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26370: Assignee: Apache Spark > Fix resolution of higher-order function for the same

[jira] [Created] (SPARK-26367) Remove ReplaceExceptWithFilter from nonExcludableRules

Xiao Li created SPARK-26367: --- Summary: Remove ReplaceExceptWithFilter from nonExcludableRules Key: SPARK-26367 URL: https://issues.apache.org/jira/browse/SPARK-26367 Project: Spark Issue Type:

[jira] [Commented] (SPARK-26367) Remove ReplaceExceptWithFilter from nonExcludableRules

[ https://issues.apache.org/jira/browse/SPARK-26367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720514#comment-16720514 ] ASF GitHub Bot commented on SPARK-26367: gatorsmile opened a new pull request #23316:

[jira] [Commented] (SPARK-26359) Spark checkpoint restore fails after query restart

[ https://issues.apache.org/jira/browse/SPARK-26359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720558#comment-16720558 ] Gabor Somogyi commented on SPARK-26359: --- As I see the main issue appears because S3's

[jira] [Commented] (SPARK-26254) Move delegation token providers into a separate project

[ https://issues.apache.org/jira/browse/SPARK-26254?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720633#comment-16720633 ] Steve Loughran commented on SPARK-26254: bq. There was concern about using ServiceLoader before,

[jira] [Commented] (SPARK-19739) SparkHadoopUtil.appendS3AndSparkHadoopConfigurations to propagate full set of AWS env vars

[ https://issues.apache.org/jira/browse/SPARK-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720650#comment-16720650 ] Imran Rashid commented on SPARK-19739: -- [~ste...@apache.org] I didn't realize when using this at

[jira] [Assigned] (SPARK-23886) update query.status

[ https://issues.apache.org/jira/browse/SPARK-23886?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-23886: -- Assignee: Gabor Somogyi > update query.status > --- > >

[jira] [Assigned] (SPARK-26315) auto cast threshold from Integer to Float in approxSimilarityJoin of BucketedRandomProjectionLSHModel

[ https://issues.apache.org/jira/browse/SPARK-26315?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-26315: Assignee: (was: Apache Spark) > auto cast threshold from Integer to Float in

[jira] [Created] (SPARK-26365) spark-submit for k8s cluster doesn't propagate exit code

Oscar Bonilla created SPARK-26365: - Summary: spark-submit for k8s cluster doesn't propagate exit code Key: SPARK-26365 URL: https://issues.apache.org/jira/browse/SPARK-26365 Project: Spark

[jira] [Commented] (SPARK-26098) Show associated SQL query in Job page

[ https://issues.apache.org/jira/browse/SPARK-26098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720408#comment-16720408 ] ASF GitHub Bot commented on SPARK-26098: asfgit closed pull request #23068: [SPARK-26098][WebUI]

[jira] [Commented] (SPARK-26364) Clean up import statements in pandas udf tests

[ https://issues.apache.org/jira/browse/SPARK-26364?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720407#comment-16720407 ] ASF GitHub Bot commented on SPARK-26364: icexelloss opened a new pull request #23314:

  1   2   >