[jira] [Created] (SPARK-41637) ORDER BY ALL

2022-12-20 Thread Reynold Xin (Jira)
Reynold Xin created SPARK-41637: --- Summary: ORDER BY ALL Key: SPARK-41637 URL: https://issues.apache.org/jira/browse/SPARK-41637 Project: Spark Issue Type: New Feature Components: SQL

[jira] [Updated] (SPARK-41635) GROUP BY ALL

2022-12-20 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-41635: Description: This patch implements GROUP BY ALL, similar to the one initially implemented in Duck

[jira] [Created] (SPARK-41635) GROUP BY ALL

2022-12-20 Thread Reynold Xin (Jira)
Reynold Xin created SPARK-41635: --- Summary: GROUP BY ALL Key: SPARK-41635 URL: https://issues.apache.org/jira/browse/SPARK-41635 Project: Spark Issue Type: New Feature Components: SQL

[jira] [Created] (SPARK-41588) Make "Rule id not found" error message more actionable

2022-12-19 Thread Reynold Xin (Jira)
Reynold Xin created SPARK-41588: --- Summary: Make "Rule id not found" error message more actionable Key: SPARK-41588 URL: https://issues.apache.org/jira/browse/SPARK-41588 Project: Spark Issue Ty

[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)

2021-10-19 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430780#comment-17430780 ] Reynold Xin commented on SPARK-36913: - My concern is not about JDBC (I should've com

[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)

2021-10-18 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17430324#comment-17430324 ] Reynold Xin commented on SPARK-36913: - I'm actually a bit worried about adding stuff

[jira] [Comment Edited] (SPARK-34198) Add RocksDB StateStore as external module

2021-02-14 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284479#comment-17284479 ] Reynold Xin edited comment on SPARK-34198 at 2/14/21, 6:59 PM: ---

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore as external module

2021-02-14 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284479#comment-17284479 ] Reynold Xin commented on SPARK-34198: - I don't know the intricate details of it but

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore as external module

2021-02-14 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17284474#comment-17284474 ] Reynold Xin commented on SPARK-34198: - [~kabhwan] let me talk to the team that built

[jira] [Commented] (SPARK-31399) closure cleaner is broken in Spark 3.0

2020-04-09 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17080265#comment-17080265 ] Reynold Xin commented on SPARK-31399: - This is bad... [~sowen] and [~joshrosen]  did

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-29 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17070688#comment-17070688 ] Reynold Xin commented on SPARK-22231: - [~fqaiser94] thanks for your persistence and

[jira] [Commented] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2020-03-18 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17061916#comment-17061916 ] Reynold Xin commented on SPARK-25728: - It's too big of a change and realistically sp

[jira] [Resolved] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2020-03-18 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-25728. - Resolution: Won't Fix > SPIP: Structured Intermediate Representation (Tungsten IR) for generatin

[jira] [Commented] (SPARK-29189) Add an option to ignore block locations when listing file

2020-01-24 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023340#comment-17023340 ] Reynold Xin commented on SPARK-29189: - This is great, but how would users know when

[jira] [Commented] (SPARK-27117) current_date/current_timestamp should be reserved keywords in ansi parser mode

2020-01-24 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17023317#comment-17023317 ] Reynold Xin commented on SPARK-27117: - I changed the title to make it more clear to

[jira] [Updated] (SPARK-27117) current_date/current_timestamp should be reserved keywords in ansi parser mode

2020-01-24 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-27117: Summary: current_date/current_timestamp should be reserved keywords in ansi parser mode (was: cur

[jira] [Commented] (SPARK-29175) Make maven central repository in IsolatedClientLoader configurable

2020-01-22 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021586#comment-17021586 ] Reynold Xin commented on SPARK-29175: - I think the config should be more clear, e.g.

[jira] [Commented] (SPARK-27871) LambdaVariable should use per-query unique IDs instead of globally unique IDs

2020-01-22 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17021583#comment-17021583 ] Reynold Xin commented on SPARK-27871: - Why would anybody want to turn this off? I fe

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-01-14 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17015705#comment-17015705 ] Reynold Xin commented on SPARK-22231: - Hey sorry. Been pretty busy. I will take a lo

[jira] [Comment Edited] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-01-02 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007215#comment-17007215 ] Reynold Xin edited comment on SPARK-22231 at 1/3/20 4:18 AM: -

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-01-02 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007215#comment-17007215 ] Reynold Xin commented on SPARK-22231: - [~fqaiser94] you convinced me with #2. It'd b

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-01-02 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17007117#comment-17007117 ] Reynold Xin commented on SPARK-22231: - Makes sense. One question (I've asked about t

[jira] [Commented] (SPARK-28264) Revisiting Python / pandas UDF

2019-12-08 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16990958#comment-16990958 ] Reynold Xin commented on SPARK-28264: - Sounds good. Thanks for doing this [~hyukjin.

[jira] [Commented] (SPARK-29931) Declare all SQL legacy configs as will be removed in Spark 4.0

2019-11-17 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16976134#comment-16976134 ] Reynold Xin commented on SPARK-29931: - You can say "This config will be removed in S

[jira] [Created] (SPARK-28264) Revisiting Python / pandas UDF

2019-07-05 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-28264: --- Summary: Revisiting Python / pandas UDF Key: SPARK-28264 URL: https://issues.apache.org/jira/browse/SPARK-28264 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-27386) Improve partition transform parsing

2019-04-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16818457#comment-16818457 ] Reynold Xin commented on SPARK-27386: - [~rdblue] when will you fix this? > Improve

[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers

2019-04-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16814709#comment-16814709 ] Reynold Xin commented on SPARK-26218: - The no-exception is by design. Imagine you ha

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: mgaido91 opened a new pull request #23372: [SPARK-26366][SQL][BACKPORT

[jira] [Updated] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26362: Labels: releasenotes (was: ) > Remove 'spark.driver.allowMultipleContexts' to disallow multiple S

[jira] [Issue Comment Deleted] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26362: Comment: was deleted (was: asfgit closed pull request #23311: [SPARK-26362][CORE] Remove 'spark.d

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: mgaido91 opened a new pull request #23350: [SPARK-26366][SQL][BACKPORT

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: mgaido91 opened a new pull request #23315: [SPARK-26366][SQL] ReplaceE

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: gatorsmile closed pull request #23350: [SPARK-26366][SQL][BACKPORT-2.3]

[jira] [Commented] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734363#comment-16734363 ] Reynold Xin commented on SPARK-26366: - Please make sure you guys tag these tickets w

[jira] [Updated] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Labels: correctness (was: ) > Except with transform regression >

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: asfgit closed pull request #23315: [SPARK-26366][SQL] ReplaceExceptWit

[jira] [Issue Comment Deleted] (SPARK-26246) Infer timestamp types from JSON

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26246: Comment: was deleted (was: asfgit closed pull request #23201: [SPARK-26246][SQL] Inferring Timest

[jira] [Comment Edited] (SPARK-26246) Infer timestamp types from JSON

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734361#comment-16734361 ] Reynold Xin edited comment on SPARK-26246 at 1/4/19 5:22 PM: -

[jira] [Commented] (SPARK-26246) Infer timestamp types from JSON

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734361#comment-16734361 ] Reynold Xin commented on SPARK-26246: - |Is there an option flag for this? This is a

[jira] [Commented] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16734355#comment-16734355 ] Reynold Xin commented on SPARK-26362: - [~hyukjin.kwon] please make sure we add relea

[jira] [Commented] (SPARK-23693) SQL function uuid()

2018-12-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16728078#comment-16728078 ] Reynold Xin commented on SPARK-23693: - [~tashoyan] the issue with calling uuid direc

[jira] [Commented] (SPARK-26222) Scan: track file listing time

2018-12-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16720957#comment-16720957 ] Reynold Xin commented on SPARK-26222: - So I spent some time looking at the code base

[jira] [Issue Comment Deleted] (SPARK-26222) Scan: track file listing time

2018-12-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26222: Comment: was deleted (was: xuanyuanking opened a new pull request #23298: [SPARK-26222][SQL] Track

[jira] [Created] (SPARK-26368) Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex

2018-12-13 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26368: --- Summary: Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex Key: SPARK-26368 URL: https://issues.apache.org/jira/browse/SPARK-26368 Project:

[jira] [Commented] (SPARK-26222) Scan: track file listing time

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16715663#comment-16715663 ] Reynold Xin commented on SPARK-26222: - Do we do any file listing in non-physical pha

[jira] [Issue Comment Deleted] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Comment: was deleted (was: User 'rxin' has created a pull request for this issue: https://github.c

[jira] [Issue Comment Deleted] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Comment: was deleted (was: User 'rxin' has created a pull request for this issue: https://github.c

[jira] [Resolved] (SPARK-26139) Support passing shuffle metrics to exchange operator

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-26139. - Resolution: Fixed Fix Version/s: 3.0.0 > Support passing shuffle metrics to exchange oper

[jira] [Issue Comment Deleted] (SPARK-26193) Implement shuffle write metrics in SQL

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26193: Comment: was deleted (was: rxin commented on a change in pull request #23207: [SPARK-26193][SQL]

[jira] [Issue Comment Deleted] (SPARK-26139) Support passing shuffle metrics to exchange operator

2018-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26139: Comment: was deleted (was: User 'xuanyuanking' has created a pull request for this issue: https://

[jira] [Commented] (SPARK-26193) Implement shuffle write metrics in SQL

2018-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16706368#comment-16706368 ] Reynold Xin commented on SPARK-26193: - Can we simplify it and add those metrics only

[jira] [Resolved] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-26226. - Resolution: Fixed Fix Version/s: 3.0.0 > Update query tracker to report timeline for phas

[jira] [Created] (SPARK-26241) Add queryId to IncrementalExecution

2018-11-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26241: --- Summary: Add queryId to IncrementalExecution Key: SPARK-26241 URL: https://issues.apache.org/jira/browse/SPARK-26241 Project: Spark Issue Type: Bug C

[jira] [Updated] (SPARK-26241) Add queryId to IncrementalExecution

2018-11-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26241: Issue Type: Sub-task (was: Bug) Parent: SPARK-26221 > Add queryId to IncrementalExecution

[jira] [Assigned] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-26226: --- Assignee: Reynold Xin > Update query tracker to report timeline for phases, rather than dur

[jira] [Created] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26226: --- Summary: Update query tracker to report timeline for phases, rather than duration Key: SPARK-26226 URL: https://issues.apache.org/jira/browse/SPARK-26226 Project: Spark

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and ins

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and ins

[jira] [Updated] (SPARK-26129) Instrumentation for query planning time

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26129: Issue Type: Sub-task (was: New Feature) Parent: SPARK-26221 > Instrumentation for query p

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and ins

[jira] [Updated] (SPARK-26223) Scan: track metastore operation time

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26223: Description: The Scan node should report how much time it spent in metastore operations. Similar

[jira] [Created] (SPARK-26225) Scan: track decoding time for row-based data sources

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26225: --- Summary: Scan: track decoding time for row-based data sources Key: SPARK-26225 URL: https://issues.apache.org/jira/browse/SPARK-26225 Project: Spark Issue Type

[jira] [Created] (SPARK-26223) Scan: track metastore operation time

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26223: --- Summary: Scan: track metastore operation time Key: SPARK-26223 URL: https://issues.apache.org/jira/browse/SPARK-26223 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-26222) Scan: track file listing time

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26222: --- Summary: Scan: track file listing time Key: SPARK-26222 URL: https://issues.apache.org/jira/browse/SPARK-26222 Project: Spark Issue Type: Sub-task Co

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and inst

[jira] [Created] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26221: --- Summary: Improve Spark SQL instrumentation and metrics Key: SPARK-26221 URL: https://issues.apache.org/jira/browse/SPARK-26221 Project: Spark Issue Type: Umbre

[jira] [Commented] (SPARK-24498) Add JDK compiler for runtime codegen

2018-11-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16702515#comment-16702515 ] Reynold Xin commented on SPARK-24498: - Why don't we close the ticket? I heard we wou

[jira] [Resolved] (SPARK-26141) Enable custom shuffle metrics implementation in shuffle write

2018-11-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-26141. - Resolution: Fixed Fix Version/s: 3.0.0 > Enable custom shuffle metrics implementation in

[jira] [Assigned] (SPARK-26142) Implement shuffle read metrics in SQL

2018-11-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-26142: --- Assignee: Yuanjian Li > Implement shuffle read metrics in SQL > ---

[jira] [Updated] (SPARK-26141) Enable custom shuffle metrics implementation in shuffle write

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26141: Summary: Enable custom shuffle metrics implementation in shuffle write (was: Enable passing in cu

[jira] [Created] (SPARK-26142) Implement shuffle read metrics in SQL

2018-11-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26142: --- Summary: Implement shuffle read metrics in SQL Key: SPARK-26142 URL: https://issues.apache.org/jira/browse/SPARK-26142 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-26140) Enable custom shuffle metrics reporter in shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable custom shuffle metrics reporter in shuffle reader (was: Enable custom shuffle met

[jira] [Updated] (SPARK-26140) Enable custom shuffle metrics implementation in shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable custom shuffle metrics implementation in shuffle reader (was: Enable custom shuff

[jira] [Updated] (SPARK-26140) Enable custom shuffle metrics reporter into shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable custom shuffle metrics reporter into shuffle reader (was: Enable passing in a cus

[jira] [Created] (SPARK-26141) Enable passing in custom shuffle metrics implementation in shuffle write

2018-11-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26141: --- Summary: Enable passing in custom shuffle metrics implementation in shuffle write Key: SPARK-26141 URL: https://issues.apache.org/jira/browse/SPARK-26141 Project: Spark

[jira] [Resolved] (SPARK-26129) Instrumentation for query planning time

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-26129. - Resolution: Fixed Fix Version/s: 3.0.0 > Instrumentation for query planning time > --

[jira] [Created] (SPARK-26140) Pull TempShuffleReadMetrics creation out of shuffle layer

2018-11-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26140: --- Summary: Pull TempShuffleReadMetrics creation out of shuffle layer Key: SPARK-26140 URL: https://issues.apache.org/jira/browse/SPARK-26140 Project: Spark Issue

[jira] [Updated] (SPARK-26140) Enable passing in a custom shuffle metrics reporter into shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable passing in a custom shuffle metrics reporter into shuffle reader (was: Allow pass

[jira] [Updated] (SPARK-26140) Allow passing in a custom shuffle metrics reporter into shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Allow passing in a custom shuffle metrics reporter into shuffle reader (was: Pull TempSh

[jira] [Created] (SPARK-26139) Support passing shuffle metrics to exchange operator

2018-11-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26139: --- Summary: Support passing shuffle metrics to exchange operator Key: SPARK-26139 URL: https://issues.apache.org/jira/browse/SPARK-26139 Project: Spark Issue Type

[jira] [Created] (SPARK-26129) Instrumentation for query planning time

2018-11-20 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26129: --- Summary: Instrumentation for query planning time Key: SPARK-26129 URL: https://issues.apache.org/jira/browse/SPARK-26129 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-21559) Remove Mesos fine-grained mode

2018-11-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-21559: Target Version/s: 3.0.0 Labels: release-notes (was: ) > Remove Mesos fine-grained m

[jira] [Commented] (SPARK-17967) Support for list or other types as an option for datasources

2018-11-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16677418#comment-16677418 ] Reynold Xin commented on SPARK-17967: - BTW how important is this? Seems like for CSV

[jira] [Commented] (SPARK-25841) Redesign window function rangeBetween API

2018-10-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1559#comment-1559 ] Reynold Xin commented on SPARK-25841: - I posted api proposal sketches in  https://is

[jira] [Updated] (SPARK-25843) Redesign rangeBetween API

2018-10-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25843: Description: See parent ticket for more information. Two proposals with sketches:   Proposal 1. c

[jira] [Created] (SPARK-25862) Remove rangeBetween APIs introduced in SPARK-21608

2018-10-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25862: --- Summary: Remove rangeBetween APIs introduced in SPARK-21608 Key: SPARK-25862 URL: https://issues.apache.org/jira/browse/SPARK-25862 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-23084. - Resolution: Won't Fix Fix Version/s: (was: 2.4.0) This was merged but then reverted d

[jira] [Reopened] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-23084: - > Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark >

[jira] [Created] (SPARK-25845) Fix MatchError for calendar interval type in rangeBetween

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25845: --- Summary: Fix MatchError for calendar interval type in rangeBetween Key: SPARK-25845 URL: https://issues.apache.org/jira/browse/SPARK-25845 Project: Spark Issue

[jira] [Created] (SPARK-25844) Implement Python API once we have a new API

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25844: --- Summary: Implement Python API once we have a new API Key: SPARK-25844 URL: https://issues.apache.org/jira/browse/SPARK-25844 Project: Spark Issue Type: Sub-tas

[jira] [Updated] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25842: Target Version/s: 2.4.0 > Deprecate APIs introduced in SPARK-21608 > -

[jira] [Created] (SPARK-25843) Redesign rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25843: --- Summary: Redesign rangeBetween API Key: SPARK-25843 URL: https://issues.apache.org/jira/browse/SPARK-25843 Project: Spark Issue Type: Sub-task Compon

[jira] [Created] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25842: --- Summary: Deprecate APIs introduced in SPARK-21608 Key: SPARK-25842 URL: https://issues.apache.org/jira/browse/SPARK-25842 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25841: Description: As I was reviewing the Spark API changes for 2.4, I found that through organic, ad-h

[jira] [Created] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25841: --- Summary: Redesign window function rangeBetween API Key: SPARK-25841 URL: https://issues.apache.org/jira/browse/SPARK-25841 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25841: Description: As I was reviewing the Spark API changes for 2.4, I found that through organic, ad-h

[jira] [Created] (SPARK-25496) Deprecate from_utc_timestamp and to_utc_timestamp

2018-09-20 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25496: --- Summary: Deprecate from_utc_timestamp and to_utc_timestamp Key: SPARK-25496 URL: https://issues.apache.org/jira/browse/SPARK-25496 Project: Spark Issue Type: B

[jira] [Resolved] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-09-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-23715. - Resolution: Won't Fix Fix Version/s: (was: 2.4.0) > from_utc_timestamp returns incorr

[jira] [Reopened] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-09-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-23715: - Assignee: (was: Wenchen Fan) > from_utc_timestamp returns incorrect results for some UTC d

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-09-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16622953#comment-16622953 ] Reynold Xin commented on SPARK-23715: - the current behavior is that it only takes a

  1   2   3   4   5   6   7   8   9   10   >