[jira] [Created] (SPARK-41637) ORDER BY ALL

2022-12-20 Thread Reynold Xin (Jira)
Reynold Xin created SPARK-41637: --- Summary: ORDER BY ALL Key: SPARK-41637 URL: https://issues.apache.org/jira/browse/SPARK-41637 Project: Spark Issue Type: New Feature Components: SQL

[jira] [Updated] (SPARK-41635) GROUP BY ALL

2022-12-20 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-41635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-41635: Description: This patch implements GROUP BY ALL, similar to the one initially implemented in

[jira] [Created] (SPARK-41635) GROUP BY ALL

2022-12-20 Thread Reynold Xin (Jira)
Reynold Xin created SPARK-41635: --- Summary: GROUP BY ALL Key: SPARK-41635 URL: https://issues.apache.org/jira/browse/SPARK-41635 Project: Spark Issue Type: New Feature Components: SQL

[jira] [Created] (SPARK-41588) Make "Rule id not found" error message more actionable

2022-12-19 Thread Reynold Xin (Jira)
Reynold Xin created SPARK-41588: --- Summary: Make "Rule id not found" error message more actionable Key: SPARK-41588 URL: https://issues.apache.org/jira/browse/SPARK-41588 Project: Spark Issue

[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)

2021-10-19 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430780#comment-17430780 ] Reynold Xin commented on SPARK-36913: - My concern is not about JDBC (I should've commented on the

[jira] [Commented] (SPARK-36913) Implement createIndex and IndexExists in JDBC (MySQL dialect)

2021-10-19 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-36913?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17430324#comment-17430324 ] Reynold Xin commented on SPARK-36913: - I'm actually a bit worried about adding stuff like this to

[jira] [Comment Edited] (SPARK-34198) Add RocksDB StateStore as external module

2021-02-14 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284479#comment-17284479 ] Reynold Xin edited comment on SPARK-34198 at 2/14/21, 6:59 PM: --- I don't

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore as external module

2021-02-14 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284479#comment-17284479 ] Reynold Xin commented on SPARK-34198: - I don't know the intricate details of it but I suspect it's a

[jira] [Commented] (SPARK-34198) Add RocksDB StateStore as external module

2021-02-14 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17284474#comment-17284474 ] Reynold Xin commented on SPARK-34198: - [~kabhwan] let me talk to the team that built our internal

[jira] [Commented] (SPARK-31399) closure cleaner is broken in Spark 3.0

2020-04-09 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-31399?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17080265#comment-17080265 ] Reynold Xin commented on SPARK-31399: - This is bad... [~sowen] and [~joshrosen]  did you look into

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-03-29 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17070688#comment-17070688 ] Reynold Xin commented on SPARK-22231: - [~fqaiser94] thanks for your persistence and my apologies for

[jira] [Commented] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2020-03-18 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17061916#comment-17061916 ] Reynold Xin commented on SPARK-25728: - It's too big of a change and realistically speaking probably

[jira] [Resolved] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2020-03-18 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-25728. - Resolution: Won't Fix > SPIP: Structured Intermediate Representation (Tungsten IR) for

[jira] [Commented] (SPARK-29189) Add an option to ignore block locations when listing file

2020-01-24 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023340#comment-17023340 ] Reynold Xin commented on SPARK-29189: - This is great, but how would users know when to set this?

[jira] [Commented] (SPARK-27117) current_date/current_timestamp should be reserved keywords in ansi parser mode

2020-01-24 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17023317#comment-17023317 ] Reynold Xin commented on SPARK-27117: - I changed the title to make it more clear to end users what's

[jira] [Updated] (SPARK-27117) current_date/current_timestamp should be reserved keywords in ansi parser mode

2020-01-24 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27117?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-27117: Summary: current_date/current_timestamp should be reserved keywords in ansi parser mode (was:

[jira] [Commented] (SPARK-29175) Make maven central repository in IsolatedClientLoader configurable

2020-01-22 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29175?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021586#comment-17021586 ] Reynold Xin commented on SPARK-29175: - I think the config should be more clear, e.g.

[jira] [Commented] (SPARK-27871) LambdaVariable should use per-query unique IDs instead of globally unique IDs

2020-01-22 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-27871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17021583#comment-17021583 ] Reynold Xin commented on SPARK-27871: - Why would anybody want to turn this off? I feel like this

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-01-14 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17015705#comment-17015705 ] Reynold Xin commented on SPARK-22231: - Hey sorry. Been pretty busy. I will take a look this week. >

[jira] [Comment Edited] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-01-02 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007215#comment-17007215 ] Reynold Xin edited comment on SPARK-22231 at 1/3/20 4:18 AM: - [~fqaiser94] 

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-01-02 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007215#comment-17007215 ] Reynold Xin commented on SPARK-22231: - [~fqaiser94] you convinced me with #2. It'd be very verbose

[jira] [Commented] (SPARK-22231) Support of map, filter, withColumn, dropColumn in nested list of structures

2020-01-02 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-22231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17007117#comment-17007117 ] Reynold Xin commented on SPARK-22231: - Makes sense. One question (I've asked about this before):

[jira] [Commented] (SPARK-28264) Revisiting Python / pandas UDF

2019-12-08 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16990958#comment-16990958 ] Reynold Xin commented on SPARK-28264: - Sounds good. Thanks for doing this [~hyukjin.kwon]! >

[jira] [Commented] (SPARK-29931) Declare all SQL legacy configs as will be removed in Spark 4.0

2019-11-17 Thread Reynold Xin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29931?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16976134#comment-16976134 ] Reynold Xin commented on SPARK-29931: - You can say "This config will be removed in Spark 4.0 or a

[jira] [Created] (SPARK-28264) Revisiting Python / pandas UDF

2019-07-05 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-28264: --- Summary: Revisiting Python / pandas UDF Key: SPARK-28264 URL: https://issues.apache.org/jira/browse/SPARK-28264 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-27386) Improve partition transform parsing

2019-04-15 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16818457#comment-16818457 ] Reynold Xin commented on SPARK-27386: - [~rdblue] when will you fix this? > Improve partition

[jira] [Commented] (SPARK-26218) Throw exception on overflow for integers

2019-04-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16814709#comment-16814709 ] Reynold Xin commented on SPARK-26218: - The no-exception is by design. Imagine you have an ETL job

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: mgaido91 opened a new pull request #23372:

[jira] [Updated] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26362: Labels: releasenotes (was: ) > Remove 'spark.driver.allowMultipleContexts' to disallow multiple

[jira] [Issue Comment Deleted] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26362: Comment: was deleted (was: asfgit closed pull request #23311: [SPARK-26362][CORE] Remove

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: mgaido91 opened a new pull request #23350:

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: mgaido91 opened a new pull request #23315: [SPARK-26366][SQL]

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: gatorsmile closed pull request #23350:

[jira] [Commented] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16734363#comment-16734363 ] Reynold Xin commented on SPARK-26366: - Please make sure you guys tag these tickets with correctness

[jira] [Updated] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Labels: correctness (was: ) > Except with transform regression >

[jira] [Issue Comment Deleted] (SPARK-26366) Except with transform regression

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26366?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26366: Comment: was deleted (was: asfgit closed pull request #23315: [SPARK-26366][SQL]

[jira] [Issue Comment Deleted] (SPARK-26246) Infer timestamp types from JSON

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26246: Comment: was deleted (was: asfgit closed pull request #23201: [SPARK-26246][SQL] Inferring

[jira] [Comment Edited] (SPARK-26246) Infer timestamp types from JSON

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16734361#comment-16734361 ] Reynold Xin edited comment on SPARK-26246 at 1/4/19 5:22 PM: - Is there an

[jira] [Commented] (SPARK-26246) Infer timestamp types from JSON

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16734361#comment-16734361 ] Reynold Xin commented on SPARK-26246: - |Is there an option flag for this? This is a breaking change

[jira] [Commented] (SPARK-26362) Remove 'spark.driver.allowMultipleContexts' to disallow multiple Spark contexts

2019-01-04 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26362?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16734355#comment-16734355 ] Reynold Xin commented on SPARK-26362: - [~hyukjin.kwon] please make sure we add releasenotes label to

[jira] [Commented] (SPARK-23693) SQL function uuid()

2018-12-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16728078#comment-16728078 ] Reynold Xin commented on SPARK-23693: - [~tashoyan] the issue with calling uuid directly is that it

[jira] [Commented] (SPARK-26222) Scan: track file listing time

2018-12-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16720957#comment-16720957 ] Reynold Xin commented on SPARK-26222: - So I spent some time looking at the code base to understand

[jira] [Issue Comment Deleted] (SPARK-26222) Scan: track file listing time

2018-12-13 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26222: Comment: was deleted (was: xuanyuanking opened a new pull request #23298: [SPARK-26222][SQL]

[jira] [Created] (SPARK-26368) Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex

2018-12-13 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26368: --- Summary: Make it clear that getOrInferFileFormatSchema doesn't create InMemoryFileIndex Key: SPARK-26368 URL: https://issues.apache.org/jira/browse/SPARK-26368

[jira] [Commented] (SPARK-26222) Scan: track file listing time

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16715663#comment-16715663 ] Reynold Xin commented on SPARK-26222: - Do we do any file listing in non-physical phase? E.g. if

[jira] [Issue Comment Deleted] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Comment: was deleted (was: User 'rxin' has created a pull request for this issue:

[jira] [Issue Comment Deleted] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Comment: was deleted (was: User 'rxin' has created a pull request for this issue:

[jira] [Resolved] (SPARK-26139) Support passing shuffle metrics to exchange operator

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-26139. - Resolution: Fixed Fix Version/s: 3.0.0 > Support passing shuffle metrics to exchange

[jira] [Issue Comment Deleted] (SPARK-26193) Implement shuffle write metrics in SQL

2018-12-10 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26193: Comment: was deleted (was: rxin commented on a change in pull request #23207: [SPARK-26193][SQL]

[jira] [Issue Comment Deleted] (SPARK-26139) Support passing shuffle metrics to exchange operator

2018-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26139?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26139: Comment: was deleted (was: User 'xuanyuanking' has created a pull request for this issue:

[jira] [Commented] (SPARK-26193) Implement shuffle write metrics in SQL

2018-12-02 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26193?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16706368#comment-16706368 ] Reynold Xin commented on SPARK-26193: - Can we simplify it and add those metrics only to the same

[jira] [Resolved] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-26226. - Resolution: Fixed Fix Version/s: 3.0.0 > Update query tracker to report timeline for

[jira] [Created] (SPARK-26241) Add queryId to IncrementalExecution

2018-11-30 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26241: --- Summary: Add queryId to IncrementalExecution Key: SPARK-26241 URL: https://issues.apache.org/jira/browse/SPARK-26241 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-26241) Add queryId to IncrementalExecution

2018-11-30 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26241: Issue Type: Sub-task (was: Bug) Parent: SPARK-26221 > Add queryId to

[jira] [Assigned] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-26226: --- Assignee: Reynold Xin > Update query tracker to report timeline for phases, rather than

[jira] [Created] (SPARK-26226) Update query tracker to report timeline for phases, rather than duration

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26226: --- Summary: Update query tracker to report timeline for phases, rather than duration Key: SPARK-26226 URL: https://issues.apache.org/jira/browse/SPARK-26226 Project:

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and

[jira] [Updated] (SPARK-26129) Instrumentation for query planning time

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26129: Issue Type: Sub-task (was: New Feature) Parent: SPARK-26221 > Instrumentation for query

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and

[jira] [Updated] (SPARK-26223) Scan: track metastore operation time

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26223?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26223: Description: The Scan node should report how much time it spent in metastore operations. Similar

[jira] [Created] (SPARK-26225) Scan: track decoding time for row-based data sources

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26225: --- Summary: Scan: track decoding time for row-based data sources Key: SPARK-26225 URL: https://issues.apache.org/jira/browse/SPARK-26225 Project: Spark Issue

[jira] [Created] (SPARK-26223) Scan: track metastore operation time

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26223: --- Summary: Scan: track metastore operation time Key: SPARK-26223 URL: https://issues.apache.org/jira/browse/SPARK-26223 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-26222) Scan: track file listing time

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26222: --- Summary: Scan: track file listing time Key: SPARK-26222 URL: https://issues.apache.org/jira/browse/SPARK-26222 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26221?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26221: Description: This is an umbrella ticket for various small improvements for better metrics and

[jira] [Created] (SPARK-26221) Improve Spark SQL instrumentation and metrics

2018-11-29 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26221: --- Summary: Improve Spark SQL instrumentation and metrics Key: SPARK-26221 URL: https://issues.apache.org/jira/browse/SPARK-26221 Project: Spark Issue Type:

[jira] [Commented] (SPARK-24498) Add JDK compiler for runtime codegen

2018-11-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24498?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16702515#comment-16702515 ] Reynold Xin commented on SPARK-24498: - Why don't we close the ticket? I heard we would get mixed

[jira] [Resolved] (SPARK-26141) Enable custom shuffle metrics implementation in shuffle write

2018-11-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-26141. - Resolution: Fixed Fix Version/s: 3.0.0 > Enable custom shuffle metrics implementation in

[jira] [Assigned] (SPARK-26142) Implement shuffle read metrics in SQL

2018-11-23 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reassigned SPARK-26142: --- Assignee: Yuanjian Li > Implement shuffle read metrics in SQL >

[jira] [Updated] (SPARK-26141) Enable custom shuffle metrics implementation in shuffle write

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26141?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26141: Summary: Enable custom shuffle metrics implementation in shuffle write (was: Enable passing in

[jira] [Created] (SPARK-26142) Implement shuffle read metrics in SQL

2018-11-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26142: --- Summary: Implement shuffle read metrics in SQL Key: SPARK-26142 URL: https://issues.apache.org/jira/browse/SPARK-26142 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-26140) Enable custom shuffle metrics reporter in shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable custom shuffle metrics reporter in shuffle reader (was: Enable custom shuffle

[jira] [Updated] (SPARK-26140) Enable custom shuffle metrics implementation in shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable custom shuffle metrics implementation in shuffle reader (was: Enable custom

[jira] [Updated] (SPARK-26140) Enable custom shuffle metrics reporter into shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable custom shuffle metrics reporter into shuffle reader (was: Enable passing in a

[jira] [Created] (SPARK-26141) Enable passing in custom shuffle metrics implementation in shuffle write

2018-11-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26141: --- Summary: Enable passing in custom shuffle metrics implementation in shuffle write Key: SPARK-26141 URL: https://issues.apache.org/jira/browse/SPARK-26141 Project:

[jira] [Resolved] (SPARK-26129) Instrumentation for query planning time

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26129?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-26129. - Resolution: Fixed Fix Version/s: 3.0.0 > Instrumentation for query planning time >

[jira] [Created] (SPARK-26140) Pull TempShuffleReadMetrics creation out of shuffle layer

2018-11-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26140: --- Summary: Pull TempShuffleReadMetrics creation out of shuffle layer Key: SPARK-26140 URL: https://issues.apache.org/jira/browse/SPARK-26140 Project: Spark

[jira] [Updated] (SPARK-26140) Enable passing in a custom shuffle metrics reporter into shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Enable passing in a custom shuffle metrics reporter into shuffle reader (was: Allow

[jira] [Updated] (SPARK-26140) Allow passing in a custom shuffle metrics reporter into shuffle reader

2018-11-21 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26140?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-26140: Summary: Allow passing in a custom shuffle metrics reporter into shuffle reader (was: Pull

[jira] [Created] (SPARK-26139) Support passing shuffle metrics to exchange operator

2018-11-21 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26139: --- Summary: Support passing shuffle metrics to exchange operator Key: SPARK-26139 URL: https://issues.apache.org/jira/browse/SPARK-26139 Project: Spark Issue

[jira] [Created] (SPARK-26129) Instrumentation for query planning time

2018-11-20 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-26129: --- Summary: Instrumentation for query planning time Key: SPARK-26129 URL: https://issues.apache.org/jira/browse/SPARK-26129 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-21559) Remove Mesos fine-grained mode

2018-11-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-21559: Target Version/s: 3.0.0 Labels: release-notes (was: ) > Remove Mesos fine-grained

[jira] [Commented] (SPARK-17967) Support for list or other types as an option for datasources

2018-11-06 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677418#comment-16677418 ] Reynold Xin commented on SPARK-17967: - BTW how important is this? Seems like for CSV people can just

[jira] [Commented] (SPARK-25841) Redesign window function rangeBetween API

2018-10-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=1559#comment-1559 ] Reynold Xin commented on SPARK-25841: - I posted api proposal sketches in 

[jira] [Updated] (SPARK-25843) Redesign rangeBetween API

2018-10-28 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25843?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25843: Description: See parent ticket for more information. Two proposals with sketches:   Proposal 1.

[jira] [Created] (SPARK-25862) Remove rangeBetween APIs introduced in SPARK-21608

2018-10-28 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25862: --- Summary: Remove rangeBetween APIs introduced in SPARK-21608 Key: SPARK-25862 URL: https://issues.apache.org/jira/browse/SPARK-25862 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

2018-10-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-23084. - Resolution: Won't Fix Fix Version/s: (was: 2.4.0) This was merged but then reverted

[jira] [Reopened] (SPARK-23084) Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark

2018-10-26 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23084?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-23084: - > Add unboundedPreceding(), unboundedFollowing() and currentRow() to PySpark >

[jira] [Created] (SPARK-25845) Fix MatchError for calendar interval type in rangeBetween

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25845: --- Summary: Fix MatchError for calendar interval type in rangeBetween Key: SPARK-25845 URL: https://issues.apache.org/jira/browse/SPARK-25845 Project: Spark

[jira] [Created] (SPARK-25844) Implement Python API once we have a new API

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25844: --- Summary: Implement Python API once we have a new API Key: SPARK-25844 URL: https://issues.apache.org/jira/browse/SPARK-25844 Project: Spark Issue Type:

[jira] [Updated] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25842: Target Version/s: 2.4.0 > Deprecate APIs introduced in SPARK-21608 >

[jira] [Created] (SPARK-25843) Redesign rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25843: --- Summary: Redesign rangeBetween API Key: SPARK-25843 URL: https://issues.apache.org/jira/browse/SPARK-25843 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-25842) Deprecate APIs introduced in SPARK-21608

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25842: --- Summary: Deprecate APIs introduced in SPARK-21608 Key: SPARK-25842 URL: https://issues.apache.org/jira/browse/SPARK-25842 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25841: Description: As I was reviewing the Spark API changes for 2.4, I found that through organic,

[jira] [Created] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25841: --- Summary: Redesign window function rangeBetween API Key: SPARK-25841 URL: https://issues.apache.org/jira/browse/SPARK-25841 Project: Spark Issue Type: Umbrella

[jira] [Updated] (SPARK-25841) Redesign window function rangeBetween API

2018-10-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25841?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-25841: Description: As I was reviewing the Spark API changes for 2.4, I found that through organic,

[jira] [Created] (SPARK-25496) Deprecate from_utc_timestamp and to_utc_timestamp

2018-09-20 Thread Reynold Xin (JIRA)
Reynold Xin created SPARK-25496: --- Summary: Deprecate from_utc_timestamp and to_utc_timestamp Key: SPARK-25496 URL: https://issues.apache.org/jira/browse/SPARK-25496 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-09-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-23715. - Resolution: Won't Fix Fix Version/s: (was: 2.4.0) > from_utc_timestamp returns

[jira] [Reopened] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-09-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin reopened SPARK-23715: - Assignee: (was: Wenchen Fan) > from_utc_timestamp returns incorrect results for some UTC

[jira] [Commented] (SPARK-23715) from_utc_timestamp returns incorrect results for some UTC date/time values

2018-09-20 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23715?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16622953#comment-16622953 ] Reynold Xin commented on SPARK-23715: - the current behavior is that it only takes a timestamp type

  1   2   3   4   5   6   7   8   9   10   >