[jira] [Resolved] (SPARK-24778) DateTimeUtils.getTimeZone method returns GMT time if timezone cannot be parsed

2019-03-03 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-24778. Resolution: Fixed Fix Version/s: 3.0.0 The issue has been fixed already by using ZoneId.of

[jira] [Commented] (SPARK-26016) Encoding not working when using a map / mapPartitions call

2019-03-03 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16782778#comment-16782778 ] Maxim Gekk commented on SPARK-26016: > nothing reinterprets the bytes according to a

[jira] [Created] (SPARK-27057) Common trait for limit exec operators

2019-03-05 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27057: -- Summary: Common trait for limit exec operators Key: SPARK-27057 URL: https://issues.apache.org/jira/browse/SPARK-27057 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-27109) Refactoring of TimestampFormatter and DateFormatter

2019-03-08 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27109: -- Summary: Refactoring of TimestampFormatter and DateFormatter Key: SPARK-27109 URL: https://issues.apache.org/jira/browse/SPARK-27109 Project: Spark Issue Type: I

[jira] [Created] (SPARK-27199) Replace TimeZone by ZoneId in TimestampFormatter API

2019-03-19 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27199: -- Summary: Replace TimeZone by ZoneId in TimestampFormatter API Key: SPARK-27199 URL: https://issues.apache.org/jira/browse/SPARK-27199 Project: Spark Issue Type:

[jira] [Created] (SPARK-27212) Eliminate TimeZone to ZoneId conversion in stringToTimestamp

2019-03-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27212: -- Summary: Eliminate TimeZone to ZoneId conversion in stringToTimestamp Key: SPARK-27212 URL: https://issues.apache.org/jira/browse/SPARK-27212 Project: Spark Iss

[jira] [Created] (SPARK-27222) Support Instant and LocalDate in Literal.apply

2019-03-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27222: -- Summary: Support Instant and LocalDate in Literal.apply Key: SPARK-27222 URL: https://issues.apache.org/jira/browse/SPARK-27222 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-27242) Avoid using default time zone in formatting TIMESTAMP/DATE literals

2019-03-22 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27242: -- Summary: Avoid using default time zone in formatting TIMESTAMP/DATE literals Key: SPARK-27242 URL: https://issues.apache.org/jira/browse/SPARK-27242 Project: Spark

[jira] [Created] (SPARK-27252) Make current_date() independent from time zones

2019-03-22 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27252: -- Summary: Make current_date() independent from time zones Key: SPARK-27252 URL: https://issues.apache.org/jira/browse/SPARK-27252 Project: Spark Issue Type: Impro

[jira] [Commented] (SPARK-26325) Interpret timestamp fields in Spark while reading json (timestampFormat)

2019-03-25 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16801011#comment-16801011 ] Maxim Gekk commented on SPARK-26325: Can you try Z instead of 'Z'? > Interpret time

[jira] [Created] (SPARK-27325) Support implicit encoders for LocalDate and Instant

2019-03-30 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27325: -- Summary: Support implicit encoders for LocalDate and Instant Key: SPARK-27325 URL: https://issues.apache.org/jira/browse/SPARK-27325 Project: Spark Issue Type: N

[jira] [Created] (SPARK-27327) New JSON benchmarks: functions, dataset parsing

2019-03-30 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27327: -- Summary: New JSON benchmarks: functions, dataset parsing Key: SPARK-27327 URL: https://issues.apache.org/jira/browse/SPARK-27327 Project: Spark Issue Type: Impro

[jira] [Created] (SPARK-27344) Support the LocalDate and Instant classes in Java Bean encoders

2019-04-01 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27344: -- Summary: Support the LocalDate and Instant classes in Java Bean encoders Key: SPARK-27344 URL: https://issues.apache.org/jira/browse/SPARK-27344 Project: Spark

[jira] [Created] (SPARK-27357) Convert timestamps to/from dates independently from time zones

2019-04-03 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27357: -- Summary: Convert timestamps to/from dates independently from time zones Key: SPARK-27357 URL: https://issues.apache.org/jira/browse/SPARK-27357 Project: Spark I

[jira] [Updated] (SPARK-27357) Cast timestamps to/from dates independently from time zones

2019-04-03 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27357: --- Summary: Cast timestamps to/from dates independently from time zones (was: Convert timestamps to/fr

[jira] [Created] (SPARK-27398) Get rid of sun.nio.cs.StreamDecoder in CreateJacksonParser

2019-04-06 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27398: -- Summary: Get rid of sun.nio.cs.StreamDecoder in CreateJacksonParser Key: SPARK-27398 URL: https://issues.apache.org/jira/browse/SPARK-27398 Project: Spark Issue

[jira] [Created] (SPARK-27401) Refactoring conversion of Date/Timestamp to/from java.sql.Date/Timestamp

2019-04-06 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27401: -- Summary: Refactoring conversion of Date/Timestamp to/from java.sql.Date/Timestamp Key: SPARK-27401 URL: https://issues.apache.org/jira/browse/SPARK-27401 Project: Spark

[jira] [Created] (SPARK-27405) Restrict the range of generated random timestamps

2019-04-07 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27405: -- Summary: Restrict the range of generated random timestamps Key: SPARK-27405 URL: https://issues.apache.org/jira/browse/SPARK-27405 Project: Spark Issue Type: Imp

[jira] [Created] (SPARK-27422) CurrentDate should return local date

2019-04-09 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27422: -- Summary: CurrentDate should return local date Key: SPARK-27422 URL: https://issues.apache.org/jira/browse/SPARK-27422 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-27357) Cast timestamps to/from dates independently from time zones

2019-04-09 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk resolved SPARK-27357. Resolution: Not A Problem > Cast timestamps to/from dates independently from time zones >

[jira] [Created] (SPARK-27423) Cast DATE to/from TIMESTAMP according to SQL standard

2019-04-09 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27423: -- Summary: Cast DATE to/from TIMESTAMP according to SQL standard Key: SPARK-27423 URL: https://issues.apache.org/jira/browse/SPARK-27423 Project: Spark Issue Type:

[jira] [Created] (SPARK-27438) Increase precision of to_timestamp

2019-04-10 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27438: -- Summary: Increase precision of to_timestamp Key: SPARK-27438 URL: https://issues.apache.org/jira/browse/SPARK-27438 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-27522) Test migration from INT96 to TIMESTAMP_MICROS in parquet

2019-04-19 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27522: -- Summary: Test migration from INT96 to TIMESTAMP_MICROS in parquet Key: SPARK-27522 URL: https://issues.apache.org/jira/browse/SPARK-27522 Project: Spark Issue Ty

[jira] [Created] (SPARK-27527) Improve description of Timestamp and Date types

2019-04-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27527: -- Summary: Improve description of Timestamp and Date types Key: SPARK-27527 URL: https://issues.apache.org/jira/browse/SPARK-27527 Project: Spark Issue Type: Docum

[jira] [Created] (SPARK-27528) Use Parquet logical type TIMESTAMP_MICROS by default

2019-04-20 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27528: -- Summary: Use Parquet logical type TIMESTAMP_MICROS by default Key: SPARK-27528 URL: https://issues.apache.org/jira/browse/SPARK-27528 Project: Spark Issue Type:

[jira] [Updated] (SPARK-27533) Date/timestamps CSV benchmarks

2019-04-21 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27533: --- Summary: Date/timestamps CSV benchmarks (was: CSV benchmarks date/timestamp ops ) > Date/timestamp

[jira] [Created] (SPARK-27533) CSV benchmarks date/timestamp ops

2019-04-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27533: -- Summary: CSV benchmarks date/timestamp ops Key: SPARK-27533 URL: https://issues.apache.org/jira/browse/SPARK-27533 Project: Spark Issue Type: Test Com

[jira] [Updated] (SPARK-27533) Date and timestamp CSV benchmarks

2019-04-21 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-27533: --- Summary: Date and timestamp CSV benchmarks (was: Date/timestamps CSV benchmarks) > Date and timest

[jira] [Created] (SPARK-27535) Date and timestamp JSON benchmarks

2019-04-21 Thread Maxim Gekk (JIRA)
Maxim Gekk created SPARK-27535: -- Summary: Date and timestamp JSON benchmarks Key: SPARK-27535 URL: https://issues.apache.org/jira/browse/SPARK-27535 Project: Spark Issue Type: Test Com

[jira] [Commented] (SPARK-27450) Timestamp cast fails when the ISO8601 string omits minutes, seconds or milliseconds

2019-05-04 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27450?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833132#comment-16833132 ] Maxim Gekk commented on SPARK-27450: The cast functions supports limited number of t

[jira] [Commented] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833949#comment-16833949 ] Maxim Gekk commented on SPARK-27638: [~srowen] The date literal should be casted to

[jira] [Comment Edited] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833949#comment-16833949 ] Maxim Gekk edited comment on SPARK-27638 at 5/6/19 3:57 PM:

[jira] [Commented] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833957#comment-16833957 ] Maxim Gekk commented on SPARK-27638: It works with explicit to_date: {code:scala} sc

[jira] [Comment Edited] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16833957#comment-16833957 ] Maxim Gekk edited comment on SPARK-27638 at 5/6/19 4:10 PM:

[jira] [Commented] (SPARK-27638) date format yyyy-M-dd string comparison not handled properly

2019-05-06 Thread Maxim Gekk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-27638?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16834214#comment-16834214 ] Maxim Gekk commented on SPARK-27638: [~pengbo] Are you going to propose a PR for tha

[jira] [Created] (SPARK-34138) Keep dependants cached while refreshing v1 tables

2021-01-16 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34138: -- Summary: Keep dependants cached while refreshing v1 tables Key: SPARK-34138 URL: https://issues.apache.org/jira/browse/SPARK-34138 Project: Spark Issue Type: Sub

[jira] [Created] (SPARK-34143) Adding partitions to fully partitioned v2 table

2021-01-17 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34143: -- Summary: Adding partitions to fully partitioned v2 table Key: SPARK-34143 URL: https://issues.apache.org/jira/browse/SPARK-34143 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-34149) DSv2: `ALTER TABLE .. ADD PARTITION` does not refresh table cache

2021-01-17 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34149: -- Summary: DSv2: `ALTER TABLE .. ADD PARTITION` does not refresh table cache Key: SPARK-34149 URL: https://issues.apache.org/jira/browse/SPARK-34149 Project: Spark

[jira] [Updated] (SPARK-34149) DSv2: `ALTER TABLE .. ADD PARTITION` does not refresh table cache

2021-01-17 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34149?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34149: --- Description: For example, the test below: {code:scala} test("SPARK-X: refresh cache in partiti

[jira] [Created] (SPARK-34153) Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

2021-01-18 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34153: -- Summary: Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()` Key: SPARK-34153 URL: https://issues.apache.org/jira/browse/SPARK-34153 Project: Spark

[jira] [Created] (SPARK-34161) Check re-caching of v2 table dependents after table altering

2021-01-19 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34161: -- Summary: Check re-caching of v2 table dependents after table altering Key: SPARK-34161 URL: https://issues.apache.org/jira/browse/SPARK-34161 Project: Spark Iss

[jira] [Updated] (SPARK-34161) Check re-caching of v2 table dependents after table altering

2021-01-19 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34161?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34161: --- Description: Add tests to unified test suites and check that dependants of v2 table are still cached

[jira] [Created] (SPARK-34197) refreshTable() should not invalidate the relation cache for temporary views

2021-01-21 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34197: -- Summary: refreshTable() should not invalidate the relation cache for temporary views Key: SPARK-34197 URL: https://issues.apache.org/jira/browse/SPARK-34197 Project: Spar

[jira] [Created] (SPARK-34207) Rename `isTemporaryTable` to `isTempView` in `SessionCatalog`

2021-01-22 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34207: -- Summary: Rename `isTemporaryTable` to `isTempView` in `SessionCatalog` Key: SPARK-34207 URL: https://issues.apache.org/jira/browse/SPARK-34207 Project: Spark Is

[jira] [Updated] (SPARK-34207) Rename `isTemporaryTable` to `isTempView` in `SessionCatalog`

2021-01-22 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34207: --- Priority: Trivial (was: Major) > Rename `isTemporaryTable` to `isTempView` in `SessionCatalog` > --

[jira] [Commented] (SPARK-34213) LOAD DATA doesn't refresh v1 table cache

2021-01-23 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17270722#comment-17270722 ] Maxim Gekk commented on SPARK-34213: I am working on the issue. > LOAD DATA doesn't

[jira] [Created] (SPARK-34213) LOAD DATA doesn't refresh v1 table cache

2021-01-23 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34213: -- Summary: LOAD DATA doesn't refresh v1 table cache Key: SPARK-34213 URL: https://issues.apache.org/jira/browse/SPARK-34213 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-34215) Keep table cached after truncation

2021-01-24 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34215: -- Summary: Keep table cached after truncation Key: SPARK-34215 URL: https://issues.apache.org/jira/browse/SPARK-34215 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-34251) TRUNCATE TABLE resets stats for non-empty v1 table

2021-01-26 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34251?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17272231#comment-17272231 ] Maxim Gekk commented on SPARK-34251: I am working on bug fix. > TRUNCATE TABLE rese

[jira] [Created] (SPARK-34251) TRUNCATE TABLE resets stats for non-empty v1 table

2021-01-26 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34251: -- Summary: TRUNCATE TABLE resets stats for non-empty v1 table Key: SPARK-34251 URL: https://issues.apache.org/jira/browse/SPARK-34251 Project: Spark Issue Type: Bu

[jira] [Created] (SPARK-34262) ALTER TABLE .. SET LOCATION doesn't refresh v1 table cache

2021-01-27 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34262: -- Summary: ALTER TABLE .. SET LOCATION doesn't refresh v1 table cache Key: SPARK-34262 URL: https://issues.apache.org/jira/browse/SPARK-34262 Project: Spark Issue

[jira] [Updated] (SPARK-34262) ALTER TABLE .. SET LOCATION doesn't refresh v1 table cache

2021-01-27 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34262: --- Description: The example below portraits the issue: 1. Create a source table: {code:sql} spark-sql>

[jira] [Created] (SPARK-34266) Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

2021-01-27 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34266: -- Summary: Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()` Key: SPARK-34266 URL: https://issues.apache.org/jira/browse/SPARK-34266 Proj

[jira] [Created] (SPARK-34267) Remove `refreshTable()` from `SessionState`

2021-01-27 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34267: -- Summary: Remove `refreshTable()` from `SessionState` Key: SPARK-34267 URL: https://issues.apache.org/jira/browse/SPARK-34267 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-34267) Remove `refreshTable()` from `SessionState`

2021-01-27 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34267?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34267: --- Description: Spark already has `SessionCatalog.refreshTable` and `CatalogImpl.refreshTable`. One mor

[jira] [Created] (SPARK-34282) Unify v1 and v2 TRUNCATE TABLE tests

2021-01-28 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34282: -- Summary: Unify v1 and v2 TRUNCATE TABLE tests Key: SPARK-34282 URL: https://issues.apache.org/jira/browse/SPARK-34282 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-34282) Unify v1 and v2 TRUNCATE TABLE tests

2021-01-28 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34282: --- Description: Extract TRUNCATE TABLE tests to the common place to run them for V1 and v2 datasources.

[jira] [Created] (SPARK-34290) Support v2 TRUNCATE TABLE

2021-01-29 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34290: -- Summary: Support v2 TRUNCATE TABLE Key: SPARK-34290 URL: https://issues.apache.org/jira/browse/SPARK-34290 Project: Spark Issue Type: Sub-task Componen

[jira] [Commented] (SPARK-34290) Support v2 TRUNCATE TABLE

2021-01-29 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17274247#comment-17274247 ] Maxim Gekk commented on SPARK-34290: I am working on this. > Support v2 TRUNCATE TA

[jira] [Created] (SPARK-34301) Use logical plan of alter table in `CatalogImpl.recoverPartitions()`

2021-01-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34301: -- Summary: Use logical plan of alter table in `CatalogImpl.recoverPartitions()` Key: SPARK-34301 URL: https://issues.apache.org/jira/browse/SPARK-34301 Project: Spark

[jira] [Created] (SPARK-34302) Migrate ALTER TABLE .. CHANGE COLUMN to new resolution framework

2021-01-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34302: -- Summary: Migrate ALTER TABLE .. CHANGE COLUMN to new resolution framework Key: SPARK-34302 URL: https://issues.apache.org/jira/browse/SPARK-34302 Project: Spark

[jira] [Updated] (SPARK-34302) Migrate ALTER TABLE .. CHANGE COLUMN to new resolution framework

2021-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34302: --- Description: # Create the Command logical node for ALTER TABLE .. CHANGE COLUMN # Remove AlterTableA

[jira] [Created] (SPARK-34303) Migrate ALTER TABLE ... SET LOCATION to new resolution framework

2021-01-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34303: -- Summary: Migrate ALTER TABLE ... SET LOCATION to new resolution framework Key: SPARK-34303 URL: https://issues.apache.org/jira/browse/SPARK-34303 Project: Spark

[jira] [Updated] (SPARK-34303) Migrate ALTER TABLE ... SET LOCATION to new resolution framework

2021-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34303: --- Description: # Create the Command logical node for ALTER TABLE ... SET LOCATION # Remove AlterTableS

[jira] [Created] (SPARK-34304) Remove view checks in v1 alter table commands

2021-01-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34304: -- Summary: Remove view checks in v1 alter table commands Key: SPARK-34304 URL: https://issues.apache.org/jira/browse/SPARK-34304 Project: Spark Issue Type: Sub-tas

[jira] [Created] (SPARK-34305) Unify v1 and v2 ALTER TABLE .. SET SERDE tests

2021-01-31 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34305: -- Summary: Unify v1 and v2 ALTER TABLE .. SET SERDE tests Key: SPARK-34305 URL: https://issues.apache.org/jira/browse/SPARK-34305 Project: Spark Issue Type: Sub-ta

[jira] [Updated] (SPARK-34305) Unify v1 and v2 ALTER TABLE .. SET SERDE tests

2021-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34305: --- Description: Extract ALTER TABLE .. SET SERDE tests to the common place to run them for V1 and v2 da

[jira] [Updated] (SPARK-34302) Migrate ALTER TABLE .. CHANGE COLUMN to new resolution framework

2021-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34302: --- Description: # Create the Command logical node for ALTER TABLE .. CHANGE COLUMN # Remove AlterTableA

[jira] [Updated] (SPARK-34303) Migrate ALTER TABLE ... SET LOCATION to new resolution framework

2021-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34303?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34303: --- Description: # Create the Command logical node for ALTER TABLE ... SET LOCATION # Remove AlterTableS

[jira] [Commented] (SPARK-29391) Default year-month units

2021-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29391?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275933#comment-17275933 ] Maxim Gekk commented on SPARK-29391: For now, I am not sure that we are still going

[jira] [Commented] (SPARK-34161) Check re-caching of v2 table dependents after table altering

2021-01-31 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34161?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17275958#comment-17275958 ] Maxim Gekk commented on SPARK-34161: This was resolved by https://github.com/apache/

[jira] [Created] (SPARK-34312) Support partition truncation by `SupportsPartitionManagement`

2021-02-01 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34312: -- Summary: Support partition truncation by `SupportsPartitionManagement` Key: SPARK-34312 URL: https://issues.apache.org/jira/browse/SPARK-34312 Project: Spark Is

[jira] [Updated] (SPARK-34312) Support partition truncation by `SupportsPartitionManagement`

2021-02-01 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34312: --- Description: Add new method `truncatePartition` in `SupportsPartitionManagement` and `truncatePartit

[jira] [Created] (SPARK-34314) Wrong discovered partition value

2021-02-01 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34314: -- Summary: Wrong discovered partition value Key: SPARK-34314 URL: https://issues.apache.org/jira/browse/SPARK-34314 Project: Spark Issue Type: Bug Compon

[jira] [Updated] (SPARK-34314) Wrong discovered partition value

2021-02-01 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34314?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34314: --- Affects Version/s: 3.1.0 3.0.2 2.4.8 > Wrong discovere

[jira] [Created] (SPARK-34332) Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

2021-02-02 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34332: -- Summary: Unify v1 and v2 ALTER TABLE .. SET LOCATION tests Key: SPARK-34332 URL: https://issues.apache.org/jira/browse/SPARK-34332 Project: Spark Issue Type: Sub

[jira] [Updated] (SPARK-34332) Unify v1 and v2 ALTER TABLE .. SET LOCATION tests

2021-02-02 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34332?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34332: --- Description: Extract ALTER TABLE .. SET LOCATION tests to the common place to run them for V1 and v2

[jira] [Created] (SPARK-34360) Support table truncation by v2 Table Catalogs

2021-02-04 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34360: -- Summary: Support table truncation by v2 Table Catalogs Key: SPARK-34360 URL: https://issues.apache.org/jira/browse/SPARK-34360 Project: Spark Issue Type: Sub-tas

[jira] [Updated] (SPARK-34360) Support table truncation by v2 Table Catalogs

2021-02-04 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34360: --- Description: Add new method `truncateTable` to the TableCatalog interface with default implementatio

[jira] [Created] (SPARK-34371) Run datetime rebasing tests for parquet DSv1 and DSv2

2021-02-04 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34371: -- Summary: Run datetime rebasing tests for parquet DSv1 and DSv2 Key: SPARK-34371 URL: https://issues.apache.org/jira/browse/SPARK-34371 Project: Spark Issue Type:

[jira] [Created] (SPARK-34377) Support parquet datasource options to control datetime rebasing in read

2021-02-05 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34377: -- Summary: Support parquet datasource options to control datetime rebasing in read Key: SPARK-34377 URL: https://issues.apache.org/jira/browse/SPARK-34377 Project: Spark

[jira] [Created] (SPARK-34385) Unwrap SparkUpgradeException in v1 Parquet datasource

2021-02-06 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34385: -- Summary: Unwrap SparkUpgradeException in v1 Parquet datasource Key: SPARK-34385 URL: https://issues.apache.org/jira/browse/SPARK-34385 Project: Spark Issue Type:

[jira] [Updated] (SPARK-34385) Unwrap SparkUpgradeException in v2 Parquet datasource

2021-02-06 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34385: --- Summary: Unwrap SparkUpgradeException in v2 Parquet datasource (was: Unwrap SparkUpgradeException i

[jira] [Commented] (SPARK-34386) "Proleptic" date off by 10 days when returned by .collectAsList

2021-02-06 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280268#comment-17280268 ] Maxim Gekk commented on SPARK-34386: [~bysza] Thanks for the ping. This is expected

[jira] [Commented] (SPARK-34386) "Proleptic" date off by 10 days when returned by .collectAsList

2021-02-06 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17280269#comment-17280269 ] Maxim Gekk commented on SPARK-34386: [~bysza] You can find more details in the blog

[jira] [Created] (SPARK-34397) Support v2 `MSCK REPAIR TABLE`

2021-02-07 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34397: -- Summary: Support v2 `MSCK REPAIR TABLE` Key: SPARK-34397 URL: https://issues.apache.org/jira/browse/SPARK-34397 Project: Spark Issue Type: Sub-task Com

[jira] [Created] (SPARK-34401) Update public docs about altering cached tables/views

2021-02-08 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34401: -- Summary: Update public docs about altering cached tables/views Key: SPARK-34401 URL: https://issues.apache.org/jira/browse/SPARK-34401 Project: Spark Issue Type:

[jira] [Created] (SPARK-34404) Support Avro datasource options to control datetime rebasing in read

2021-02-08 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34404: -- Summary: Support Avro datasource options to control datetime rebasing in read Key: SPARK-34404 URL: https://issues.apache.org/jira/browse/SPARK-34404 Project: Spark

[jira] [Updated] (SPARK-34404) Support Avro datasource options to control datetime rebasing in read

2021-02-08 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34404: --- Description: Add new Avro option similar to the SQL configs  {{spark.sql.legacy.parquet.datetimeRebas

[jira] [Commented] (SPARK-34392) Invalid ID for offset-based ZoneId since Spark 3.0

2021-02-09 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281838#comment-17281838 ] Maxim Gekk commented on SPARK-34392: The "GMT+8:00" string is unsupported format in

[jira] [Comment Edited] (SPARK-34392) Invalid ID for offset-based ZoneId since Spark 3.0

2021-02-09 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34392?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17281838#comment-17281838 ] Maxim Gekk edited comment on SPARK-34392 at 2/9/21, 3:26 PM: -

[jira] [Created] (SPARK-34418) Check v1 TRUNCATE TABLE preserves partitions

2021-02-10 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34418: -- Summary: Check v1 TRUNCATE TABLE preserves partitions Key: SPARK-34418 URL: https://issues.apache.org/jira/browse/SPARK-34418 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-34424) HiveOrcHadoopFsRelationSuite fails with seed 610710213676

2021-02-12 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34424: -- Summary: HiveOrcHadoopFsRelationSuite fails with seed 610710213676 Key: SPARK-34424 URL: https://issues.apache.org/jira/browse/SPARK-34424 Project: Spark Issue T

[jira] [Created] (SPARK-34431) Only load hive-site.xml once

2021-02-13 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34431: -- Summary: Only load hive-site.xml once Key: SPARK-34431 URL: https://issues.apache.org/jira/browse/SPARK-34431 Project: Spark Issue Type: Improvement Co

[jira] [Created] (SPARK-34434) Mention DS rebase options in SparkUpgradeException

2021-02-14 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34434: -- Summary: Mention DS rebase options in SparkUpgradeException Key: SPARK-34434 URL: https://issues.apache.org/jira/browse/SPARK-34434 Project: Spark Issue Type: I

[jira] [Created] (SPARK-34437) Update Spark SQL guide about rebase DS options and SQL configs

2021-02-14 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34437: -- Summary: Update Spark SQL guide about rebase DS options and SQL configs Key: SPARK-34437 URL: https://issues.apache.org/jira/browse/SPARK-34437 Project: Spark I

[jira] [Created] (SPARK-34439) Recognize `spark_catalog` in new identifier while view/table renaming

2021-02-15 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34439: -- Summary: Recognize `spark_catalog` in new identifier while view/table renaming Key: SPARK-34439 URL: https://issues.apache.org/jira/browse/SPARK-34439 Project: Spark

[jira] [Created] (SPARK-34440) Allow saving/loading datetime in ORC w/o rebasing

2021-02-15 Thread Maxim Gekk (Jira)
Maxim Gekk created SPARK-34440: -- Summary: Allow saving/loading datetime in ORC w/o rebasing Key: SPARK-34440 URL: https://issues.apache.org/jira/browse/SPARK-34440 Project: Spark Issue Type: New

[jira] [Updated] (SPARK-34440) Allow saving/loading datetime in ORC w/o rebasing

2021-02-15 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34440: --- Fix Version/s: (was: 3.1.0) 3.2.0 > Allow saving/loading datetime in ORC w/o

[jira] [Updated] (SPARK-34440) Allow saving/loading datetime in ORC w/o rebasing

2021-02-15 Thread Maxim Gekk (Jira)
[ https://issues.apache.org/jira/browse/SPARK-34440?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maxim Gekk updated SPARK-34440: --- Description: Currently, Spark always performs rebasing of date/timestamp columns in ORC datasource b

  1   2   3   4   5   6   7   8   9   10   >