Jakob-al28 opened a new pull request, #2580: URL: https://github.com/apache/systemds/pull/2580
The Spark round trip tests cover multi-file layouts, deletion vectors, partitioning, column mapping and schema evolution. This adds three that were missing: a log that has been checkpointed, a column type SystemDS cannot map, and a table more than one engine has written to. | test | covers | | --- | --- | | `checkpointBackedHistoryRead` | a log that rolled into a parquet checkpoint, with a delete after it, so the snapshot comes from a checkpoint plus trailing commits | | `unsupportedColumnTypeRead` | DATE / TIMESTAMP / DECIMAL columns, which have no SystemDS value type and must be rejected by name before decoding | | `systemdsCreateSparkAppendRead` | SystemDS creates the table, Spark appends to it, so a mixed engine commit history still reads back as one table | -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
