coderfender commented on PR #3351:
URL:
https://github.com/apache/datafusion-comet/pull/3351#issuecomment-3880335958
@mbutrovich , thank you for the comment. These tests were essentially
exposing bugs with our writer which are not caught by our current fallbacks
1. Empty file/dir creation
empty file should be skipped while write to file
SPARK-33901: ctas should not change table's schema
SPARK-37160: CREATE TABLE AS SELECT with CHAR_AS_VARCHAR
2. Insert overwrite bugs (5 tests)
Insert overwrite table command should output correct schema: basic
SPARK-29174 Support LOCAL in INSERT OVERWRITE DIRECTORY
SPARK-38336 INSERT INTO with default columns
SPARK-38811 INSERT INTO on ALTER TABLE ADD COLUMNS
SPARK-43071: INSERT INTO from non-projection queries
3. Other
parquet timestamp conversion (timestamp96)
INSERT INTO TABLE - complex type but different names
Write Spark version into Parquet metadata
write path implements onTaskCommit API correctly
I created separate issues for each failing tests, annotated them in the
diffs . However, per my discussion with @comphead , this PR is just the first
step to copy those tests into comet test suite to lay foundation for a future
comet writer only test suite in CI .
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
