dtenedor commented on PR #44939:
URL: https://github.com/apache/spark/pull/44939#issuecomment-1919526449
> @dtenedor Could you write or modify your new test to check V2 DS
implementation. I put a breakpoint in CSVPartitionReaderFactory.
And your test didn't hit the breakpoint.
@MaxGekk Good question, reproducing it now, requesting to run this new
`test("SPARK-46890: CSV fails on a column with default and without enforcing
schema")` in IntelliJ brings up a drop-down for `CSVv1Suite` or `CSVv2Suite`. I
am able to hit that breakpoint on the latter but not the former.
<img width="851" alt="image"
src="https://github.com/apache/spark/assets/99207096/d57c6901-db41-4740-ab75-0d83dcf877f9";>
<img width="934" alt="image"
src="https://github.com/apache/spark/assets/99207096/dd0b118c-c142-49cb-a74c-c20920ac6e88";>
Interestingly, I copied the unit test to a new PR and it fails for both CSV
V1 and CSV V2. So this PR fixes it for both versions, but only V2 hits the
breakpoint you suggested.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
