EdisonZiXing edited a comment on issue #11427: URL: https://github.com/apache/arrow/issues/11427#issuecomment-945495667
@amol- Thx for replying ,master! The way 'not working' is that the same error comes when I try the settings above. Seems like the last row of my file cause the errors. The original file is 1.5GB Then I trying to make a minimal CSV file with the last row: Step1 When I use pandas.read_csv() to read the file as pandas.DataFrame ,it worked. Step 2 Then I take out the last 10 row of the DataFrame and use pandas.to_csv() got another file test_minimal.csv Step 3 Uesd pyarrow.csv.read_csv() to read the test_minimal,csv again ,then succeed! So I wonder if the pyarrow.csv.read_csv() NaN detecting problem? Looking forwards to your replies! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
