n3world commented on pull request #10202: URL: https://github.com/apache/arrow/pull/10202#issuecomment-836778830
> > Is the complexity you don't like the callback function or the ability to add a field with any value and not just null? > > The callback is my main concern. I think we can add a "missing" bit to ParsedValueDescr. Hopefully that wouldn't slow down the conversion of non-missing values. If a bit is added to ParsedValueDesc would it be taken away from offset to try and keep the struct at 32bits? Also, this proposed change would only work for resolving [ARROW-12001](https://issues.apache.org/jira/browse/ARROW-12001) and not the original intent which was essentially to allow the caller the ability to determine how rows with incorrect column counts should be handled. At a minimum I would like to be able to skip the rows and collect information about the skipped rows so it could be presented to a user saying when and where malformed rows were found. The fill in nulls or truncate rows were just simple additions I added to be able to process those rows if that was desired. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
