Thank you for the responses, Wes and Antoine. I've sent to private@ and then filed https://issues.apache.org/jira/browse/ARROW-7435
@Antoine I'm glad to hear the IPC layer is a security boundary. Could you please clarify what that means exactly? Do you mean (please check all that apply): 1. Arrow IPC is designed to produce defined behavior given any input? 2. Arrow IPC is designed such that a nefarious Arrow file can't cause reads or writes outside of the file's memory boundaries? 3. If I notice a specially-crafted file inspires Arrow IPC to violate these first two design goals, that's a bug I should report ASAP? In brief: I'm asking whether security is a project goal or more of a pleasant-but-unsupported consequence of Arrow's design. *Should* I use Arrow IPC (with Validate()) to read files produced by untrusted (out-of-process) code? Enjoy life, Adam On Wed, Dec 18, 2019 at 12:30 PM Antoine Pitrou <anto...@python.org> wrote: > > Hi Adam, > > The validation was significantly enhanced in git master. In particular, > Array::ValidateFull() will validate data more or less extensively. I > wouldn't be surprised if not everything is validated, though. Feel free > to open JIRA tickets about missing validation checks. > > However, if you are in control of the current process, then you can > pretty much craft an invalid array in all kinds of manners. The main > security boundary is the IPC layer and other file format implementations > (CSV, Parquet...). > > Regards > > Antoine. > > > Le 18/12/2019 à 17:42, Adam Hooper a écrit : > > My project parses Arrow files produced by untrusted code. > > > > It looks to me like the "validate" function should help me avoid > undefined > > behavior given an invalid Arrow file. I found a bug in the function: even > > after validation, an invalid Arrow file can trigger undefined behavior. > > > > Is security a goal of the Arrow project/format? If so, how shall I report > > this bug without endangering other users in my situation? > > > > Enjoy life, > > Adam > > > -- Adam Hooper +1-514-882-9694 http://adamhooper.com
