carloea2 opened a new issue, #8268: URL: https://github.com/apache/texera/issues/8268
### What happened? `ArrowTableTupleProvider` treats every binary value beginning with the six bytes `pickle` as Texera's pickled-object envelope. The actual envelope uses the ten-byte header `pickle` followed by four spaces. Legitimate raw bytes such as `b"picklehello"` are therefore passed to `pickle.loads` and fail instead of round-tripping unchanged. Before: a raw binary value beginning with `pickle` can raise `UnpicklingError` while being read from Arrow. Expected: only the complete Texera pickle header triggers unpickling; other binary values remain bytes. ### How to reproduce? 1. Build an Arrow binary column containing `b"picklehello"`. 2. Read it through `ArrowTableTupleProvider`. 3. Access the field and observe `_pickle.UnpicklingError`. ### Version or branch 1.3.0-incubating-SNAPSHOT on main ### Commit hash 50321e403c82df299a13deb50a7f9849dd93bdba ### Relevant log output _pickle.UnpicklingError: could not find MARK -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
