Stefanietry opened a new issue, #9582:
URL: https://github.com/apache/paimon/issues/9582

   ### Search before asking
   
   - [x] I searched in the [issues](https://github.com/apache/paimon/issues) 
and found nothing similar.
   
   
   ### Motivation
   
   blob-descriptor-field stores blob values as serialized BlobDescriptor bytes 
inline in data files. In Spark, reading such fields as normal binary data 
should return the actual blob bytes, while blob-as-descriptor=true should 
return  descriptor bytes.                                                       
                                                                                
                                                                               
                                                                                
                                                                                
                                                                            
   Currently, spark may fail to resolve the actual blob data because the read 
path does not always attach the required descriptor reader to the reconstructed 
blob value. 
   
   ### Solution
   
   Pass a table-level UriReaderFactory through the spark scan/read path, and 
wrap read rows with a descriptor-resolving row when blob-as-descriptor=false.   
                                                                             
                                                                                
                                                                                
                                                                            
   The wrapper reattaches the proper reader to BlobRef values, so spark can 
read actual blob bytes from blob-descriptor-field. When 
blob-as-descriptor=true, spark keeps returning serialized descriptor bytes.     
                      
   
   
   ### Anything else?
   
   _No response_
   
   ### Are you willing to submit a PR?
   
   - [ ] I'm willing to submit a PR!


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to