allinux opened a new issue, #34603:
URL: https://github.com/apache/arrow/issues/34603

   ### Describe the bug, including details regarding any error messages, 
version, and platform.
   
   There is a bug where '' is loaded as \x00 when WithDictionaryDefault is set 
to true when all data in the column is ''.
   If processed as false, it is normally loaded as ''.
   It seems to occur during RLE processing, but need to check.
   
   sample_parquet.zip contains src.parquet and dest.parquet files, src.parquet 
is a file with '' and dest.parquet is a file reloaded with 
WithDictionaryDefault set to true.
   
   * python code for check
   `In [1]: import pandas as pd
   
   In [2]: src = pd.read_parquet('src.parquet')
   
   In [3]: src
   Out[3]: 
     reserved1
   0          
   1          
   2          
   3          
   4          
   5          
   
   In [4]: src.iloc[0].to_dict()
   Out[4]: {'reserved1': ''}
   
   In [5]: dest = pd.read_parquet('dest.parquet')
   
   In [6]: dest
   Out[6]: 
     reserved1
   0         
   1         
   2         
   3         
   4         
   5         
   
   In [7]: dest.iloc[0].to_dict()
   Out[7]: {'reserved1': '\x00'}`
   
[sample_parquet.zip](https://github.com/apache/arrow/files/11000320/sample_parquet.zip)
   
   
   ### Component(s)
   
   Go


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to