sohailshk opened a new pull request, #12259:
URL: https://github.com/apache/arrow-rs/pull/12259

   # Which issue does this PR close?
   No issue; the repro is below.
   
   # Rationale for this change
   A dictionary key that points at a null dictionary value is a logical null, 
but the JSON writer encodes it as `""`, because the dictionary encoder only 
receives the key validity buffer. For keys `[0, 1, null]` over values `["a", 
null]`, the writer outputs `[{"my_dict":"a"},{"my_dict":null},{"my_dict":""}]`. 
In the #6503 review, alamb pointed out this exact output: "will write a JSON 
string with whatever happened to be in that NULL slot (likely an empty string 
`""`)". The RunEndEncoded and Null cases in `make_encoder` already use 
`logical_nulls()`; Dictionary was the odd one out.
   
   # What changes are included in this PR?
   `make_encoder` passes the dictionary's `logical_nulls()` instead of 
`nulls()`. `test_writer_null_dict` (added in #6503 to stop a panic) recorded 
`""` for this case; it now expects `null`.
   
   # Are these changes tested?
   Yes, `test_writer_null_dict` fails on main and passes with the change.
   
   # Are there any significant user-facing changes?
   Yes, small: a dictionary key pointing at a null value is now written as null 
(or omitted without explicit nulls) instead of `""`.
   
   # AI usage disclosure
   The implementation and tests were written with AI assistance.
   
   Thanks for taking a look!
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to