Abhi2939 commented on issue #41102: URL: https://github.com/apache/beam/issues/41102#issuecomment-6763405230
Thanks @jk-shah, I'll pick this up. My plan, unless maintainers prefer otherwise: 1. Reproduce on master with the YAML example from the description, plus a plain Python `beam.io.WriteToJson` repro. 2. For `orient='records', lines=True` (the YAML default), write each row with `json.dumps` so floats round-trip exactly, instead of going through `DataFrame.to_json`. I'll check that output for NaN/None, timestamps and nested values matches what the pandas path produces today, so this doesn't change anything besides float precision. 3. For the other orients that still go through pandas, default `double_precision=15` when the caller doesn't pass it, and note the remaining limit for 16-17 significant digits in the docstring. 4. Add tests that round-trip values like `0.1234567890123` and `3.3333333333333335`, and document the `double_precision` kwarg. @jk-shah or any maintainer: if you'd rather have only the `double_precision=15` default (smaller change) or only the `json.dumps` path, let me know and I'll adjust before opening the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
