mxtymoshyk commented on issue #26918: URL: https://github.com/apache/beam/issues/26918#issuecomment-6445520224
I couldn't reproduce this with Java StringUtf8Coder 2.77.0 and Python StrUtf8Coder 2.77.0. Both give identical bytes in both coder contexts, and each decodes the other's output (tested with "abc", "ab\0c", "ÿ" and "光线"). The Java coder writes a length prefix only in the nested context, for example when the string sits inside a KV or a list. In the outer context it writes just the UTF-8 bytes. Python does the same. For "abc": - outer: 61 62 63 - nested: 03 61 62 63 If you decode the nested bytes with Python's plain decode(), which expects the outer context, you get '\x03abc'. That looks like what the report describes, and it comes from using the wrong context. It isn't a difference between the SDKs. This also matches chamikaramj's comment above and the spec in standard_coders.yaml. If you still see a failure, please share the pipeline or the exact bytes and how you decode them, and we can look again. Otherwise I think this can be closed as working as intended. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
