hyperevo opened a new issue, #945: URL: https://github.com/apache/pulsar-client-go/issues/945
#### Expected behavior Situation: 1. Using an avro schema definition where at least one field is of type byte. 2. Providing data, in the form of a native golang struct with a corresponding property of type []byte It is expected that the []byte property should be encoded as-is into the binary payload using the avro codec. #### Actual behavior In this situation, when encoding the native golang struct into a pulsar payload using the function here: https://github.com/apache/pulsar-client-go/blob/d9b18d0690c15d5da94c9f324d9c2618e3d60bfd/pulsar/schema.go#L253, it first encodes the []byte type into a base64 string during the json.Marshal. Then after that it is converted into binary using the avro codec. This final binary data is significantly larger in size than it would have been if it was transmitted directly as the raw bytes. Here is an example: Original golang byte array: [172 12 53 97 9 70 89 247 94 3 56 242 127 146 9 209] Base64 encoded text from byte array: rAw1YQlGWfdeAzjyf5IJ0Q== The byte array of the final encoded binary payload (which is just the byte array representation of the base64 encoded string): [114 65 119 49 89 81 108 71 87 102 100 101 65 122 106 121 102 53 73 74 48 81 61 61] In this example the encoded payload that the pulsar-client-go transmits to the pulsar queue is 50% larger in terms of bytes. This can lead to a dramatic loss to performance when throughput is the bottleneck. #### Steps to reproduce Encode a []byte object using the avro schema encode function here https://github.com/apache/pulsar-client-go/blob/d9b18d0690c15d5da94c9f324d9c2618e3d60bfd/pulsar/schema.go#L253 and look at the output. #### System configuration commit 504e589c438d35a6165f48148778b84565da8d7e -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@pulsar.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
