bvaradar commented on issue #2066: URL: https://github.com/apache/hudi/issues/2066#issuecomment-709436915
THis looks fine to me. Just wondering instead of default implementation of ComplexKeyGenerator which concatenates all the fields, can you try implementing a key-generator which generates md5/sha checkums to study the behavior. Assuming the original length of the record key is large and there is pattern to exploit compression, the checksum approach could perform better. Also, see if you can try gzip as opposed to snappy. ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected]
