Hey everyone, Because of an issue we experienced, I found out that kafka message keys written by the Ingress are not hashed. They are basically a concatenation of the class and the labels. This means that all points related to a class go into the same partition / group of partitions.
Of course, this is fine in most cases. In the case where data of a specific class is significantly bigger than the rest of the data, this can be an issue (big strings versus int / floats) where some store threads will have lots of work to do and others not so much. Besides performance concerns, do you think having the key hashed (which would spread out messages of any class over all partitions but still keep the order guarantee for a given GTS) could cause problems? Maybe this could be an option? Thanks for your input! -- You received this message because you are subscribed to the Google Groups "Warp 10 users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/warp10-users/1113aa0f-a266-4b75-9735-2f61813f3ab3n%40googlegroups.com.
