Steven Aerts created SAMZA-762:
----------------------------------
Summary: java arrays not usable as keys anymore
Key: SAMZA-762
URL: https://issues.apache.org/jira/browse/SAMZA-762
Project: Samza
Issue Type: Bug
Components: kafka
Affects Versions: 0.9.0
Reporter: Steven Aerts
Since the integration of the new kafka producer into samza, the partitioner is
defined like
[this|https://github.com/apache/samza/blob/0.9.0/samza-kafka/src/main/scala/org/apache/samza/util/KafkaUtil.scala#L49]:
{code}
return abs(envelope.getPartitionKey.hashCode()) % numPartitions
{code}
This is problematic for java arrays, as their hash-code is independent of their
content (unlike in scala). So a specific array based key is randomly assigned
to a partition.
In 0.8 it was possible to configure your partitioner with
{{systems.kafka.producer.partitioner.class}} and select for example the default
murmur2 based partitioner of kafka.
But this is also not possible anymore.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
