Hi Taylor,
After writing my own producer with a custom round-robin partitioner, it seems
the problem is solved now. Here’s my topology:
BrokerHosts brokerHosts = new ZkHosts(_kafkaZkConnect);
TridentKafkaConfig kafkaConfig = new TridentKafkaConfig(brokerHosts,
_topic);
Stream inputStream = _topology.newStream("bytes", new
TransactionalTridentKafkaSpout(kafkaConfig))
.parallelismHint(_numKafkaPartitions).name("reading-kafka")
.each(new Fields("bytes"), new Reporter(graphiteConfig,
"tuples", "received")).name("reporting-to-graphite");
Best regards,Huy, Le Van
INSIGHT@ NUI Galway
Lower Dangan, IDA Business Park,Galway, Ireland
On Wednesday, Dec 10, 2014 at 6:52 p.m., P. Taylor Goetz <[email protected]>,
wrote:
Can you share your topology code? It sound like you may not be setting the
parallelism hint in the right place.
-Taylor
On Dec 4, 2014, at 3:32 PM, Huy Le Van <[email protected]> wrote:
Could someone help me please?
Best regards,Huy, Le Van
On Thursday, Dec 4, 2014 at 3:35 p.m., Huy Le Van
<[email protected]>, wrote:
Hi,
I’m trying to tune Kafka Trident (Transactional) and seeing that the ‘spout0’
bolt uses only one executor. The problem is exactly as described in
https://groups.google.com/forum/#!msg/storm-user/bI7976v9R5g/fulzpnPmzkEJ
However, my Kafka topic has 16 partitions and I already set parallelismHint of
TransactionalTridentKafkaSpout to 16. What am I doing wrong here? Please advise.
Many thanks,Huy, Le Van
signature.asc
Description: PGP signature
