Re: Produce 1 million events/seconds

2014-12-21 Thread Pramod Deshmukh
4 at 10:54 PM, Pramod Deshmukh wrote: > I have a requirement to prove kafka producer can produce 1 million > events/second to Kafka cluster. > > So far, best I could achieve is 200k events/sec on topic with 2 > partitions. The latency increases with adding more partitions s

Produce 1 million events/seconds

2014-12-21 Thread Pramod Deshmukh
I have a requirement to prove kafka producer can produce 1 million events/second to Kafka cluster. So far, best I could achieve is 200k events/sec on topic with 2 partitions. The latency increases with adding more partitions so I want to test with 2 partitions for now. Below are the details along

Re: [DISCUSS] Kafka Security Specific Features

2014-07-22 Thread Pramod Deshmukh
kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66) at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44) On Fri, Jul 18, 2014 at 1:20 PM, Pramod Deshmukh wrote: > Thanks Joe, I don't see any Out of memory error. Now I get except

Re: [DISCUSS] Kafka Security Specific Features

2014-07-18 Thread Pramod Deshmukh
t; http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ********/ > > > On Fri, Jul 18, 2014 at 10:24 AM, Pramod Deshmukh > wrote: > > > Hello Raja/Joe, > > When I turn on securit

Re: [DISCUSS] Kafka Security Specific Features

2014-07-18 Thread Pramod Deshmukh
) java.lang.OutOfMemoryError: Java heap space *broker log* INFO begin ssl handshake for localhost/127.0.0.1:50199//127.0.0.1:9092 On Thu, Jul 17, 2014 at 6:07 PM, Pramod Deshmukh wrote: > Correct, I don't see any exceptions when i turn off security. Consumer is > able to consume the message.

Re: [DISCUSS] Kafka Security Specific Features

2014-07-17 Thread Pramod Deshmukh
lango wrote: > Can you try with turning off security to check if this error happens only > on secure mode? > > Thanks, > Raja. > > > > > On Thu, Jul 17, 2014 at 3:51 PM, Pramod Deshmukh > wrote: > > > Thanks Raja, it was helpful > > > > Now I am

Re: [DISCUSS] Kafka Security Specific Features

2014-07-17 Thread Pramod Deshmukh
presented secure kafka configuration and usage at last meet up. So hope > this > video recording <http://www.ustream.tv/recorded/48396701>would help. You > can skip to about 59 min to jump to security talk. > > Thanks, > Raja. > > > On Wed, Jul 16, 2014 at 5:57 PM,

Re: [DISCUSS] Kafka Security Specific Features

2014-07-16 Thread Pramod Deshmukh
Hello Joe, Is there a configuration or example to test Kafka security piece? Thanks, Pramod On Wed, Jul 16, 2014 at 5:20 PM, Pramod Deshmukh wrote: > Thanks Joe, > > This branch works. I was able to proceed. I still had to set scala version > to 2.9.2 in kafka-run-class.sh. &g

Re: [DISCUSS] Kafka Security Specific Features

2014-07-16 Thread Pramod Deshmukh
ecurity LLC > http://www.stealth.ly > Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop> > ********/ > > > On Wed, Jul 16, 2014 at 3:49 PM, Pramod Deshmukh > wrote: > > > Thanks Joe for this, > > > > I c

Re: [DISCUSS] Kafka Security Specific Features

2014-07-16 Thread Pramod Deshmukh
Thanks Joe for this, I cloned this branch and tried to run zookeeper but I get Error: Could not find or load main class org.apache.zookeeper.server.quorum.QuorumPeerMain I see scala version is still set to 2.8.0 if [ -z "$SCALA_VERSION" ]; then SCALA_VERSION=2.8.0 fi Then I instal