Re: reg Kafka Node num

2017-10-23 Thread pravin kumar
h negative integers. -1 for the first bootstrap server, -2 > for the second and so on. > > > > ------ Forwarded message -- > From: pravin kumar <pk007...@gmail.com> > To: users@kafka.apache.org > Cc: > Bcc: > Date: Fri, 20 Oct 2017 14:44:04 +0530 >

StreamTasks

2017-10-24 Thread pravin kumar
I have created a stream with topic contains 5 partitions and expected to create 5 stream tasks ,i got [admin@nms-181 ]$ ls 0_0 0_1 0_2 0_3 0_4 1_0 1_1 1_2 1_3 1_4 SingleConsumerMultiConsumerUsingStreamx4 is my application_id and this 1_0,1_... contains localStateStore and 0_0,0_

regarding number of Stream Tasks

2017-10-24 Thread pravin kumar
I have created a stream with topic contains 5 partitions and expected to create 5 stream tasks ,i got 10 tasks as 0_0 0_1 0_2 0_3 0_4 1_0 1_1 1_2 1_3 1_4 my doubt is:im expected to have 5 tasks how it produced 10 tasks here are some logs: [2017-10-24 10:27:35,284] INFO

GlobalKStore

2017-11-01 Thread pravin kumar
i have created 3 topics with 10 partitions each i have intended to store processed information in globalKtable now i have did with individual Ktable to Output topic then to GlobalKtable #// KStreamBuilder builder=new KStreamBuilder(); KStream inputStream =

WordCount Example using GlobalKStore

2017-11-01 Thread pravin kumar
i have created 3 inputtopics with 10 partitions each and output Topic with 10 partitions I did wordcount example and stored it in GlobalKTable. i initally stored counted value in LocalStateStore and then it to GlobalStateStore. i have atteated the code here:

Re: regarding number of Stream Tasks

2017-10-31 Thread pravin kumar
enerated due to repartitioning > which would cause the extra tasks. > If you provide the topology we would be able to tell you. > Thanks, > Damian > > On Tue, 24 Oct 2017 at 10:14 pravin kumar <pk007...@gmail.com> wrote: > > > I have created a stream with topic contains 5 partiti

Re: regarding number of Stream Tasks

2017-10-31 Thread pravin kumar
asks processing the partitions from the > repartitioning. > > On Tue, 31 Oct 2017 at 10:56 pravin kumar <pk007...@gmail.com> wrote: > > > I have created a stream with topic contains 5 partitions and expected to > > create 5 stream tasks ,i got 10 tasks as > &g

reg kafka API versions

2017-10-20 Thread pravin kumar
I have run a Multiple Partition and Multiple Consumer application.Then i have altered the no of Partitions, in log im getting Using older server API v0 to send HEARTBEAT {group_id= altermultipartitionmulticonsumer2,group_generation_id=6, member_id=C2-b7b4f485-d0c3-466b-b0e7-b3de834832e9} to

Doubts in KStreams

2018-02-21 Thread pravin kumar
I have studied KafkaStreams, but not clearly understood 1.Can someone explain about Fault tolerence. 2.I have topicA and topicB with 4 partitions, so it created fourTasks, I have created it in singleJVM.But i need to knw how it works in multiple JVM and if one jvm goes down,how it another jvm

usage of depricated method in kafka 2_12.1.0.0

2018-02-21 Thread pravin kumar
i have tried wikifeed example with Kafka 2_12.1.0.0.the count method is now depricated , previously in kafka_2.11-0.10.2.1 i have given count(localStateStoreName). how to give the statestore name in Kafka 2_12.1.0.0. i have attached the code below, package kafka.examples.wikifeed; import

Doubts about multiple instance in kafka

2018-02-21 Thread pravin kumar
(ConsumerConfig.CLIENT_ID_CONFIG,"C1"); //properties.put(ConsumerConfig.CLIENT_ID_CONFIG,"C2"); properties.put(ConsumerConfig.CLIENT_ID_CONFIG,"C3"); return properties; } public static void produceInput(){ Random random=new Random();

ProcessorTopology

2018-02-22 Thread pravin kumar
Can we give the ouput of one processorTopology as the input to another processorTopology. if it is possible,how can we do it. Can anyone provide it with any example

Tasks across MultipleJVM

2018-02-28 Thread pravin kumar
al static String CONSUMER_WIKIFEED_LAMBDA="ConsumerWikiFeedLambda1"; public static void main(String[] args) { ProducerInput(); ConsumerOutput(); } public static void ProducerInput(){ String[] users={"pravin","kumar","er

Producing more number of Records than expected

2018-03-06 Thread pravin kumar
I have run wikifeed example. i have three topics: wikifeedInputtopicDemo2-10 partitions wikifeedOutputtopicDemo2-10 partitions sumoutputeventopicDemo2-5 partitions i have produced 10 records.but in the inputTopic(wikifeedInputtopicDemo2) it receives more than 10 records. can someone

Producing more number of Records than expected

2018-03-06 Thread pravin kumar
OTSTRAP_SERVERS="localhost:9092"; final static String CONSUMER_WIKIFEED_LAMBDA="ConsumerWikiFeedLambda1"; public static void main(String[] args) { ProducerInput(); ConsumerOutput(); } public static void ProducerInput(){ String[] users={"

Consumer slowness issue

2018-03-27 Thread pravin kumar
mStreams::close));*/ } public static void ProducerInput(){ String[] users={"pravin","kumar","erica", "bob", "joe", "damian", "tania", "phil", "sam",

Fwd: Consumer slowness issue

2018-03-27 Thread pravin kumar
-- Forwarded message -- From: "pravin kumar" <pk007...@gmail.com> Date: 27-Mar-2018 6:11 PM Subject: Consumer slowness issue To: <users@kafka.apache.org> Cc: i have two topics with 5 partitions each wikifeedInputT10 KafkaProducer produces 10 element