Hi Again & sorry to take your time. But am puzzled by what I cannot explain 
why.The parallelism is set to 448. There are 112 tasks per TM. Why is Flink NOT 
allocating ALL 448 slots? It allocates only 1/2 of it.I also bumped up the # 
buffers to equate a 2GiB in each TM & see no difference :-(So I incremented my 
total-slots = 448. Kafka topic also has 448 partitions.Why am I having such a 
bad luck with this!!!!!!!??? LOL!!Thanks for your attention Aljoscha.

      From: amir bahmanyari <amirto...@yahoo.com>
 To: Aljoscha Krettek <aljos...@apache.org>; User <user@flink.apache.org> 
 Sent: Thursday, September 22, 2016 10:10 AM
 Subject: Re: How can I prove ....
Hi Again, following is from the dashboard while wverything is supposedlt 
running.No real-time change in send/received/#of records...but one node is 
definitely producing a *.out file...And all TMs are reporting in their *.log 
files. And the process will eventually end , but very slow.Thanks again 

      From: amir bahmanyari <amirto...@yahoo.com>
 To: Aljoscha Krettek <aljos...@apache.org>; User <user@flink.apache.org> 
 Sent: Thursday, September 22, 2016 9:16 AM
 Subject: Re: How can I prove ....
Thanks Aljoscha,Thats why I am wondering about this. I dont see send/receive 
columns change at all....just 0's all the time.The only thing that changes is 
time stamp.Is this an indication that the nodes in the cluster are not 
participating in execution of the data?Thanks again.Amir-

      From: Aljoscha Krettek <aljos...@apache.org>
 To: amir bahmanyari <amirto...@yahoo.com>; User <user@flink.apache.org> 
 Sent: Thursday, September 22, 2016 5:01 AM
 Subject: Re: How can I prove ....
Hi,depending on the data source you might not be able to stress CPU/MEM because 
the source might be to slow. As long as you see the numbers increasing in the 
Flink Dashboard for all operators you should be good.
On Thu, 22 Sep 2016 at 00:26 amir bahmanyari <amirto...@yahoo.com> wrote:

That all nodes in a Flink Cluster are involved simultaneously in processing the 
data?Programmatically, graphically...I need to stress CPU , MEM and all 
resources to their max.How can I guarantee this is happening in Flink 
Cluster?Out of 4 nodes, this is the highest resource usage I see from 
"top"...Everything else is not even close...top - 22:22:45 up 41 days,  2:39,  
1 user,  load average: 1.76, 1.55, 1.28Tasks: 344 total,   1 running, 343 
sleeping,   0 stopped,   0 zombie%Cpu(s):  5.4 us,  1.0 sy,  0.0 ni, 93.5 id,  
0.1 wa,  0.0 hi,  0.0 si,  0.0 stKiB Mem:  11551564+total, 65702020 used, 
49813632 free,   115072 buffersKiB Swap:        0 total,        0 used,        
0 free.  3148420 cached Mem

I am pretty sure I can push FlinkRunner to way more extent than this....And 
thats where true realistic perf numbers start showing up.Thanks+regards,Amir-




Reply via email to