ohhh... AIX..
oprofile & latencytop is for Linux only :(

so. If i understand correctly
-- You know that slowdown you are interesting lay in protobuf
-- all other [network / request data size / request complexity ] is the 
same at the start and after the week

from my exp [oh, also very limited] AIX networking is the last thing to 
check

definitely, i cant to propose smth usefull... :(

may be, 
-- You alloc tooo memory
-- You need to use profiling tools
-- reduce problem to smth small & publish code

Dmitry

пятница, 11 января 2013 г., 12:45:40 UTC+4 пользователь debo...@2pirad.com 
написал:
>
> Thanks for your suggestions. Please find my response below:
>
>
> On Friday, January 11, 2013 1:16:19 PM UTC+5:30, Дмитрий Дьяченко wrote:
>>
>> if i undestand correctly, You have a looong-run application.
>>
> Yes, you are right, my application is a really looooong running one. It is 
> expected to run 24 x 7 x 365.
>  
>
>> - at start You have 2 ms cycle request-response
>>
> The actual request response cycle is a bit more than 2 ms, because after 
> the request is recieved by the application (#9 in my list), it does a few 
> things, which may take longer. But, the time taken to decode the protobuf 
> request is 2 ms at start.
>  
>
>> - after a week You have 200 ms cycle
>>
> Right, the time taken to decode gradually increases and becomes 200 ms. If 
> left running for longer, it even reached 1 sec in 15 days time.
>  
>
>> - Your app is correct and only place where problem may lay is protobuf?
>>
> From the logs which are there in my application, I have seen that only the 
> time taken to decode is gradually increasing. I have also done some memory 
> and thread profiling of my application, but did not find any irregularities 
> (like increase in memory/threads)
>  
>
>>
>> can You do the following?
>> -- run 'latencytop' at start and after a week?
>> -- run 'oprofile' at start and after a week?
>>
> My application is running on IBM AIX server (using IBM JVM). Are the above 
> tools available for AIX?
>  
>
>> -- may be its reasonable to writeout times from crititical points of Ypur 
>> app? smth like: before req? after req? before response - after response
>>
> I have logs specific for this purpose which are there at the start and end 
> of these points. Like, before decode and after decode, etc. From these logs 
> I found out the point which was gradually slowing down.
>
>
>> from my [very limited] exp a such problems traditionally becomes from 
>> unintendend sources -- memory allocator, log sub-system, database, swap, 
>> etc...
>>
> The only other component involved over here (other than the application 
> and JMV) is the network. But, I don't think network is the issue because 
> the decoding is done after the request object is fully received from the 
> client. Apache MINA is responsible for getting the request from the client.
>  
>
>>
>> Thanks,
>> Dmitry
>>
>> пятница, 11 января 2013 г., 10:50:54 UTC+4 пользователь 
>> debo...@2pirad.com написал:
>>>
>>> My application uses Google Protocol Buffer with Apache MINA. There are 
>>> clients who connect to the MINA Connector, send request to the server and 
>>> then wait for the response from the server. The flow of control is as 
>>> follows:
>>> 1. Client sends request
>>>     2. Request received at MINA filter (<Class> 
>>> extends CumulativeProtocolDecoder)
>>>         3. doDecode() method is called
>>>             4. A request object (generated from *.proto file) is created 
>>> using the <RequestObject>.parseFrom(bytes)
>>>                 5. The request is passed on to the IOHandler (<Class> 
>>> extends IoHandlerAdapter)
>>>                     6. messageReceived() method is called
>>>                         7. In this method, the request object (from #7) 
>>> is processed to create the list of requests which has been sent by the 
>>> client. 
>>> *                        8. At this point, we have noticed that the 
>>> time taken to process the request object (#7) is gradually degrading with 
>>> time. From a initial period of around 2 ms, the time period is going up to 
>>> 200 ms in just 8 days of continuous usage. And this value gradually 
>>> increases with time.*
>>>                             9. The request list is processed in the 
>>> application
>>>                           10. The response object is created
>>>                       11. The response is sent to the client.
>>>
>>> Any suggestion would be highly appreciated.
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Protocol Buffers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/protobuf/-/aJPz4InH5sQJ.
To post to this group, send email to protobuf@googlegroups.com.
To unsubscribe from this group, send email to 
protobuf+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/protobuf?hl=en.

Reply via email to