The important profile lines are ncalls <http://carneab4.memset.net:8000/?sort=calls> tottime <http://carneab4.memset.net:8000/?sort=time> percall cumtime <http://carneab4.memset.net:8000/?sort=cumulative> percall filename <http://carneab4.memset.net:8000/?sort=module>:lineno <http://carneab4.memset.net:8000/?sort=nfl>(function) 1 0.006 0.006 963.335 963.335 ./ThriftTestServer.py:3(<module> <http://carneab4.memset.net:8000/?func_name=%3Cmodule%3E>) 1 0.000 0.000 963.304 963.304 /usr/lib64/python2.7/site-packages/thrift/server/TServer.py:102(serve <http://carneab4.memset.net:8000/?func_name=serve>) 2 0.000 0.000 963.303 481.652 /usr/lib64/python2.7/site-packages/thrift/transport/TSocket.py:188(accept <http://carneab4.memset.net:8000/?func_name=accept>) 2 0.000 0.000 963.303 481.652 /usr/lib64/python2.7/socket.py:201(accept <http://carneab4.memset.net:8000/?func_name=accept>) 2 963.303 481.652 963.303 481.652 {method 'accept' of '_socket.socket' objects}
On 16 June 2016 at 20:01, Glenn Pierce <[email protected]> wrote: > That code is only called once. > All the time is spent after execution leaves my server function. So it is > somewhere in the generated thrift code. > > Just to be certain I generated the test data at the start of the program > like > > results = [('2016-05-20T14:01:01+0000', [22222])] * 60000 > > But I get the same reults. 2 minutes twenty seconds. > > My python server process runs at over 100% cpu. > > > Thanks > > > > > > On 16 June 2016 at 19:23, Randy Abernethy <[email protected]> wrote: > >> Hi Glenn, >> >> While two minutes is a little surprising, I suspect opening/closng the >> file >> every time the getData() method is called will be problematic. File open >> operations are usually an order of magnitude more expensive than file read >> or write operations. Try opening the file in the handler constructor, >> buffering it, closing the file and then responding to getData requests >> using the buffer. >> >> -Randy >> >> >> On Thu, Jun 16, 2016 at 9:08 AM, Glenn Pierce < >> [email protected]> wrote: >> >> > Hi I wonder if someone could give advice on a performance issue I have. >> > >> > I have written a test case in Python >> > >> > The server is >> > >> > class TestServerThriftHandler(object): >> > >> > def getData(self): >> > with open('data.json', 'r') as f: >> > results = ujson.loads(f.read()) >> > results = [(r['timestamp'], r['values']) for r in >> results] >> > print "size of list", len(results) >> > data = [ThriftTestGroup(timestamp=row[0], >> > values=[float(x) if x is not >> None >> > else float('NaN') for x in row[1]]) for row in results] >> > >> > return data >> > >> > >> > if __name__ == "__main__": >> > handler = TestServerThriftHandler() >> > processor = ThriftTestService.Processor(handler) >> > transport = TSocket.TServerSocket("127.0.0.1", 7777) >> > tfactory = TTransport.TBufferedTransportFactory() >> > pfactory = TBinaryProtocol.TBinaryProtocolFactory() >> > server = TServer.TThreadedServer(processor, transport, tfactory, >> > pfactory) >> > print 'Starting server' >> > server.serve() >> > >> > >> > The interface is >> > >> > struct ThriftTestGroup { >> > >> > 1: required Timestamp timestamp; >> > 2: required list<double> values; >> > } >> > >> > service ThriftTestService { >> > >> > list<ThriftTestGroup> getData() throws ( 1:TestException ex); >> > } >> > >> > >> > Basically getData() loads some example 58773 entries into a list from a >> > file. >> > Each entry is a string representing the time like >> > 2016-05-20T13:58:59+0000 and one or more double values. >> > In this test there is only one double value. >> > >> > My client on the same host calls this method and it takes 2 minutes 14 >> > seconds ? >> > My code loading the data in the server in instant. >> > >> > Does anyone have an idea why this is so slow ? >> > >> > Any advice would be great. >> > >> > Thanks >> > >> > >
