That code is only called once.
All the time is spent after execution leaves my server function. So it is
somewhere in the generated thrift code.
Just to be certain I generated the test data at the start of the program
like
results = [('2016-05-20T14:01:01+0000', [22222])] * 60000
But I get the same reults. 2 minutes twenty seconds.
My python server process runs at over 100% cpu.
Thanks
On 16 June 2016 at 19:23, Randy Abernethy <[email protected]> wrote:
> Hi Glenn,
>
> While two minutes is a little surprising, I suspect opening/closng the file
> every time the getData() method is called will be problematic. File open
> operations are usually an order of magnitude more expensive than file read
> or write operations. Try opening the file in the handler constructor,
> buffering it, closing the file and then responding to getData requests
> using the buffer.
>
> -Randy
>
>
> On Thu, Jun 16, 2016 at 9:08 AM, Glenn Pierce <
> [email protected]> wrote:
>
> > Hi I wonder if someone could give advice on a performance issue I have.
> >
> > I have written a test case in Python
> >
> > The server is
> >
> > class TestServerThriftHandler(object):
> >
> > def getData(self):
> > with open('data.json', 'r') as f:
> > results = ujson.loads(f.read())
> > results = [(r['timestamp'], r['values']) for r in
> results]
> > print "size of list", len(results)
> > data = [ThriftTestGroup(timestamp=row[0],
> > values=[float(x) if x is not None
> > else float('NaN') for x in row[1]]) for row in results]
> >
> > return data
> >
> >
> > if __name__ == "__main__":
> > handler = TestServerThriftHandler()
> > processor = ThriftTestService.Processor(handler)
> > transport = TSocket.TServerSocket("127.0.0.1", 7777)
> > tfactory = TTransport.TBufferedTransportFactory()
> > pfactory = TBinaryProtocol.TBinaryProtocolFactory()
> > server = TServer.TThreadedServer(processor, transport, tfactory,
> > pfactory)
> > print 'Starting server'
> > server.serve()
> >
> >
> > The interface is
> >
> > struct ThriftTestGroup {
> >
> > 1: required Timestamp timestamp;
> > 2: required list<double> values;
> > }
> >
> > service ThriftTestService {
> >
> > list<ThriftTestGroup> getData() throws ( 1:TestException ex);
> > }
> >
> >
> > Basically getData() loads some example 58773 entries into a list from a
> > file.
> > Each entry is a string representing the time like
> > 2016-05-20T13:58:59+0000 and one or more double values.
> > In this test there is only one double value.
> >
> > My client on the same host calls this method and it takes 2 minutes 14
> > seconds ?
> > My code loading the data in the server in instant.
> >
> > Does anyone have an idea why this is so slow ?
> >
> > Any advice would be great.
> >
> > Thanks
> >
>
