HI Nikhil, Thanks for putting this together.
What you are seeing seeing here is queueing effects in the webserver (Jetty, in our case) ... you are flooding the backend with work and there is contention within, resulting in each successive REST call having to wait longer and longer. Using your example app to stress the backend (setting the delay to 10ms) I have, in fact, found one contention point ... when debugging is enabled for the PublisherServiceLogging class, then this calls MemberExecutionDtoUtils#clone(...), which in turn creates a new JAXBContext each time. This is unnecessary (JAXBContext is thread-safe) and expensive to do; instead it should be cached. As a workaround, if you move debugging up to INFO or higher for PublisherServiceLogging in WEB-INF/logging.properties: log4j.logger.org.apache.isis.applib.services.publish.PublisherServiceLogging=INFO,PublisherServiceLogging log4j.additivity.org.apache.isis.applib.services.publish.PublisherServiceLogging=false then I see no queueing effects with 10ms delay (the response time is from 4 to 7 ms). Meantime, I've raised ISIS-1867 [1] to properly fix the issue. Thx Dan [1] https://issues.apache.org/jira/browse/ISIS-1867 On Thu, 15 Feb 2018 at 09:05 Nikhil Dhamapurkar < [email protected]> wrote: > Hi Dan, > > I have created isis-example project repository [1] it has simple-app and > small publisher application that sends post calls to apache isis Simple-app. > I tried to add tenancy but was not able to get it working, have commented > the code in the simple app, the login screen was preventing > isis-module-security-admin/pass from logging in. > > I believe if you just run Startup.java class of http-publisher maven > project while apache isis simple-app is running ( existing isis app should > work as I have commented the tenancy code in my application.) > > The request / response takes ~ 1 sec for creating a new object if delay > between requests is 5 sec, I have used vertx’s httpClient and also added a > vetx webclient that sends a create request for simpleObjects, if the delay > between object creation is reduced the response time increases. > > [1] https://[email protected]/dhamapurkar/isis-example.git > > Regards > Nikhil > > From: Dan Haywood > Sent: 01 February 2018 18:53 > To: [email protected] > Subject: Re: RE: Swagger / Apache Isis response batching > > Hi Nikhil, > > Could you put together a sample app to demonstrate the issue, in particular > how you are calling the rest API? > > I'm not aware of any batching code in Isis itself, but I think I am > probably not fully understanding what you're aiming to do. A sample app > would make sure we're on the same page and give me a head start for > developing an improvement to Isis if it turns out we need one. > > Thx, > Dan. > > On Thu, 1 Feb 2018, 07:39 Andi Huber, <[email protected]> wrote: > > > Ok, thanks Nikhil, > > > > this sounds like an issue - worth investigating - to me. > > > > Clarification: Tanancy logic is handled by the 'security' module that's > > provided by the 'incode platform', which is not part of Isis 'core'. > > > > Just to let you know: It's unlikely, that I myself, will look into > > incode's source-code, but hopefully someone else will help out! > > > > Regards, Andi > > > > > > On 2018/02/01 05:05:08, Nikhil Dhamapurkar <[email protected]> > > wrote: > > > Hi Andi,> > > > > > > I believe the issue should be easy to reproduce and gets elevated in > > an application which supports tenancy because when the object is going > > to be rendered for UI or sent to client it will pass through the > > implementation of interface ApplicationTenancyEvaluator which will need > > to read data from user or roles.> > > > > > > If you publish bulk insert requests 20-30 via swagger you will notice > > the data will get inserted in DB much ahead than the responses received > > on the client handler.> > > > I added small instrumentation in code with stop watch which had total > > time taken for the request /response, Time spent in tenancy checks , > > time spent in Menu calls in Apache ISIS but the total client time is > > much more than sum of tenancy + method call execution time.> > > > > > > Regards> > > > Nikhil> > > > > > > From: Andi Huber> > > > Sent: 31 January 2018 14:46> > > > To: [email protected]> > > > Subject: Re: Swagger / Apache Isis response batching> > > > > > > Hi Nikhil,> > > > > > > I guess there is no such option, but I might be wrong.> > > > > > > From my understanding, any request you send to Isis (and swagger), is> > > > processed within a transaction Likely you don't get a response unless> > > > this particular transaction has completed (either with success or > not).> > > > > > > Not sure if this applies to your use-case, but you might solve this by> > > > reducing the number of records done per batch.> > > > > > > If you believe this is an issue with Isis, we could look into it, but> > > > would need more information on how to reproduce the issue.> > > > > > > regards, Andi> > > > > > > > > > On 2018/01/29 16:04:12, Nikhil Dhamapurkar <[email protected] > >> > > > wrote:> > > > >> > > > > Hi,>> > > > >> > > > > We are persisting data in DB using isis / swagger URI around 5500> > > > records. I can see the inserts in Database are being done at > acceptable> > > > times; in a few milliseconds.>> > > > >> > > > > But I can see that swagger / apache isis is batching the responses > and> > > > not sending them asap this delays the client from getting the response> > > > back in time.>> > > > >> > > > > Since he response is not received the client in its next cycle send> > > > the same request, is there a property where I can disable this > > batching ?>> > > > >> > > > > I have tried adding c3p0 for connection pooling and cached tenancy> > > > changes ins some cases to gain performance improvement but its not> > > > helping much.>> > > > >> > > > >> > > > > Statistics :>> > > > > DB insert per record ~ 2ms.>> > > > > The response time seen in client increases ( maybe queued ?) = 30 > sec> > > > and more for later inserts.>> > > > >> > > > > Regards>> > > > > Nikhil>> > > > >> > > > >> > > > > > > > > > > > > > > >
