The 130ms request seems entirely reasonable. There's also only one request listed there so it may at the very least be an irregular occurrence though you do mention that more causes more latency but that cannot be seen from the Stackdriver Trace graph screenshot you've attached.
More importantly, not every request is the same and different factors may play a role in the latency you describe. - Networks may experience heavier latency at different times depending on congestion. Internal Google networks are no exception. We have not however experienced any heavy latency with the Cloud Datastore in some time so this is not likely the cause. Additionally, one would have to show consistent latency over a given period of time for network congestion to be a more likely cause. - Are your keys properly sharded? Though Cloud Datastore can generate sharded keys upon committing entities, it is possible for users to use their own keys. In these cases, one must take particular care to shard keys as this empowers the Datastore to locate entities more quickly while reducing the risks of data contention. The article Avoiding datastore contention <https://cloud.google.com/appengine/articles/scaling/contention> though somewhat dated, still holds valid precepts. - What are the fetched entities? Entities may vary in size greatly. Currently, the maximum size of an entity is 1,048,572 bytes <https://cloud.google.com/datastore/docs/concepts/limits>. Entities of this size may take more time to transmit *along the wire*. It's difficult to assess the performance expectation you present without more detailed information such as key distribution, entity size/contents, consistency in latency and parameters of load testing you might have done to confirm this is a platform issue. Based on the information you've provided, I would generally recommend the following: - Splitting up requests if you know that they will require serial interdependent Datastore fetches - Using memcache whenever possible to keep frequently accessed information closer - Consider using the form of Get that retrieves multiple entities at once <https://cloud.google.com/appengine/docs/standard/java/javadoc/com/google/appengine/api/datastore/DatastoreService.html#get-java.lang.Iterable-> to reduce time costs Note also that the Cloud Datastore does not have any latency/performance-related SLAs <https://cloud.google.com/datastore/sla> at this time. Nevertheless, if after going through the above recommendations, you load test and find there to be consistent latency issues, I'd recommend filing an issue on the Issue Tracker <https://issuetracker.google.com/issues/new>. We frequently that forum as well. Hope this addressees your concerns. Nicholas On Wednesday, March 22, 2017 at 7:56:24 AM UTC-4, Ivo Bellin Salarin wrote: > > The datastore.Get implementation can use cache (memcached). But I can't > see that kind of calls on your graph. > > Perhaps the size of entities you're trying to get is the reason for this > gap? > > Le mer. 22 mars 2017 12:30, Rajesh Gupta < > [email protected]> a écrit : > >> Hello, >> I am analyzing the trace of the request. Please find attached. >> >> The datastore_v3.Get() api call duration varies from 11ms to 131ms. >> >> If there are more datastore_v3.Get(), then the response time is highly >> unpredictable. >> >> Regards, >> Rajesh >> *www.VeersoftSolutions.com <http://www.VeersoftSolutions.com>* >> *www.GainERP.com <https://www.gainerp.com>* >> *Accounting/Inventory/Orders/Sales/Purchase on Google Cloud Platform and >> Mobile* >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To post to this group, send email to [email protected]. >> Visit this group at https://groups.google.com/group/google-appengine. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/google-appengine/CA%2BS7ijaD-HfM5FZbVWDvc76J%3DmAcpiyXszEsFyzp1HBtap%2Bz9g%40mail.gmail.com >> >> <https://groups.google.com/d/msgid/google-appengine/CA%2BS7ijaD-HfM5FZbVWDvc76J%3DmAcpiyXszEsFyzp1HBtap%2Bz9g%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> For more options, visit https://groups.google.com/d/optout. >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at https://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/a2c61928-bfae-479d-b2ae-5d34bdae7079%40googlegroups.com. For more options, visit https://groups.google.com/d/optout.
