Thanks for reply
All right, I've got next obstacle because of the differences in
production and development implementations.
I have Node, Product and Attribute models:
public class Product {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key id;
@Persistent(mappedBy = "product")
private Set<Attribute> attributes = new HashSet<Attribute>();
private Key nodeKey;
...
}
public class Attribute {
@PrimaryKey
@Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
private Key id;
@Persistent
private String name;
@Persistent
private Text value;
@Persistent
private Product product;
...
}
So, when I call pm.makePersistent(product); on my local machine it
takes ~200ms to persist 1 Product and a large(~100 name/values pairs)
collection of related Attributes. But on the production server the
same build runs ~7sec (from 5 to 10sec)
Commenting this line makes response timeout less than 1 sec on
production. That is why I'm sure the delay is in the persisting
operation.
BTW, is it true that low-level api and regular one(JDO) operate with
datastore in a different way?
I mean, is regular JDO api just a wrapper for low-level datastore api
or they utilize different mechanisms to send data to datastore?
Can I speed up my application by using low-level api?
Thank you
On Aug 17, 2:45 pm, "Jason (Google)" <[email protected]> wrote:
> Naturally, there will be some distinctions between the services/components
> used in the production environment and those included with the local SDK.
> We're working hard to keep this distinction as transparent as possible, but
> we do not currently have a document listing the implementation differences
> between the local and production components. That said, if you have any
> specific questions on how the local SDK works in relation to the production
> environment, please ask.
> - Jason
>
> On Fri, Aug 14, 2009 at 5:09 PM, PLX <[email protected]> wrote:
>
> > I appreciate
>
> > This is very useful information. Can you provide me with a link to
> > dev.server documentation, please? I want to review all differences
> > between production and development servers.
>
> > Thank you
>
> > On Aug 12, 3:26 pm, PLX <[email protected]> wrote:
> > > Hi,
>
> > > I'm trying to understand, how is it possible to insert a large amount
> > > of data to datastore with JDO. But I have a memory leak problem.
>
> > > PersistenceManager pm = null;
> > > for (int i = 0; i < 100; i++) {
> > > try {
> > > pm = PMF.get().getPersistenceManager();
> > > List<Product> products = new Vector<Product>();
> > > for (int j = 0; j < 1000; j++) {
> > > Product p = new Product();
> > > products.add(p);
> > > }
> > > pm.makePersistentAll(products);
> > > products.clear();
> > > System.out.println("committed " + i);
> > > } catch (Exception e) {
> > > e.printStackTrace();
> > > }
> > > finally {
> > > pm.close();
> > > }
>
> > > }
>
> > > I've tried:
> > > - to use the same PM instance all the time without recreating it after
> > > every 1000 inserts.
> > > - pm.flush();
> > > - pm.evictAll();
> > > - pm.makePersistent(p); instead of creating the list and calling
> > > pm.makePersistentAll() for it
>
> > > Nothing helps. After approximately 55000 inserts I get the same:
>
> > > java.lang.OutOfMemoryError: Java heap space
> > > at java.lang.reflect.Method.copy(Method.java:143)
> > > at java.lang.reflect.ReflectAccess.copyMethod
> > > (ReflectAccess.java:118)
> > > at sun.reflect.ReflectionFactory.copyMethod
> > > (ReflectionFactory.java:282)
> > > at java.lang.Class.copyMethods(Class.java:2748)
> > > at java.lang.Class.getMethods(Class.java:1410)
> > > at
>
> > com.google.appengine.tools.development.ApiProxyLocalImpl.getDispatchMethod
> > > (ApiProxyLocalImpl.java:149)
> > > at
> > > com.google.appengine.tools.development.ApiProxyLocalImpl.makeSyncCall
> > > (ApiProxyLocalImpl.java:88)
> > > at com.google.apphosting.api.ApiProxy.makeSyncCall
> > > (ApiProxy.java:79)
> > > at
> > > com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall
> > > (DatastoreApiHelper.java:48)
> > > at com.google.appengine.api.datastore.DatastoreServiceImpl
> > > $2.run(DatastoreServiceImpl.java:169)
> > > at
> > > com.google.appengine.api.datastore.TransactionRunner.runInTransaction
> > > (TransactionRunner.java:30)
> > > at com.google.appengine.api.datastore.DatastoreServiceImpl.put
> > > (DatastoreServiceImpl.java:157)
> > > at com.google.appengine.api.datastore.DatastoreServiceImpl.put
> > > (DatastoreServiceImpl.java:137)
> > > at com.google.appengine.api.datastore.DatastoreServiceImpl.put
> > > (DatastoreServiceImpl.java:129)
> > > at
>
> > org.datanucleus.store.appengine.RuntimeExceptionWrappingDatastoreService.put
> > > (RuntimeExceptionWrappingDatastoreService.java:92)
> > > at
> > > org.datanucleus.store.appengine.DatastorePersistenceHandler.put
> > > (DatastorePersistenceHandler.java:108)
> > > at
> > > org.datanucleus.store.appengine.DatastorePersistenceHandler.put
> > > (DatastorePersistenceHandler.java:94)
> > > at
> > > org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObject
> > > (DatastorePersistenceHandler.java:195)
> > > at
> > > org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent
> > > (JDOStateManagerImpl.java:3185)
> > > at org.datanucleus.state.JDOStateManagerImpl.makePersistent
> > > (JDOStateManagerImpl.java:3161)
> > > at org.datanucleus.ObjectManagerImpl.persistObjectInternal
> > > (ObjectManagerImpl.java:1298)
> > > at org.datanucleus.ObjectManagerImpl.persistObject
> > > (ObjectManagerImpl.java:1175)
> > > at org.datanucleus.jdo.JDOPersistenceManager.jdoMakePersistent
> > > (JDOPersistenceManager.java:669)
> > > at org.datanucleus.jdo.JDOPersistenceManager.makePersistentAll
> > > (JDOPersistenceManager.java:723)
> > > at test.servlet.RunServlet.doGet(RunServlet.java:46)
> > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:
> > > 693)
> > > at javax.servlet.http.HttpServlet.service(HttpServlet.java:
> > > 806)
> > > at org.mortbay.jetty.servlet.ServletHolder.handle
> > > (ServletHolder.java:487)
> > > at org.mortbay.jetty.servlet.ServletHandler
> > > $CachedChain.doFilter(ServletHandler.java:1093)
> > > at
> > > com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter
> > > (TransactionCleanupFilter.java:43)
> > > at org.mortbay.jetty.servlet.ServletHandler
> > > $CachedChain.doFilter(ServletHandler.java:1084)
> > > at
> > > com.google.appengine.tools.development.StaticFileFilter.doFilter
> > > (StaticFileFilter.java:124)
>
> > > Simply saying: does JDO have something like JPA's session.clear() ?
>
> > > Thanks
>
>
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~----------~----~----~----~------~----~------~--~---
