[appengine-java] Datastore in task

[Timofey Koolin]

Now i write import data into datastore and clean datastore.

When I import data - i upload zip file to servlet, into servlet I
unzip it and create some tasks for import data.
First time, every task in first run can't access to datastore:

03-29 09:49AM 25.793 /ImportTask 500 10067ms 0cpu_ms 0kb AppEngine-
Google; (+http://code.google.com/appengine)
0.1.0.2 - - [29/Mar/2010:09:49:35 -0700] "POST /ImportTask HTTP/1.1"
500 0 "http://import.latest.tim-test-001.appspot.com/ImportTask";
"AppEngine-Google; (+http://code.google.com/appengine)"
"import.latest.tim-test-001.appspot.com"
W 03-29 09:49AM 35.860
Request was aborted after waiting too long to attempt to service your
request. Most likely, this indicates that you have reached your
simultaneous dynamic request limit. This is almost always due to
excessively high latency in your app. Please see
http://code.google.com/appengine/docs/quotas.html for more details.

Yesterday I think that I overquote CPU/minute when start 3-4 threads
of import data.

Today i write clean servlet and run it in one thread. I have same. I
read - free quota is 15 CPU minutes/minute. I have 1 thread of clean
servlet:

            DatastoreService ds =
DatastoreServiceFactory.getDatastoreService();
            for (String kind : kinds) {
                log("kind: " + kind);
                Query query = new Query(kind);
                query.setKeysOnly();
                query.addSort("__key__");
                int size;
                do {
                    log("prepare query");
                    PreparedQuery pq = ds.prepare(query);
                    ArrayList<Key> keys = new ArrayList<Key>(max_ids);
                    for (Entity e : pq.asIterable(withLimit(max_ids)))
{
                        keys.add(e.getKey());
                    }
                    ds.delete(keys);
                    size = keys.size();
                    log("delete: " + size);
                    count += size;
                } while (size > 0);
            }
            out.println("Deleted all: " + count);
        } catch (DeadlineExceededException ex) {
            Queue q = QueueFactory.getDefaultQueue();
            q.add(url("/
Clean").method(TaskOptions.Method.GET).param("start", "" + count));
            out.println("Deleted: " + Integer.toString(count));
        } finally {
            out.close();
        }

First time I start it by url and first created task have same problem
as above. But task, which i create in task (when i have timeoute
exception) haveno it problem.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine-java?hl=en.