Re: [appengine-java] How do I query by key?

2012-03-22 Thread Jeff Schnitzer
Let's say you have a Profile entity with an id of 123. The easiest way to retrieve it is to use DatastoreService.get(), not a query. Construct a Key and pass that to get(): DatastoreService ds = DatastoreServiceFactory.getDatastoreService(); Key key = KeyFactory.createKey(Profile, 123L); Entity

Re: [appengine-java] Quick setup and deploy for university class lab time?

2012-03-21 Thread Jeff Schnitzer
Can students install Eclipse and run Hello, World on GAE in 3.5 hours? Almost certainly. How much more sophisticated can the sample project get? This is a hard thing to judge without knowing your students. Back when I was a CSc student, data structures was a 200-level class and few (if any)

Re: [appengine-java] How to handle class version issues in DataStore

2012-03-12 Thread Jeff Schnitzer
That's not quite correct - in the low-level datastore, this isn't an issue because absent properties will simply be missing from the Entity. In Objectify, absent properties are ignored. You can add fields of primitive types without ill effect - they will simply have their default values. This

Re: [appengine-java] GAE performance, the log, and a missing 500ms

2012-03-11 Thread Jeff Schnitzer
Have you tried using appstats? (isn't this group supposed to be readonly?) Jeff On Sun, Mar 11, 2012 at 12:28 PM, n3phele n3ph...@gmail.com wrote: Hi Im trying to understand the performance of my GAE app, which appears to be declining over time. Here is a log from my Java app .. please

Re: [appengine-java] Reminder: Google App Engine community support is moving to Stack Overflow

2012-03-08 Thread Jeff Schnitzer
I find that you're either one of those people who lives on StackOverflow or you're one of those people who only visits when they're searching for something specific. I'm the later myself; I can't follow SO the way I follow an email list. Truth be told, I have the google-appengine list and the

Re: [appengine-java] Logging exceptions all over!

2012-03-06 Thread Jeff Schnitzer
I also see a few entries like this in my logs. My site is still up, however. (this mailing list is about to go read-only - further discussion should go on the regular google-appeng...@googlegroups.com mailing list) Jeff On Tue, Mar 6, 2012 at 2:38 PM, Zappa kbhdk1...@gmail.com wrote:

Re: [appengine-java] Best way to dynamic scheduled task

2012-03-05 Thread Jeff Schnitzer
idea, and I think that will work. My doubt was about the use of the resource... I think that using task queue (as suggested by Ian Marshall) is less expansive than a cron job... Are you agree? Thanks Alessandro On Monday, March 5, 2012 4:19:04 PM UTC+1, Jeff Schnitzer wrote: When you set

Re: [appengine-java] Eclipse shortcut key to launch/debug app engine web application

2012-03-03 Thread Jeff Schnitzer
This. Please. This has been killing me for years. GWT adds a little restart button on the console. I want that button for non-GWT apps - and specifically a hotkey that will press it. For some apps you can effectively get this by executing touch war/WEB-INF/appengine-web.xml. However,

Re: [appengine-java] Re: Problem with exception on server startup when GAE sdk is in svn

2012-03-01 Thread Jeff Schnitzer
Time to migrate to Git! :-) Jeff On Thu, Mar 1, 2012 at 9:25 AM, Rick Mangi r...@broadcastr.com wrote: Dilbert, see my reply on the ticket. Max - that comment about svn being writable is nonsense. I wish you guys would talk to your own support engineers. I tried that solution with my

[appengine-java] Javaland scheduler behavior

2012-02-28 Thread Jeff Schnitzer
There's been a lot of discussion of the scheduler behavior in Pythonland, but not much about it's eccentricities in Javaland. I have a threadsafe=true Java app. Let's say every request completes in exactly 1s. Settings are: idle instances min 1 max 1, latency auto/auto. Here is what I

Re: [appengine-java] Re: Javaland scheduler behavior

2012-02-28 Thread Jeff Schnitzer
just a bit scared now to get over my daily quotas. François On Tuesday, February 28, 2012 5:34:36 PM UTC+1, Jeff Schnitzer wrote: There's been a lot of discussion of the scheduler behavior in Pythonland, but not much about it's eccentricities in Javaland. I have a threadsafe=true Java app

Re: [appengine-java] Re: Javaland scheduler behavior

2012-02-28 Thread Jeff Schnitzer
I just realized I sent this to the wrong (java-specific, deprecated) Google Group. For those of you on both the google-appengine list and this one, I apologize in advance for the duplicate thread I'm about to start :-( Jeff On Tue, Feb 28, 2012 at 12:25 PM, Jeff Schnitzer j

Re: [appengine-java] max keys for memcache

2012-02-22 Thread Jeff Schnitzer
You are overthinking this. To the point of lunacy. Jeff On Wed, Feb 22, 2012 at 5:07 PM, Kesava Neeli nke...@gmail.com wrote: We would like to have a map of all users in datastore in mem cache so that we can look up for any username/email/number to verify if he is already an user of our

Re: [appengine-java] max keys for memcache

2012-02-22 Thread Jeff Schnitzer
Your requirement appears to be able to look up user by email. The datastore does exactly this for you. The idea of loading your entire datastore into memcache just so you can do lookups is totally absurd. Jeff On Wed, Feb 22, 2012 at 6:05 PM, Kesava Neeli nke...@gmail.com wrote: Jeff, If

Re: [appengine-java] max keys for memcache

2012-02-21 Thread Jeff Schnitzer
Are all of these 1-2 million users going to be active at the same time? If not, why do you care if it all fits in memcache? On the other hand, if they *are* all going to be active at once, I would be very curious to know what happens... Jeff On Tue, Feb 21, 2012 at 2:07 PM, Kesava Neeli

Re: [appengine-java] Re: Is it safe for one ancestor to have millions of children in one List object?

2012-02-20 Thread Jeff Schnitzer
The biggest question is why would you want to do this? From the low-level api perspective, you can't have a list property with more than 5k items. But maybe JDO models this list as a proxy to a query. When you try to load that proxy it's going to load all 5m items out of the datastore and you

Re: [appengine-java] scheduling 1000 emails

2012-02-20 Thread Jeff Schnitzer
You can schedule tasks to execute in the future. It doesn't guarantee that the task will be executed then, however. You might find the queue backed up, in which case you'll get a long delay then all the tasks executing rapidly. Alternatively, you can throttle the task queue to execute once per

Re: [appengine-java] Remote API security ?

2012-02-18 Thread Jeff Schnitzer
On Fri, Feb 17, 2012 at 10:22 PM, meric mivi...@gmail.com wrote: Can i use the remote api on standalone applications that users, other than the admin, can download and use ? Can the users see the username/password of my GAE account that i have to put in my app to access the api ? Just

Re: [appengine-java] Batch puts

2012-02-07 Thread Jeff Schnitzer
From my experiments, writing large #s of entities to a single EG in a batch put is quite fast and does not cause the throughput problem. Jeff On Tue, Feb 7, 2012 at 4:43 PM, Andrew Myers am2...@gmail.com wrote: I'm sure this is a FAQ but I haven't been able to find the answer so far. I have

Re: [appengine-java] Re: How do I run DeferredTasks in a Unit Test

2012-02-02 Thread Jeff Schnitzer
This is a blast from the past. There are now official tools for unit testing deferred tasks: http://code.google.com/appengine/docs/java/tools/localunittesting.html#Writing_Deferred_Task_Tests However, ignore the latch stuff - it works erratically, and if you have an even slightly complicated

Re: [appengine-java] Re: Logging when using local unit testing

2012-01-30 Thread Jeff Schnitzer
A appreciate the out-of-the-box suggestion, but this strikes me as an extraordinarily invasive change just to get logging to render the way it is supposed to. Presumably there is something in the GAE stack that monkeys with java.util.logging. I just want to know what it is and how to beat it

[appengine-java] Logging when using local unit testing

2012-01-27 Thread Jeff Schnitzer
Does anyone have logging working when using the local unit testing system (ie LocalServiceTestHelper)? For the life of me I cannot get any log output. System.out works, but the jdk logger does not. I'm running my (TestNG) tests through the eclipse plugin. I'm pretty certain this isn't a TestNG

Re: [appengine-java] how to store java Date type in specific timezone?

2012-01-25 Thread Jeff Schnitzer
You must store the TZ as a separate field. Alternatively, if you don't need the date to be indexed, you can convert it back and forth to a String that preserves TZ info (eg, the ISO-8601 format). Jeff On Wed, Jan 25, 2012 at 12:53 PM, Kesava Neeli nke...@gmail.com wrote: Hi, I have a

Re: [appengine-java] how to store java Date type in specific timezone?

2012-01-25 Thread Jeff Schnitzer
Taking a step back, I should say that if you're using java.util.Date with timezone info, you're doing it wrong. java.util.Date is a trainwreck, and any methods that affect an internal notion of a TZ have been deprecated. If you care about timezones, using Date is a mistake - it's best to think

Re: [appengine-java] how to store java Date type in specific timezone?

2012-01-25 Thread Jeff Schnitzer
I store DateTime, LocalDate, LocalDateTime, and DateTimeZone quite a lot, but I use Objectify (disclosure: I'm the lead author) which handles the translation for you (if you enable it). ReadableInstant objects (eg DateTime) are stored natively as java.util.Date and Local objects (LocalDate,

Re: [appengine-java] lost trying to understand JDO on BigTable: please help coding example

2012-01-24 Thread Jeff Schnitzer
On Tue, Jan 24, 2012 at 4:21 PM, John Goche johngoch...@googlemail.com wrote: The main reason I ask is that loading two entities is about twice as expensive as loading one twice the size. Tough. I cannot sacrifice code readability here. This is an intriguing attitude. You have arrived with

Re: [appengine-java] json communication with google app engine

2012-01-17 Thread Jeff Schnitzer
I use Resteasy with Jackson. It works great. Uses the standard JAX-RS annotations. http://docs.jboss.org/resteasy/docs/2.3.0.GA/userguide/html_single/index.html Jeff On Mon, Jan 16, 2012 at 11:51 PM, angstrom348 linux.mod...@gmail.com wrote: I am trying to allow for app engine interaction

Re: [appengine-java] json communication with google app engine

2012-01-17 Thread Jeff Schnitzer
As a bonus, you can also use Resteasy for webpage rendering using this thin framework on top: http://code.google.com/p/htmleasy/ (I wrote it originally, although another guy maintains it now. I still use it, and - if I may say so - it's great) Jeff On Wed, Jan 18, 2012 at 12:38 AM, Jeff

Re: [appengine-java] json communication with google app engine

2012-01-17 Thread Jeff Schnitzer
, Wilson MacGyver wmacgy...@gmail.com wrote: I'm curious if you could share why you picked resteasy over jersey of other jax-rs implementations? On Jan 17, 2012, at 7:38 PM, Jeff Schnitzer j...@infohazard.org wrote: I use Resteasy with Jackson.  It works great.  Uses the standard JAX-RS

Re: [appengine-java] Re: problem with Content-type: null with css files

2012-01-14 Thread Jeff Schnitzer
One thing that would make this helpful would be the ability to automatically delete old versions. The current limit of 10 versions means I have to manually go through and delete old ones before I can deploy new ones. It would be nice to automate a scrolling window of versions. Of course, not

Re: [appengine-java] Re: Abridged summary of google-appengine-java@googlegroups.com - 8 Messages in 5 Topics

2012-01-14 Thread Jeff Schnitzer
Google has said in the past that there is no negative performance effect from enabling XG transactions if you only use a single entity group. The rational thing to do is always enable XG transactions all the time, but be aware when you span multiple entity groups in a transaction. I think it was

Re: [appengine-java] An O/R mapper to use with cloud sql

2012-01-02 Thread Jeff Schnitzer
On Mon, Jan 2, 2012 at 12:22 PM, Arjan arjan.br...@gmail.com wrote: I keep getting errors of the hibernate configuration that can not read 'public static' properties. It's due to some reflection limitation on the GAE. So there may be a more fitting OR mapper to use in GAE. Are you absolutely

Re: [appengine-java] An O/R mapper to use with cloud sql

2011-12-31 Thread Jeff Schnitzer
If you are using cloud sql, what's wrong with hibernate? Jeff On Sat, Dec 31, 2011 at 4:24 AM, Arjan arjan.br...@gmail.com wrote: Hi all, When google announced cloud sql i was realy happy to port my web application from the nosql datastore to the MySQL instance. I'm a bit used to hibernate,

Re: [appengine-java] Re: Any inputs for How to use the DataStore for Java?

2011-12-28 Thread Jeff Schnitzer
It won't be a problem with the Low-Level API. It works just fine with exotic unicode strings. Your problem is somewhere else in your app, probably somewhere that you are generating the string or rendering it. Jeff On Wed, Dec 28, 2011 at 10:59 AM, Doug douganderson...@gmail.com wrote: Those

Re: [appengine-java] Re: Problem starting development after adding backends.xml

2011-12-22 Thread Jeff Schnitzer
I don't think very many people are using backends... and since a lot of the support here comes from the community, your question is a little esoteric. Try disabling appstats on the backend. Maybe appstats is incompatible with backends. Jeff On Thu, Dec 22, 2011 at 3:53 AM, Rick Mangi

Re: [appengine-java] datastore.delete() not deleting objects and not generating exceptions

2011-12-11 Thread Jeff Schnitzer
Yeah, if you're doing a delete by previously-known key I would expect constant time operation. I have no idea what might be causing your issue, sorry. Jeff On Sun, Dec 11, 2011 at 4:26 PM, Michael michael.ol...@gmail.com wrote: By what you wrote, I assume you're referring to the query speed,

Re: [appengine-java] datastore.delete() not deleting objects and not generating exceptions

2011-12-10 Thread Jeff Schnitzer
I haven't looked too closely at your problem, but one thing that has come up before on this list is that it's a bad idea to do things like this: repeat { delete N items; } Basically, deleting items just flags them as deleted in the underlying store. They are vacuumed up later. So if you

Re: [appengine-java] Re: An alternative to Objectify?

2011-12-08 Thread Jeff Schnitzer
On Thu, Dec 8, 2011 at 5:11 AM, datanucleus andy_jeffer...@yahoo.comwrote: Well, it sounds so. But does (or can/will) Datanucleus support at least some of the features Jeff has mentioned? Yes, several actually. And nothing is dog slow, ... though why dogs get such a bad reputation I've no

Re: [appengine-java] Re: An alternative to Objectify?

2011-12-08 Thread Jeff Schnitzer
On Thu, Dec 8, 2011 at 10:32 AM, Jeff Schnitzer j...@infohazard.org wrote: I'm sorry, but you clearly haven't actually used this cache in the real world. I've looked at DataNucleus' Level2Cache interface and noticed that it serializes requests entity-by-entity. This means that every single

Re: [appengine-java] Re: An alternative to Objectify?

2011-12-07 Thread Jeff Schnitzer
On Wed, Dec 7, 2011 at 7:03 AM, datanucleus andy_jeffer...@yahoo.comwrote: *Does anybody know whether it's as simpletransparent as with Objectify to use GAE's caching in JPA2Datanucleus on GAE?* Assuming you're using v2 of the datanucleus-appengine plugin, just define the persistence

Re: [appengine-java] Re: prefetchSize vs chunkSize

2011-11-20 Thread Jeff Schnitzer
Alfred made this comment, which sheds a lot of light on it: On Sat, Nov 19, 2011 at 8:56 PM, Alfred Fuller arfuller+appeng...@google.com wrote: if you set chunkSize and not prefetchSize every call will try to return chuckSize results (even the first call) if you set prefetchSize and

Re: [appengine-java] Which java framework to use?

2011-11-20 Thread Jeff Schnitzer
My favored stack right now: Objectify (I'm the lead developer, so this one was easy) Resteasy + Htmleasy + Guice, tightly integrated Cambridge templates (with JEXL) Coffeescript + jQuery + Handlebars.js on the frontend Heavy use of Jackson to jsonify objects, including embedded in the page html

Re: [appengine-java] Re: package javax.servlet.annotation does not exist

2011-11-20 Thread Jeff Schnitzer
Occasionally eclipse gets stupid like this. Some combination of Project Clean, closing/opening the project, and closing/opening eclipse eventually sets it right. Jeff On Sat, Nov 19, 2011 at 10:23 PM, Mauricio Porras P mauricio.porra...@gmail.com wrote: I'm having this same issue, did you

Re: [appengine-java] Is BlobInfoFactory thread safe?

2011-11-20 Thread Jeff Schnitzer
Look at the source code. BlobInfoFactory only has one piece of member data, a DatastoreService. So if DS is thread-safe, so is BIF. I don't know if DS is thread-safe though. Jeff On Sat, Nov 19, 2011 at 9:20 AM, Marcel Overdijk marceloverd...@gmail.comwrote: Is BlobInfoFactory thread safe?

[appengine-java] prefetchSize vs chunkSize

2011-11-18 Thread Jeff Schnitzer
Would someone explain the exact differences between prefetchSize and chunkSize and how these interact? The documentation doesn't really help. I presume chunkSize is the number of items that get fetched on each RPC to the datastore backend, equivalent to a fetch() call in python-land. Is

Re: [appengine-java] Google App Engine custom domain stuck in Activate this service infinite loop

2011-11-15 Thread Jeff Schnitzer
This is going to sound really weird but I've had issues like this before and switching browsers seemed to fix it. Try Firefox. Jeff On Tue, Nov 15, 2011 at 2:49 PM, MH zvi.schrei...@gmail.com wrote: Hi there. I am trying to add a custom domain from the GAE Application Settings. It

Re: [appengine-java] Re: Serializing crypto classes

2011-11-15 Thread Jeff Schnitzer
On Tue, Nov 15, 2011 at 10:30 PM, Gerald Tan woefulwab...@gmail.com wrote: Only Serializable objects can be used as properties, and Cipher isn't serializable. The solution is to store the algorithm as a String, then reinstantiating the Cipher from the algorithm string after retrieving it from

Re: [appengine-java] Re: Templating Frameworks

2011-11-13 Thread Jeff Schnitzer
I've used many different Java templating systems over the years and been terribly disappointed with all of them... until recently. I started using Cambridge Templates: http://code.google.com/p/cambridge/ Here's a good intro document: http://blog.erdincyilmazel.com/ The docs are a little rough

Re: [appengine-java] Re: Number of writes per second limitation

2011-11-04 Thread Jeff Schnitzer
On Fri, Nov 4, 2011 at 3:56 PM, Ikai Lan (Google) ika...@google.com wrote: If all 4000 entites are in a single entity group, in theory you can do this because it counts as a single transactional write. There's a maximum RPC size of 11mb (implementation detail) so if you trip this, you're in

Re: [appengine-java] Re: many different Properties or Entity for name/value pairs

2011-11-04 Thread Jeff Schnitzer
FWIW you can declare a field @Embedded MapString, anything in Objectify and this will create a series of native properties in the low level entity. Ie: class MyEntity { @Id Long id; @Embedded MapString, Long strings; } If the map contained { prop1 : 12 } then you could filter by:

Re: [appengine-java] HRD - Eventual consistency

2011-10-31 Thread Jeff Schnitzer
There's this: https://groups.google.com/d/msg/google-appengine/NdUAY0crVjg/3fJX3Gn3cOYJ Some thoughts: * I don't think there's really anything to save you from eventuality except careful consideration during the design of your data model. Pick your entity groups carefully and use XG

Re: [appengine-java] Optimizations with Datastore + JDO

2011-10-28 Thread Jeff Schnitzer
You shouldn't see any significant performance difference between the various java datastore APIs during normal operation. The JDO's classpath scanning may have a significant startup cost, but during execution the costs of RPCs and protocol buffer manipulation tend to dominate over any

Re: [appengine-java] Does JDO auto remove indexes that now have unindexed properties?

2011-10-27 Thread Jeff Schnitzer
Wild guess: If you load an entity in JDO and set a property to be exactly what it was before, this won't actually flag the entity as dirty. It won't be written when you commit the session. I don't know how you would force a write using JDO. Jeff On Thu, Oct 13, 2011 at 6:23 AM, markabrucey

Re: [appengine-java] with the 1.5.5 sdk, the local_db.bin file format has changed

2011-10-21 Thread Jeff Schnitzer
The local dev database frequently gets invalidated at SDK upgrades. It should not be considered a reliable store. No data is trashed in the production database. Jeff On Fri, Oct 21, 2011 at 10:28 AM, Tommy Fannon tfan...@gmail.com wrote: It seems to need to be deleted and re-created. I

Re: [appengine-java] Using overloaded put, get, and delete methods for transactions?

2011-10-21 Thread Jeff Schnitzer
Implicit transactions are a separate issue (and a terrible idea). When the low-level api starts a transaction, it stores this txn as a thread local. The get()/put()/etc methods that don't have a transaction are effectively this logic: Entity get(Key key) { if (there is a thread local txn)

Re: [appengine-java] Re: JDO and XG transactions - Performance

2011-10-18 Thread Jeff Schnitzer
Ugh. Is this a correct further paraphrase? * You wanted users to get exceptions which would force them to go to the manual to figure out why they are getting exceptions. There are less intrusive ways to force users into reading the manual! I really hate software like this; it's a bozo bit that

Re: [appengine-java] Re: JDO and XG transactions - Performance

2011-10-18 Thread Jeff Schnitzer
On Tue, Oct 18, 2011 at 9:07 AM, David Gay (Google) d...@google.com wrote: One other consideration: XG transactions do not work on master/slave. While the default could be different depending on whether HRD is used, that definitely has drawbacks. How can I detect if running on HRD vs M/S? I

Re: [appengine-java] Re: JDO and XG transactions - Performance

2011-10-18 Thread Jeff Schnitzer
/DatastoreService.java#388 On Tue, Oct 18, 2011 at 2:02 PM, Jeff Schnitzer j...@infohazard.org wrote: Dunno about production, but the developer mode takes issue with it. Even a simple single-entity transaction commit fails in M/S mode with the test harness - ie, without calling this method

Re: [appengine-java] Re: JDO and XG transactions - Performance

2011-10-17 Thread Jeff Schnitzer
On Mon, Oct 17, 2011 at 9:38 AM, David Gay (Google) d...@google.com wrote: Just enable it for the whole app: an XG transaction that touches only a single entity group has exactly the same performance as a non-XG transaction. In that case, why is this a configurable setting? Jeff -- You

Re: [appengine-java] Re: Facebook Login integration with java appengine

2011-10-13 Thread Jeff Schnitzer
Facebook recently (as in, a couple weeks ago) made it possible to associate multiple domains with your application. Add my-appid.appspot.com as well as aggenda.com. Jeff On Thu, Oct 13, 2011 at 3:09 AM, marco marco.m.al...@gmail.com wrote: Hi guys. (i'm newbie) I'm trying to make an existing

Re: [appengine-java] Re: Facebook Login integration with java appengine

2011-10-13 Thread Jeff Schnitzer
It's in the Basic Info section in the upper part of the page (which you scrolled past). Look for App Domain - you can put multiple entries there. Jeff On Thu, Oct 13, 2011 at 4:01 AM, marco marco.m.al...@gmail.com wrote: Hi Jeff. Thanks for the quick reply. I'm looking in Apps Aggenda Basic

Re: [appengine-java] Re: Facebook Login integration with java appengine

2011-10-13 Thread Jeff Schnitzer
Are you typing in a field called App Domain? It lets you type in multiple domains, one next to the other. Jeff On Thu, Oct 13, 2011 at 4:39 PM, marco marco.m.al...@gmail.com wrote: Hi Jeff. I've tried out what you suggest but it gives an error: my-app-id.appspot.com must be derived from your 

Re: [appengine-java] GeoPt vs. two floats

2011-10-04 Thread Jeff Schnitzer
It really isn't a big deal. Store two floats if you want. The reasons I use GeoPt a lot: * GeoPt shows up nicely in the datastore viewer * You can store arrays of GeoPt * The storage cost should be slightly less than two floats because there is metadata only for one field. * Maybe some day

Re: [appengine-java] Automatically increment version number

2011-10-04 Thread Jeff Schnitzer
Keep in mind you only get 10 versions, so you'll have to delete old ones as you add new ones. I don't see any way to automate that. Jeff On Tue, Oct 4, 2011 at 2:09 AM, Manilal libreg...@gmail.com wrote: I'm using the commandline version of appcfg command to deploy Java application. I would

Re: [appengine-java] Re: how to bring a huge (http error 413) amount of data to the server?

2011-10-04 Thread Jeff Schnitzer
That doesn't sound right. You should have no trouble POSTing 20k to GAE. I wouldn't be surprised if GETs choked though. Jeff On Tue, Oct 4, 2011 at 9:47 AM, Gerald Tan woefulwab...@gmail.com wrote: You'll have to use the Blobstore API and allow the user to upload the xml into the blobstore

Re: [appengine-java] Re: GAE - Vote counting system

2011-10-03 Thread Jeff Schnitzer
On Mon, Oct 3, 2011 at 9:24 AM, Mat Jaggard matjagg...@gmail.com wrote: Jeff - I'm a bit confused. I thought that the whole idea of the datastore was that you could read or write as much as you want, as fast as you want as long as they are not related? So one datastore write per vote (and

Re: [appengine-java] Re: GAE - Vote counting system

2011-09-30 Thread Jeff Schnitzer
Assuming the goal is massive write throughput, you don't want to do 1 write per vote. You need to aggregate writes into a batch - you can do that with pull queues, but then you're limited to the maximum throughput of a pull queue. And the biggest batch size is 1,000 which might actually be

Re: [appengine-java] Re: GAE - Vote counting system

2011-09-29 Thread Jeff Schnitzer
It's hard to say here if we're talking about the same thing, but here's how I would do it: * Updates go through to the backend, which stores write deltas in ram (not the total). * Reads read-through memcache into the datastore. * The backend writes deltas to the datastore in batch, updating

Re: [appengine-java] Re: Transactions on entities in different entity groups ?

2011-09-29 Thread Jeff Schnitzer
A month or so ago Google posted a request for trusted testers for this feature. It helps to read the mailing lists :-) Jeff On Thu, Sep 29, 2011 at 2:05 AM, Cyrille Vincey cvin...@qunb.com wrote: SERIOUSLY??? (why don't you give any visibility on this key feature, like you did with FTS?)

Re: [appengine-java] GAE - Vote counting system

2011-09-27 Thread Jeff Schnitzer
Have you looked into using pull queues combined with sharded counters? If you lease 1,000 tasks at a time you can probably condense writes considerably. There are several different sharding strategies you could take, even just one master set of shards each of which covers all items. 100 shards

Re: [appengine-java] Re: GAE - Vote counting system

2011-09-27 Thread Jeff Schnitzer
Yeah, messy. I'd use a backend for this. Possibly a set of backends if you need to shard the data for write volume. I'd use Memcache only to cache the count reads. The basic entity is just an id and a count. An increment request goes to a backend, which simply tracks the change. A batch

Re: [appengine-java] Re: The *real* cost of the billing changes

2011-09-10 Thread Jeff Schnitzer
Amazon does something like this for its excess capacity: http://aws.amazon.com/ec2/spot-instances/ ...but it's just for excess capacity. I wouldn't want this kind of behavior for my core usage because it would make my bill highly unpredictable. And it does not incentivize Google to add more

Re: [appengine-java] Re: Objectify-Appengine 2.1 released, supports Partial Indexes

2011-08-12 Thread Jeff Schnitzer
Specifically, it's the difference between calling Entity.setProperty() or Entity.setUnindexedProperty() on the low level API. Objectify interacts with the datastore through the low level API, so there is always underlying support for features. Not necessarily in a convenient form, however. Jeff

Re: [appengine-java] Appengine Rest Services

2011-08-12 Thread Jeff Schnitzer
I'm pretty happy with Resteasy: http://www.jboss.org/resteasy There was a minor trick to getting Guice integrated nicely (see my recent post on the resteasy-developers list). Other than that it works great, and you can use it to render HTML pages with a little extension:

Re: [appengine-java] Joins!

2011-08-05 Thread Jeff Schnitzer
On Fri, Aug 5, 2011 at 11:45 AM, William Levesque billleves...@gmail.com wrote: But is Google's position that all data should be denormalized? I don't think anyone would say that. I wrote up my thoughts around this subject here:

Re: [appengine-java] Contains query Low Level datastore

2011-07-19 Thread Jeff Schnitzer
It's actually much simpler than you realize. Assuming that your ListLong is an indexed property named 'friends', you just need to filter entities for 'friends' equal to your id. This is how list indexes work - if you filter on the property, the entity matches if any value in the list matches the

Re: [appengine-java] Re: appspot.com without www domain names - but not a general word to remember and refer to -

2011-07-17 Thread Jeff Schnitzer
Shrug. Given that any domain chosen cannot have google as part of it (G doesn't want to convey any kind of public endorsement of our applications), appspot seems about as good as any. Sure cloud.com would be nice but I'm pretty sure the people already using cloud.com have other ideas on the

Re: [appengine-java] Απ: ANN: BatchFB 2.1, a Java Facebook library with automatic batching

2011-07-15 Thread Jeff Schnitzer
On Thu, Jul 14, 2011 at 10:59 PM, George Moschovitis george.moschovi...@gmail.com wrote: Thanks for the release, but why don't you use Future? The Future? interface is fundamentally defective because it uses checked exceptions. Checked exceptions are one of the handful of gross design flaws

Re: [appengine-java] Best way to work with a CellTable and many-to-one relationship with objectify?

2011-07-12 Thread Jeff Schnitzer
The public/private/protected/package status of java fields is 100% irrelevant from a security perspective. It's just there to help keep your code clean. The data is still being passed across the wire in a simple, easily-decoded protocol that any sniffer can translate. If you're passing

[appengine-java] ANN: BatchFB 2.1, a Java Facebook library with automatic batching

2011-07-11 Thread Jeff Schnitzer
This is the first time I've announced this opensource project in this forum because until now there wasn't anything GAE-specific about it. You can still use BatchFB outside of GAE, but now BatchFB will use asynchronous fetching on App Engine to issue multiple batches to Facebook in parallel.

Re: [appengine-java] Re: Fast MVC Framework for GAE?

2011-07-09 Thread Jeff Schnitzer
If you're just looking for an MVC framework, here are two that I wrote: Tagonist: http://www.tagonist.org/ This is about as simple as it can possibly get. Less than 500 lines of code, no classpath scanning, zero (as in nada, none, zip) effect on startup time. It just plays clever with JSP.

Re: [appengine-java] Can we get lat/long data for given address ?

2011-07-06 Thread Jeff Schnitzer
Google geocoding. Jeff On Wed, Jul 6, 2011 at 10:14 AM, oceandrive rams...@gmail.com wrote: Can anyone tell me if there is a way to get latitude/longitude values for a given address -- You received this message because you are subscribed to the Google Groups Google App Engine for Java

Re: [appengine-java] Changing datastore property type

2011-07-06 Thread Jeff Schnitzer
On Wed, Jul 6, 2011 at 11:50 AM, jMotta jayrmo...@gmail.com wrote: Jamie, I don't know if get it right, but if it's part of the key it is not possible to change. It sounds like the id is being stored as a field/property of an entity - it's basically a foreign key. If this property is not

Re: [appengine-java] Why are app instances killed after 9000 requests

2011-07-05 Thread Jeff Schnitzer
This is an interesting issue. Hopefully someone at Google has considered this when making changes to the scheduler. Instance death after 9k requests is a known behavior of GAE (well, known if you read these forums - I don't think there is any official documentation). As a wild guess on my part,

Re: [appengine-java] Big Entities vs Small Entities

2011-07-05 Thread Jeff Schnitzer
I think John has the right idea... but since you probably don't need dynamic form fields, I would probably do it like this: class MyEntity { @Id String uid; @Embedded FormField field1; @Embedded FormField field2; ...etc } (No doubt for Twig you just substitute @Embed for Objectify's

Re: [appengine-java] Best way to work with a CellTable and many-to-one relationship with objectify?

2011-07-05 Thread Jeff Schnitzer
That's what I tend to do. Since I write consumer apps, passing my raw entities across the wire usually isn't an option in the first place - there's usually too much security-sensitive stuff, or at the very least data I would rather not disclose to a potential attacker. Also... using entities

Re: [appengine-java] Best way to work with a CellTable and many-to-one relationship with objectify?

2011-06-28 Thread Jeff Schnitzer
To be honest, whenever I start to have a less-than-perfect match between my entities and the content rendered in my client, I usually create DTOs. It's mildly annoying, but it results in a clean interface without a lot of extra payload (some of which might be security-sensitive) sent across the

Re: [appengine-java] Why the 10 second URL fetch limit?

2011-06-17 Thread Jeff Schnitzer
The problem with using the Channel API this way is not quota usage, it's the rather significant amount of work required on the client server to make this work. Making an ajax call from client javascript is trivial. Making an ajax call PLUS setting up, tearing down, and processing messages from

Re: [appengine-java] How to design an in-memory cache?

2011-06-15 Thread Jeff Schnitzer
Sounds like you could just keep it around for 30 minutes or an hour, refreshing it after it expires. What would be nice is to be able to perform the refresh asynchronously. Say it takes 5s to build a piece of data that every appserver uses for practically every request. Instead of relying on

Re: [appengine-java] Re: When to use a String as ID? (using Objectify and GWT)

2011-06-15 Thread Jeff Schnitzer
One case is when you need to save several different entities that all reference each other. Let's say A has a reference to B, and B has a reference to A. If you relied on key autogeneration you would need 3 puts: * Put A (which initializes the id) * Set B.a to the id of A, put B * Set A.b to

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org wrote: Thank you for fixing the benchmark. I am very curious.  According to this new benchmark - it's hard to tell without pushing the buttons a lot of times, but there seems to be a trend - Slim3 is somewhat faster than

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
://slim3demo.appspot.com/performance/ As a result, LL is as fast as slim3 (^^; Yasuo Higa On Thu, Jun 9, 2011 at 10:17 AM, Jeff Schnitzer j...@infohazard.org wrote: Thank you for fixing the benchmark. I am very curious.  According to this new benchmark - it's hard to tell without pushing

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing reflexion vs byte-code generation or anything like that, apt generates code, is not a runtime technology. Like or not reflexion is known to be slower than actually writing the code. This is entirely

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
in the SDK and submit a patch. Jeff On Thu, Jun 9, 2011 at 5:16 AM, Jeff Schnitzer j...@infohazard.org wrote: Ok - so what you're saying is that the extra call to list.size() before iterating through the list makes list iteration faster?  Oddly enough, this does seem to make a difference

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-09 Thread Jeff Schnitzer
/Harry_Potter_and_the_Methods_of_Rationality Until Google makes a change, maybe the other frameworks should try the same trick? On Thu, Jun 9, 2011 at 8:31 AM, Jeff Schnitzer j...@infohazard.org wrote: On Thu, Jun 9, 2011 at 1:32 AM, Gal Dolber gal.dol...@gmail.com wrote: I am not comparing

Re: [appengine-java] Re: R: Re: New GWT/App Engine site

2011-06-08 Thread Jeff Schnitzer
On Tue, Jun 7, 2011 at 11:07 PM, Giorgio Riccardi giorgio...@gmail.com wrote: Hi Jeff, 1) Polishing the navigation, as you said, it kind thought with GWT, but I absolutely agree with you that it needs to be improved... already some changes have been done, but going back at the same position

Re: [appengine-java] Request was aborted after waiting too long to attempt to service your request.

2011-06-08 Thread Jeff Schnitzer
the most confusing ... Ikai Lan Developer Programs Engineer, Google App Engine Blog: http://googleappengine.blogspot.com Twitter: http://twitter.com/app_engine Reddit: http://www.reddit.com/r/appengine On Wed, Jun 8, 2011 at 1:50 PM, Jeff Schnitzer j...@infohazard.org wrote

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Jeff Schnitzer
You are wrong. Try adding getProperty() calls to your LL performance test, and the speed advantage of the LL API goes away. I don't know what to say about Slim3, but here's my test case: http://code.google.com/p/scratchmonkey/source/browse/#svn%2Fappengine%2Fperformance-test I created 10,000

Re: [appengine-java] Does AppEngine support Servlet 3.0 ServletContainerInitializer

2011-06-08 Thread Jeff Schnitzer
Unfortunately GAE does not support Servlet 3.0. Please star: http://code.google.com/p/googleappengine/issues/detail?id=3091 Jeff On Wed, Jun 8, 2011 at 11:45 AM, Marcel Overdijk marceloverd...@gmail.com wrote: Does AppEngine support Servlet 3.0 ServletContainerInitializer for code-based

Re: [appengine-java] Re: Is the native API really so much faster than JDO and slim3?

2011-06-08 Thread Jeff Schnitzer
, Gal Dolber gal.dol...@gmail.com wrote: Slim3 is not only fast, the api is completely awesome. It has been my choice for a year now for all gae projects. It includes name safety and and amazing querying utils. Very recommendable! On Wed, Jun 8, 2011 at 3:41 PM, Jeff Schnitzer j...@infohazard.org

  1   2   3   >