On 27 Jul 2010, at 00:24, Bill wrote:
This would completely disregard my business model. A Domain is not an
Element, nor vice-versa. It's something analogous to making both
classes Elephant and Corn implement the same interface because both
have Ears.
Ha ha well you would not actually need to have Domain implement
Element to set up the same parent hierarchy. In fact a parent can be
any Key - even one that does not exist. Not sure about JDO, but in
Twig you can set the parent when you store the Element like this:
datasore.store()
.instances(someElements)
.parent(theDomainOrElement)
.returnKeysNow();
So the parent does not actually need to be referenced by the child or
visa versa.
And as you say, "this limits the concurrency of changes to
Elements to a few writes per second per Domain" which would be very
bad in an enterprise system.
It all depends on the granularity of your element groups. If you can
break them down sufficiently then you can still use transactions.
I would advise using either the top level Element or a lower level
element as your root entity. If you must use a lower level element
you would need to remove the list of child keys from a Element to stop
the cascade of updates. Every child has a reference to its parent
encoded in its key which can be used to efficiently return the list of
children using an ancestor query.
Are domains added and removed often? If not then is it sufficient to
check that the domain exists before working on your elements outside
of the transaction? Just never delete Domains - mark them as deleted
instead.
No, they're not added and removed often, and as it happens the Domain
does have an "active" flag. Nonetheless, I think you're missing the
point. If I have to interleave a constraint check with transaction
elements inside my destructive operations, I am defeating the
principle of separation of concerns. I already have to manually
verify "foreign key" relationships, which is blends concerns badly
enough!
Sounds like you have architected your solution to do too much in a
single transaction which is just not performant (or possible) with the
datastore. Time to re-think how you demarcate transactions.
A succinct explanation for a transaction is that it is supposed to
make different operations involving multiple resources atomic to the
level of isolation specified. If you can only have one resource, or
one small group of very tightly coupled resources, then transactions
give you almost no benefit. At this point the only thing I can trust
the transaction layer to do is to provide an optimistic locking
version. Highly unsatisfactory, this is!
The datastore is not an RDBMS. There are certain domain models which
simply will not work on the datastore. If you cannot break down you
data into small independent chunks your app will not work.
I appreciate your goal of not compromising your data model and
architecture for the practicalities of low-level concerns - but it
just won't work here. GAE makes it impossible to ignore that fact
that you are writing for a distributed system. You have to front up
to these problems such as sharding and distributed transactions from
day one.
Even load
operations on a non existent entities (e.g. Domain) are included in
the transaction.
Yeah. Why on earth? You can read a record a billion times and the it
will not "clobber" an update.
The load cannot clobber - its when you depend on the result of that
load to do a write that you would hit a problem.
In order to check that there is no existing entity with the same key
before writing an entity the load must be a part of the transaction or
another request could store an entity with the same key before you
committed - clobber.
Let me put it this way. I'm not looking for a way to hack this into
working. I've already hacked it into working by adding an extra, very
hacky layer to my architecture. What I would like is some sort of
explanation, and hopefully some indication that at some point this is
going to be fixed. I'd even be interested in helping out with said
fix. Google, can you provide me with any answers?
As I pointed out in my earlier reply - there is an RDBMS offering on
the road map:
http://code.google.com/appengine/business/roadmap.html
That sounds like the "fix" you are looking for unless you can bash
your data model into a shape more suited to a distributed system.
--
You received this message because you are subscribed to the Google Groups "Google
App Engine for Java" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
