On 5/11/08, Mattias Persson <[EMAIL PROTECTED]> wrote:
> Hmm allright, great that it's working though. Also it would be nice to
> know which structural changes you made to your code to make it work.
What this code does is to take a bunch of ids and for each of them (a)
replicate the musicbrainz entity with that id into the neo store (b)
if the entity has a wikipedia relationship, extract a text snippet
from wikipedia and store along with it. The musicbrainz webservice
throws a 503 if you make more than 1 request per second. Before, I
just threw a ServletException when that happened. Everything wrapped
in a single transaction. Like this:
Transaction tx = Transaction.begin();
try {
for (UUID id : artistIds) {
MushArtist artist = maf.getArtistById(id);
if (artist == null) {
// replicate data from MusicBrainz WebService
try {
Query q = new Query();
Includes inc = new Includes();
inc.include("url-rels");
Artist mbArtist = q.getArtistById(id, inc);
artist = maf.copyArtist(mbArtist);
} catch (WebServiceException e) {
// FIXME: log this error!
//tx.failure();
//continue;
throw new ServletException(e);
}
}
try {
artist.updateWikipediaBlurb();
} catch (WikipediaException e) {
// FIXME: log this error!
tx.failure();
continue;
}
}
tx.success();
} finally {
tx.finish();
}
Now, I have one transaction for every id, so that successfully
replicated data is commited immediate. If there is an exception, I
explicitly call tx.failure() continue with the next id. However, in
the code below I throw an exception and it works just the same (i.e.
in the next request I'm not getting
org.neo4j.impl.core.NotFoundException: Node[40] not found)
for (UUID id : artistIds) {
Transaction tx = Transaction.begin();
try {
MushArtist artist = maf.getArtistById(id);
if (artist == null) {
// replicate data from MusicBrainz WebService
try {
Query q = new Query();
Includes inc = new Includes();
inc.include("url-rels");
Artist mbArtist = q.getArtistById(id, inc);
artist = maf.copyArtist(mbArtist);
} catch (WebServiceException e) {
// FIXME: log this error!
//tx.failure();
//continue;
throw new ServletException(e);
}
}
try {
artist.updateWikipediaBlurb();
} catch (WikipediaException e) {
// FIXME: log this error!
tx.failure();
continue;
}
artists.add(artist);
tx.success();
} finally {
tx.finish();
}
}
As you can see, the only structural change is moving the transaction
inside the loop. It's a bit tricky to reproduce because the webservice
only fails after a certain number of requests and there are long
delays involved with actually accessing it.
Philip
_______________________________________________
Neo mailing list
[email protected]
https://lists.neo4j.org/mailman/listinfo/user
