hey guys--
If you're using Tasks and haven't tried public, dynamic Backends yet,
they're wonderful:
- for tiny apps, you can offload another $0.72/day for free (they're
dynamic!)
- for medium apps, it's a super easy way to get more RAM
- easy migration from Tasks running on frontends to running them on
public, dynamic backends (incl development, debugging, etc.)
- nearly identical debugging/production methods as frontend Tasks
- since they're public, you can launch Tasks directly on the backend,
removing the frontend from the equation, e.g. you can test out how a
Task will perform on a backend, before committing to the architecture
change.
- cool trick: keep the fast Tasks running on frontends, and use the
(expensive) backends for RAM-intensive jobs-- this is a tiny code
change, and also makes it easier to debug large jobs because the
console isn't flooded with little jobs. For my app, I use Tasks for
various indexing jobs in an online marketplace-- now, I send the
"large" sellers with 1000s of SKUs, to a B4 backend.
GAE team:
- I've noticed that my backends can 500-error but not produce a stack
trace in the logs? are other people seeing this?
my choice of migration: (python)
1. create backends.xml, define *one* backend, e.g.
backends:
- name: backend2647324 # since it's public, give it a hard-to-
guess name
class: B4 # big+fast without blowing the free budget
options: dynamic, public
2. add backend-enabled queued, by copying queues from frontends:
queue:
- name: image-processor
...other options here...
- name: image-processor-backend # note how I append "-backend"
to the name
target: backend2647324
...other options here...
3. in your code, start firing Tasks at the backend
# see bottom for taskname() trick
taskqueue.add(url=..., name=taskname("some-identifier-i-will-
recognize"),
queue_name="image-thumbnailer"+("-backend" if
is_gigantic_image(...) else ""))
4. modify your deploy script to update both the front & backends.
Mine (legacy) is in perl:
# deploy bar ==> deploys to foo-bar and to foo-bar-backend
# deploy bar-backend ==> deploys to foo-bar-backend only
# deploy bar-frontend ==> deploys to foo-bar only
$DEST = pop(@ARGV);
my $push_frontend = 1;
if ($DEST =~ s/-backend//) { $push_frontend = 0; }
my $push_backend = 1;
if ($DEST =~ s/-frontend//) { $push_backend = 0; }
if ($push_frontend) {
open(FH, "echo ".$ENV{"GAEPASSWD"}."|python2.5 ".$ENV{"GAEDIR"}."/
appcfg.py --email=".$ENV{"GAEUSER"}." --passin --application=foo-$DEST
update .|");
while (<FH>) { print; }
close FH;
}
if ($push_backend) {
open(FH, "echo ".$ENV{"GAEPASSWD"}."|python2.5 ".$ENV{"GAEDIR"}."/
appcfg.py --email=".$ENV{"GAEUSER"}." --passin --application=foo-$DEST
backends . update|");
while (<FH>) { print; }
close FH;
}
that's it! now, some jobs will go to the frontends and others to the
backends.
hope this helps,
adam
def taskname(string):
"""safely create Google App Engine Task names."""
# add timestamp and random for ultimate uniqueness, strip disallowed
chars
rnd_salt = str(random.randint(100000, 999999))
return re.sub(r'[^a-zA-Z0-9-]', '-', string +
str(datetime.datetime.now())) + "-" + rnd_salt
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
