Last Wednesday, the App Engine team hosted the latest session of its bimonthly IRC office hours. A transcript of the session and a summary of the topics covered is provided below. The next session will take place on Wednesday, September 16th from 9:00-10:00 a.m. PDT in the #appengine channel on irc.freenode.net.
--SUMMARY----------------------------------------------------------- - Q: Where can I find information on per-minute and daily quotas, maximum number of simultaneous requests, etc.? A: The Quotas doc at http://code.google.com/appengine/docs/quotas.html lists most per- minute and daily quotas. Task queue quotas and limits are listed in http://code.google.com/appengine/docs/python/taskqueue/overview.html#Quotas_and_Limits. You can also check the Quota Details page in the Admin Console to see quota information for a particular application. [7:01, 7:03] - JSF 2.0, ICEFaces 1.8.1 compatibility with App Engine -- several developers have written posts on using JSF 2.0 with App Engine (see http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine), but it appears that ICEFaces is not currently compatible. [7:04-7:07, 7:11, 7:14-7:15, 7:42, 7:56] - Tip: one way to optimize your application, especially when you're working with large entities (i.e. entities with large BLOB properties) is to create separate entities, placing the BLOB data in its own entity and referencing this from the original entity. This way, retrieving the entity doesn't necessarily pull down the large BLOBs, although they're available should you need them later -- think of a listing of file metadata (i.e. name, size, hash) where the actual file is only loaded when the user clicks on a particular filename. [7:19, 7:21] - http://code.google.com/p/googleappengine/issues/detail?id=1444 strikes again -- (Java) deployment fails because of a vague "more than 100 URLMap entries in application configuration" error. We'll work on trying to reproduce this internally, but if you're seeing this, please create a new thread in the group with the contents of your web.xml and appengine-web.xml file and the number of JSPs and static files your application includes. [7:22, 7:26, 7:28, 7:36-7:38, 7:41] - Q: Is there support for HTTPS on custom domains? A: We're waiting on broader support for SNI before we implement this. [7:26, 7:28, 7:30] - If your application experiences a denial of service attack, your application may stop serving if requests come in at 500 qps or above, the request limit for applications with billing enabled. Currently, you can filter requests by IP address if an attacker is primarily using one or several machines. The team is planning a better API for dealing with potential attacks, but unfortunately it wasn't prioritized highly enough to make it on the latest roadmap. [7:35, 7:40, 7:49] --FULL TRANSCRIPT--------------------------------------------------- [7:00pm] Jason_Google: Welcome Everyone! I'll be here for the next hour or so to answer any App Engine questions, so don't hold back. [7:00pm] joakime: [7:01pm] kaminoo: First time for me here, do we just fire off our questions? [7:01pm] Jason_Google: Yep, fire away. [7:01pm] kaminoo: Wanted to ask about Task Queues [7:01pm] joakime: Java - where can I find information about Simultaneous Request limits and quota details? i've been hitting it recently. even tho i have a $20 budget set and less than 1% utilization on all other quota constrained stuff. [7:03pm] maxoizo: Hi Jason! Can google team update offline docs with actual documentation? (last update May 8, 2009). [7:03pm] kaminoo: Python: Task Queues. Me and some other guys have posted a lot of bugs we have found in the queue system. My bugs in particular have been more than a month there, not even acknowledge. Do we bother keep testing it? Are we doing a good job at it? Will we have an answer? [7:03pm] Jason_Google: joakime: It looks like http://code.google.com/appengine/docs/python/taskqueue/overview.html#Quotas_and_Limits is the only thing that's available for now. [7:03pm] Jason_Google: maxoizo: Thanks for the suggestion. I'll try to do this for you soon, hopefully before the end of the week. Adding it to my to-dos right now. [7:04pm] maxoizo: 2Jason_Google: tnx! [7:04pm] guest07: i used the dev server for java [7:04pm] Jason_Google: kaminoo: Yes, please keep filing your bugs. We're not doing the best job acknowledging them on the public tracker, but we are tracking them internally. [7:04pm] guest07: and it seems that appengine doesn't support jsf 1.2 [7:05pm] gae-peon: ? [7:05pm] Jason_Google: guest07: Have you seen http://groups.google.com/group/google-appengine-java/web/will-it-play-in-app-engine? [7:05pm] kaminoo: Jason_Google: Great to know. The feature is GREAT. But unusable with the current problems. Plus, we had a lot of suggestions to make it even better. Hopefully we'll see some updates soon enough. tnx! [7:06pm] Jason_Google: guest07: Which links to https://sites.google.com/a/wildstartech.com/adventures-in-java/Java-Platform-Enterprise-Edition/JavaServer-Faces/sun-javaserver-faces-reference-implementation/configuring-jsf-20-to-run-on-the-google-appengine [7:06pm] gae-peon: Is 1.2.5 coming soon? [7:06pm] Jason_Google: kaminoo: Sure thing, and thanks for sending your feedback. Please keep doing so and encourage other developers to star the issues that you're reporting. [7:06pm] Jason_Google: gae-peon: Yes, pretty soon. [7:07pm] guest07: java.lang.AbstractMethodError: javax.servlet.jsp.JspFactory.getJspApplicationContext(Ljavax/servlet/ ServletContext;)Ljavax/servlet/jsp/JspApplicationContext; [7:07pm] joakime: Jason_Google: any news on Tasks / Queues / JMS-like apis for us on Java? [7:07pm] guest07: that is the error message from the dev server [7:08pm] gae-peon: Will there be better documentation on capability module in 1.2.5? [7:08pm] Jason_Google: joakime: Task Queues should be included in the next Java release. [7:08pm] LenT: For Java, is there any way with JDO, JPA or low level API to get the count of the number of entities of a kind if it is over 1000? In development environment, I get the right answer (5002) but in production I get 1000. [7:08pm] gae-peon: I mean google.appengine.api.capabilities... [7:09pm] Jason_Google: guest07: I haven't tried it myself, but I pointed the link out since other developers have seemingly had better luck getting it to run on App Engine. If you're having problems, I suggest pasting your stack trace into a new thread in the discussion group in case another developer has run into the same problem. And if I find an answer, I'll update the wiki page I linked to earlier. [7:11pm] Jason_Google: LenT: For now, I don't believe so. The disparity between the production and local environments is a bug. Please feel free to file a new report in our tracker. [7:11pm] objectuser: guest07: i've not tried 1.2 but i have jsf 2.0 working [7:11pm] Jason_Google: LenT: Have you tried using an Extent in JDO? [7:11pm] objectuser: the google group has a link of someone with a very good post on how to set it up [7:11pm] objectuser: not sure if it has 1.2-specific info though [7:12pm] joakime: Jason_Google: any possibility that future Downtime notifications (in the rather handy google-appengine-downtime group) will include Java details for the CapabilitySet and CapabilityDisabledError recommendations? [7:12pm] objectuser: but jsf 2.0 is better. [7:12pm] Jason_Google: objectuser: Is the link in the Will it Play in App Engine wiki? If not, go ahead and send it to me and I'll add it. [7:12pm] gae-peon: Anyone know a good way to handle counters (not necessarily sharded) using tasks that doesn't require memcaching for dirty flag? Big problem is the required idempotency of task handlers. [7:12pm] objectuser: i'll see if i can find it [7:13pm] Jason_Google: joakime: I noticed this yesterday myself. I'll add another to-do. [7:14pm] LenT: I haven't tried using Extent in JDO as I'm currently using JPA but I will look into it. I read in some post that even if we ask for specific fields in a query, appengine actually fetches all the fields of a kind, is this true? [7:14pm] objectuser: JSF thread: http://groups.google.com/group/google-appengine-java/browse_thread/thread/5251d6add8373a88/1eaf97a9d2be5b04 [7:14pm] objectuser: specific link of coolness: https://sites.google.com/a/wildstartech.com/adventures-in-java/Java-Platform-Enterprise-Edition/JavaServer-Faces/sun-javaserver-faces-reference-implementation/configuring-jsf-20-to-run-on-the-google-appengine [7:15pm] guest07: my problem is that i am using icefaces 1.8.1 and there is a jsf 1.2 dependency [7:15pm] Jason_Google: LenT: It will fetch all fields in the default fetch group, which includes all core types including strings, dates, and numbers. You can add lists and user-defined types to the default fetch group explicitly. [7:15pm] kaminoo: Jason_Google: Could the roadmap be updated? I would love to know on what you are working on, when are more things coming to Labs, and when do you plan to launch Task Queues out of a beta [7:15pm] Jason_Google: objectuser: Thanks [7:15pm] guest07: objectuser: thanks I will look at the posts [7:16pm] joakime: kaminoo: is there a "appengine labs" project floating around somewhere? ooh. [7:16pm] kaminoo: joakime: Task Queue is [7:16pm] Jason_Google: kaminoo: The roadmap isn't updated very often -- a few times a year. We updated it in June, I believe. We'll remove things from the roadmap as new features are launched, but it may be a few more months before we revise it again. [7:17pm] objectuser: no problem. and if it's a possibility (i realize there are often other issues), JSF 2.0 was largely backward compatible for me (was coming from myfaces 1.1) and it has all of the annotation goodness as well as facelets [7:17pm] kaminoo: Jason_Google: tnx [7:17pm] Jason_Google: kaminoo: But there's a lot of cool stuff on there as it is. [7:17pm] guest07: i already setup my vps on rackspace, but if GAE works then I might switch over [7:19pm] LenT: Jason_Google: thanks for the information. I read in another postings that if an kind includes a blob field that if the blob was separated out into another kind that updates on the original kind would be faster since all fields are updated. Is this still the case if the entity the blob was separated out into is owned by the main entity? [7:20pm] joakime: Jason_Google: how is the limit on urls counted? (I keep hitting a warning on 100 url limit on deploy) is there docs somewhere detailing it? recommendations on working with/around it? [7:20pm] guest07: objectuser: i am playing around with the devserver right now. I will let you know if I have any success [7:21pm] Jason_Google: LenT: Yes, that's a general best practice, especially if your blobs are large. [7:21pm] Jason_Google: joakime: Sorry, which limit are you hitting? [7:22pm] joakime: There's a warning on deploy (from my workstation to gae server) that the app cannot be deployed due to having more than 100 urls. [7:22pm] LenT: Jason_Google: thanks. I just wanted to make sure that the entity that the blob is in wouldn't get updated due to cascading when the main entity is updated (unless the blob entity is actually loaded and updated). [7:22pm] Jason_Google: joakime: Ah, I see. There was a thread on this in the discussion group at one point. Let me see if I can find it. [7:22pm] ceej_: any reason why http://code.google.com/appengine/docs/python/xmpp/overiew.html#Receiving_Instant_Messages is down ? [7:22pm] joakime: so i find myself juggling servlets, servlet- mappings, making new dispatch servlets, etc ... [7:23pm] Jason_Google: ceej_: Was it ever up? XMPP for Python hasn't been released. [7:24pm] ceej_: O....you should disable the link on http://code.google.com/appengine/docs/users.htm confusing [7:26pm] ceej_: can you have https for your own domain now? [7:26pm] gae-peon: What does it mean that there's a limit of 100 URL's? Is that a Java thing? If not, do you mean 100 routes in an application? [7:26pm] Jason_Google: ceej_: No, not yet. [7:27pm] ceej_: is there a road map for it? [7:28pm] Jason_Google: gae-peon: You can run into a "Found more than 100 URLMap entries" error in Java if you have too many <servlet- mapping> elements in your web.xml file. I'm trying to see what else could cause this error. [7:28pm] joakime: Jason_Google: that can't be right. i have 4 servlets + 4 servlet mappings and i hit that regularly. [7:28pm] Jason_Google: ceej_: No since it's largely outside of our control. Waiting on more SNI adoption. [7:28pm] gae-peon: So this servlet-mapping issue won't crop up in python apps? [7:28pm] Jason_Google: joakime: Right, so there must be another issue. [7:29pm] Jason_Google: gae-peon: I haven't seen this in Python, no. [7:29pm] guest07: let me try jsf 2.0 api and impl in dev [7:30pm] joakime: Jason_Google: this one? http://groups.google.com/group/google-appengine/browse_thread/thread/54860ad2b058f54e/66a6ae2df9f93e74?q=urlmap+entries#66a6ae2df9f93e74 [7:30pm] kaminoo: Jason_Google: Could you explain why https for our own domains it's outside GAE control? SNI stands for what? Sorry, don't know [7:30pm] Jason_Google: joakime: That's one of them. I'm looking for one with more content. [7:31pm] joakime: Jason_Google: there's Issue 1444 as well. but no real details. just a vague mention of a conversation on IRC. (heh) [7:31pm] jcgregorio: kaminoo: http://en.wikipedia.org/wiki/Server_Name_Indication [7:31pm] Jason_Google: kaminoo: This is largely outside the realm of my expertise, but Wikipedia has a pretty good description: http://en.wikipedia.org/wiki/Server_Name_Indication. If you look through our past transcripts, you'll find more information, since this gets raised quite a bit. [7:32pm] Jason_Google: Ah, Joe beat me to it. [7:32pm] jcgregorio: [7:32pm] kaminoo: ha!, tnx both of you, I'll look into the scripts then... [7:33pm] Jason_Google: joakime: How many static files does your application have? [7:33pm] joakime: 1 html. 1 image. and about a dozen files generated from a GWT compile. [7:34pm] joakime: (the GWT compile generates a .js and many cryptic files with long hex codes for names) [7:34pm] joakime: no jsps [7:35pm] gae-peon: Jason, are there tests for the appengine SDK? [7:35pm] maxoizo: Jason, my friend recieved this email http://dpaste.com/88952/ for him online project. What advice would you, if this project will be hosted on AppEngine? Is there some progress in your "firewall", or are still at the level of discussion? [7:36pm] Jason_Google: joakime: How many is "many"? [7:37pm] Jason_Google: gae-peon: All the code that's available now is in the Google Code project at http://code.google.com/p/googleappengine/source/browse/ [7:37pm] joakime: Jason_Google: $ find . -type f | wc -l (33) [7:37pm] joakime: not that many. [7:38pm] joakime: Jason_Google: i've traced the jetty webappcontext, plus configs, along with mappings, jetty level handlers and filters (as present in the dev server) and at most can only come up with 51 possible URL paths. [7:38pm] joakime: <-- works for Webtide (now Intalio) i'm intimately familiar with Jetty. [7:38pm] gae-peon: jason: I had already looked through the SDK but couldn't find any testing files. Do you have separate test files that just aren't checked in to the open source repo? [7:40pm] Jason_Google: maxoizo: Google's back-end has some protections in place -- e.g. your application likely won't go offline -- but you will still be charged for the extra resource usage. The team is planning a better API for dealing with potential attacks, but unfortunately it wasn't prioritized highly enough to make it on the latest roadmap. [7:40pm] LenT: In a posting "JDO request IN" topic in GAE Java group (http://groups.google.com/group/google-appengine-java/browse_thread/ thread/b91fd5f80c1b1f7/b5982556e9662622?hl=en#b5982556e9662622), a datanucleus person said that IN clause is supported at least at the datanucleus level. Is this supported in JDO with GAE? If so, is it supported in JPA? [7:41pm] Jason_Google: joakime: Out of curiosity, do you have an entry in your appengine-web.xml file specifying which files are static and which are resource? (By default, all files are uploaded as both static and resource). [7:41pm] joakime: interesting. no. no i dont. but i will shortly. [7:42pm] pct: hello, [7:42pm] Jason_Google: LenT: I don't know the JPA syntax, but in JDO, you can use .contains. So if you have a list of Strings, you can do something like ... "myList.contains(stringParam)" [7:42pm] Jason_Google: in your queries [7:42pm] Jason_Google: pct: Hi [7:42pm] guest07: so close, but icefaces 1.8.1 with jsf 2.0 is still experimental [7:42pm] guest07: http://www.icefaces.org/JForum/posts/list/14029.page [7:43pm] pct: I've got a GAE WARNING msg: urlfetch_stub.py:268] Stripped prohibited headers from URLFetch request: ['Content-Length', 'Host'] [7:43pm] Jason_Google: joakime: http://code.google.com/appengine/docs/java/config/appconfig.html#Static_Files_and_Resource_Files I don't know if that will help, but I'd be very interested in the result... [7:43pm] objectuser: guest07: bummer ... and you need some of the icefaces components i take it? makes sense. [7:43pm] joakime: Jason_Google: thanks. i found that a smidgen of a second before you pasted it. [7:43pm] pct: could someone give me suggestions? [7:43pm] joakime: Jason_Google: I'll have to get back to you during the end of month chat. (promise) [7:44pm] pct: app engine ver: 1.2.4 [7:44pm] Jason_Google: pct: I'm not much of a Python guy, unfortunately. Are you trying to set these headers yourself? [7:44pm] maxoizo: 2Jason: Tnx. Unfortunately application will go offline, cos attack more then 500req/min... [7:44pm] pct: Jason_Google: [7:45pm] objectuser: LenT: i was interested in the answer to that as well but haven't tried it ... and didn't know if it would be done in the datastore or in the JDO framework. it sure seems like an "in" query, which, i think, means it has to be done in JDO which means it's the same as running several queries, one for each element in the "in" list. happy to be wrong though. [7:45pm] joakime: Jason_Google: one last question, any plans for some sane UrlFetch performance improvements? (like fire-n-forget, or queued up and bulk http/1.1 pipelining)? (need that for integrating with things like Apple's Push Notification servers) [7:45pm] maxoizo: * 500 req/sec [7:45pm] Jason_Google: pct: According to the docs, you can't set these headers yourself: http://code.google.com/appengine/docs/python/urlfetch/overview.html#Request_Headers [7:46pm] kaminoo: interested in joakime question too [7:46pm] Jason_Google: joakime: Task queues are coming for Java soon, which means you won't have to do all of your URL Fetch calls in a single handler. That should help there. [7:46pm] guest07: objectuser: maybe in time [7:47pm] Jason_Google: joakime: Also, the Python API has asynchronous URL Fetch, which should be coming to Java soon, hopefully. [7:47pm] joakime: cool. [7:47pm] maxoizo: 2Jason: any plans for 24/7 support? [7:47pm] joakime: what about http/1.1 pipelining many requests to a single host? [7:48pm] LenT: objectuser: I also hope that is being done in the datastore as well and hope that it is also supported in JPA. [7:49pm] Jason_Google: maxoizo: Yes, unfortunately. The servers aren't actually going down but the DOS protection kicks in at that point and your app will stop serving requests. I'll look into what can be done about this more, but unfortunately, I don't think I have the best answer for you right now. [7:49pm] Jason_Google: maxoizo: No 24/7 support for the time being. I'm all you've got. [7:50pm] pct: Jason_Google: thanks [7:50pm] Jason_Google: joakime: Not sure about that one in particular. Sounds like a feature request for the issue tracker. [7:50pm] joakime: Jason_Google: look into the QoSFilter and DosFilter on the Jetty7 codebase. (they can likely be backported to Jetty 6 easily enough) [7:51pm] maxoizo: 2Jason [7:51pm] joakime: Jason_Google: never mind. they were backported into 6.1.19 it seems. [7:52pm] Jason_Google: joakime: If you don't have a chance to try the static file trick before the end of the session, please post a new question in the group. Your case sounds like one of the most extreme, since you don't have very many mappings or static files. Hopefully that will help us track down the source of this issue. [7:54pm] maxoizo: 2Jason: Can you email me, when you found best answer for my question? (m...@oizo.biz) [7:55pm] Jason_Google: maxoizo: I'm terrible at email. I'd prefer if you post in the group instead. I check that daily, and that way it won't slip off my plate. [7:56pm] guest07: 2Jason: Seems icefaces 1.8.1 w/jsf 1.2 or jsf 2.0 is not compatible with gae [7:57pm] maxoizo: 2Jason: ok, thx. I'll create a group. [7:57pm] Jason_Google: guest07: I was waiting to hear back on IceFaces compatibility. Have you tried any other versions? [7:57pm] Jason_Google: maxoizo: Thank you. [7:57pm] guest07: no 1.8.1 is their latest release [7:58pm] rsaccon: Jason_Google: has anything changed in regard to XMPP BOSH ? Now that wave has been announced and subpub server-server thing, a GAE based comet-client-endpoint solution would be highly desireable. [7:58pm] jcgregorio: joakime: pipelining is fairly hard to do and the fact that it could only be used in a tiny fraction of all requests means there isn't much chance of it getting done [7:59pm] guest07: also some nice features of icefaces like serverpush have to be disabled [8:00pm] guest07: 2jason: I can send an email about this to you if you need it [8:01pm] Jason_Google: rsaccon: Getting outside my comfort area again. The point of PubSubHubbub is to reduce the need for long-lived requests. But XMPP support is coming, as indicated on our roadmap. [8:01pm] maxoizo: Jason: can google team documented "capabilities API"? [8:02pm] Jason_Google: maxoizo: I'll follow up on this too. [8:02pm] Jason_Google: I don't know what the current plans are. [8:02pm] Jason_Google: OK Everyone. Thanks for another great Chat Time. I'll be posting the transcript and a summary to the discussion groups shortly. The next session will be in 2 weeks, Sep. 16 from 9-10 a.m. PDT. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~----------~----~----~----~------~----~------~--~---
