Hi all, Enjoy yesterday App Engine IRC Office Hours transcript below, next session should happen Wednesday, Aug 3, 2011 7PM PST on irc://irc.freenode.net/#appengine.
(Thanks a lot to dunker and his jsonbot for logging it) -------------------------------------------------------------------------------------------- 2011-07-20 18:01:26 CEST App Engine IRC Office Hours starts -------------------------------------------------------------------------------------------- <dunker> Greg said there will be sample billings for the new billing system in 1.5.2 but cant find it in the test version .. is that still going to be in 1.5.2 ? <moraes> dude what were those real time transcripts from io talks. science fiction those were! <dunker> moraes: would be a nice read filipncs ([email protected]) has joined #appengine <marzia_google> @dunker, we will have sample bills very soon - but not exactly when 1.5.2 is released kytibe ([email protected]) has joined #appengine <robertk> yeah surely there is a nice speak to text engine in g+ hangouts, right? <dunker> ah ok cool <proppy> robertk: not yet <robertk> any of you googlers here familiar with the scheduler changes that have been going on? schuppe (d8ef2d04@gateway/web/freenode/ip.216.239.45.4) has joined #appengine <rwilhelm23> Question on the scheduler - response time is the main variable when it comes to determing whether to spin up more instances, as opposed to cpi_ms and api_ms <dunker> second question is is there already being tinkered with the scheduler ? some ppl report different behaviour of their apps .. quicker shutdown and start of instances <dunker> ah rwilhelm23 is already on track <proppy> robertk: which change have you been experiencing ? <mbw> rwilhelm23: I would hope it would be request latency, using cpu/api ms would not make much sense <marzia_google> response time is a big factor - but also start up time for your app <itsPhilip> Already anyone from billing here? <marzia_google> the question is: would we be better having a request in a pending queue or starting a new instance <dunker> interesting kevlar_mac (~kevlar@nat/google/x-psdjhwmjktkgopat) has joined #appengine <dunker> oi kevlar_google kevlar_google (~kevlar@unaffiliated/eko) has joined #appengine <proppy> dunker: you scared him <dunker> hehe yeah ;] <kevlar_google> Hi. hacked (~vinces@pdpc/supporter/active/vinces) has joined #appengine schuppe_ (d8ef2d04@gateway/web/freenode/ip.216.239.45.4) has joined #appengine <robertk> @googlers ( proppy, marzia_google, tmatsuo_google) see http://groups.google.com/group/google-appengine/browse_thread/thread/baf439a6e073f6da# <dunker> well just out of curiosity how many parameters does the scheduler take into account <robertk> well, ,my post in that thread at least <dunker> and do you have simulators to test it ? <robertk> basically i keep seeing batches of instances get killed then restarted <robertk> it is odd <robertk> seeing on python apps. hr and ms <robertk> also seeing stuff get killed quick <tmatsuo_google> itsPhilip: do you have any question about billing? I'm not from the billing team though <mbw> the last post also mentions that always on instances are not used at all, Ive seen that also <robertk> yeah seen that too <robertk> on an hr app <itsPhilip> I am interested if Google will react to the recent traffic price changes of amazon and msft. <proppy> robertk: gathering some info about the instance behaviour changes <proppy> robertk: I will get back here in a few minutes <marzia_google> it's difficult to diagnose specific scheduler questions in realtime on chat, so I'll forward the thread you referenced - and also if you can provide app ids offline, that will be helpful, and any relevant logging issues <marzia_google> so in summary - need more information, will investigate <robertk> you da man proppy <itsPhilip> [msft & amazon made incoming traffic free and amazon also decreased costs for outgoing traffic] <robertk> marzia_google: yeah not sure i'm seeing specific logging issues. just strange patterns with my instances <proppy> robertk: idle-at-eviction for python had dropped, and we have since added more capacity to adapt <proppy> robertk: let us now if you see similar behaviour in the future <marzia_google> our billing plans have not currently changed from those announced at I/O <nubbius> Hi, i've got a question <robertk> we are all unfortunately aware.... :/ <robertk> nubbius: just ask it <nubbius> Anyone know when the limit of URLFetch expand to more than 1mb? <marzia_google> the request size for URLFetches will be 1mb indefinitely - the response size was lifted to 32mb <proppy> mbw: yes, dynamic instances will get priority over always on instance now <marzia_google> we have nothing new to announce there vinces (~vinces@pdpc/supporter/active/vinces) has joined #appengine <itsPhilip> marzia_google: Don't you expect google to get into the pricing competition? <robertk> proppy: funny thing is that i'm seeing instances actively serving go away and get replaced. low mem usage <uriel> question: when will billing be enabled for Go? <Wooble> nubbius: 2 december 2010 <tmatsuo_google> itsPhilip: As far as I know, I don't think so, especially in a short-term, but in a long run, there is a posibility for us to change in the future because we want to offer competitive price, but we can not compare IaaS and PaaS directly though. <uriel> will that have to wait for the new billing system? <robertk> proppy: ah, so is that change because always on is going away? <nubbius> Marzia we use GAE to upload documents in Google Docs and use this method, so we can't upload from our app docs bigger than 1 Mbyte <itsPhilip> tmatsuo_google: Isn't Azure PaaS? ;-) <proppy> robertk: scheduler will now send request to always on instances only if there are no dynamic instance, or all dynamic instances are busy <marzia_google> @nubbius understood. it would definitely be nice to have <dunker> is there any clue to be shared on how the transition with the scheduler is going to take place ? as peoples billings depend on it is there going to be a "big switch" day or something ? <robertk> proppy: interesting <proppy> nubbius: maybe you could bug the Docs API team to provide resumable upload ? <nubbius> Also we have an application that treat personal data (nubbius.com), and we would like to know when SSL will be avalaible for GAE custom domain apps <proppy> nubbius: aa_ is a Googler working on Docs API <marzia_google> we always look to optimize the scheduler, independently from any billing changes <proppy> aa_: he might be able to help when back online PSIlits ([email protected]) has joined #appengine <marzia_google> with the new billing we will provide more options for you to trade off performance and instances <dunker> thing is changing the scheduler changes the bill <marzia_google> basically giving you the option of choosing not to use aspects of the scheduler <ikai_google> I'm back. I HATE TIME ZONES, real meeting is at 1 <dunker> re ikai_google <tmatsuo_google> itsPhilip: ah sorry, but we're still confident our current pricing model is competitive :) <proppy> ikai_google: ahah <dunker> ah thats nice marzia_google <kevlar_google> ikai_google: gcal ftw :) <proppy> ikai_google: http://www.tmzns.com/ <ikai_google> I'm USING gcal <marzia_google> eg for 1.5.2 as in the release notes, we will allow you to modify some scheduling parameters <kevlar_google> lol <ikai_google> eastern time and bogota are not the same thing <ikai_google> one accounts fot DST <ikai_google> sigh <ikai_google> facepalm <marzia_google> but you can always opt to use the scheduler as designed <dunker> yeah like that <itsPhilip> marzia_google: Is it still Google's current plan to roll out the new pricing before multi threaded python instances will be available? <kevlar_google> ikai_google: mine asks me to change my timezone whenever my laptop's timezone changes, ymmv <dunker> greg said there were going to be knobs but this makes it more clear thnx <ikai_google> kevlar_google: dude that's a total YMMV, it didn't ask me to change mine <marzia_google> yes, see greg's FAQ re: half price instances for python <ikai_google> kevlar_google: so I manually set it .. you would think all GMT -5s are equal. they are not <dunker> owja any ETA on python threads ? <dunker> hehe <ikai_google> dunker: with 2.7 <dunker> ok cool <robertk> so with the new scheduler will we get penalized for high latency requests? <nubbius> @aa_. Could you help us with limit upload in Google Docs from GAE? It's reported, viewed in user groups but there is not a solution to avoid 1 MB limit PSIlits ([email protected]) has joined #appengine <robertk> ie will we be able to adjust it so that we'll still get more instances? <uriel> marzia_google: will Go instances be one-fourth price? Go has much, much, much lower memory usage than either python or java ;) <marzia_google> in the future we will allow you to pay to keep more instances around, yes <dunker> the knobs are a interesting concept as users are able the tune themselves <marzia_google> we don't have the details for go pricing at this time <itsPhilip> Hypothetical Question: What average price increase does Google expect internally? <dunker> any details of the "open source" variant ? <robertk> marzia_google: was that a response to my question? <uriel> marzia_google: I see, so no billing for Go until new billing system then? it would be nice if pricing for Go took into account its much lower resource consumption ;) <proppy> itsPhilip: I think those details will come along the pricing calculator <marzia_google> @robertk - unless you don't think so, yes :) <ikai_google> uriel: a lot of things would be nice <robertk> marzia_google: actually i don't.... ;) <mbw> marzia_google: so, if requests are still over 1 second, it will still punish you by not spinning up instances? even though the pending queue latency will go up and it should spin up more instances? <proppy> robertk: please rephrase then :) <marzia_google> haha ok, so to be more clear <marzia_google> we will allow you to adjust the scheduler to be more or less agressive in spinning up instances <uriel> ikai_google: I understand :) I just think it would be unfair to charge the same for Go instances when they use a small fraction of the memory/cpu <marzia_google> right now the knobs we are releasing soon will allow you to introduce more latency and have fewer instances, and take instances down more aggressively <Wooble> uriel: presumably each instance will be able to handle multiple requests at some point. <marzia_google> in the future we will introduce knobs that do the opposite <robertk> marzia_google: so right now requests with > 1000ms latency will get fewer resources, they'll get throttle_code=X in the logs, and get into a very bad feedback loop for a while. if we're abel to adjust the scheduler, will we still face issues with high latency applications? <marzia_google> allow you to more aggressively keep instances around <mbw> marzia_google: robertk and I understand that, we are specifically asking if the 1s punishment barrier is still in effect <uriel> Wooble: that would be nice too, obviously :) <marzia_google> so if you are using the auto-scheduler, yes <marzia_google> if you choose to adjust the scheduler to be more aggressive at spinning up instances <marzia_google> that is another story <ikai_google> uriel: we try to make pricing as fair as possible. HOWEVER: https://groups.google.com/d/msg/google-appengine/ob-kMuDAAqc/JGvSGLMkvP0J <itsPhilip> There was a heavy hr outage recently. I looked like you moved all hr apps to another datacenter. As a consequence the average latency decreased by a factor of 2-3. How can we rely on the new pricing structure if the latency can vary that much? schuppe (d8ef2d04@gateway/web/freenode/ip.216.239.45.4) has joined #appengine <marzia_google> @itsPhilip to what HR outage are you referring? Google_Wesley (d8ef2d04@gateway/web/freenode/ip.216.239.45.4) has joined #appengine <uriel> ikai_google: well, to be able to enable billing for Go at all would be an improvement from now :)) <marzia_google> we had an outage on thursday that primarily affected M/S apps <itsPhilip> I don't know anymore when it was @mbw can you remeber it? You contacted google because of it. <mbw> marzia_google: we saw a few odd things on HR, I wouldnt call it an outage. We just say memcache flushed (which is fine) and a few more timeouts than normal on datastore <robertk> itsPhilip: i think HR was up, but memcache was flushed <marzia_google> HR apps are served out of multiple datacenters by design <marzia_google> yes, we had an issue with more aggresive than usual memcache flushing <ikai_google> uriel: The Go stuff still needs a lot before I'd say the API is close to complete <marzia_google> but that's not related to HR <mbw> marzia_google: thats misleading... they are only served out of ONE datacenter at a time <ikai_google> mbw: not completely true. the datastore instances are in multiple data centers <mbw> yes, but the app != datastore <uriel> ikai_google: I see, and there wont be billing until it is API-complete? (that seems reasonable, just wondering) <ikai_google> mbw: app > datastore and serving <ikai_google> it's a superset. <ikai_google> if the front end web serving goes down, we failover to a second data center <kevlar_google> "oh oh, it's magic! You know..." <uriel> ikai_google: also, while I agree that things aren't always fair, to charge the same for Go instances would mean it is always unfair ;) I'm quite certain Go always uses considerably less memory than Java or Python, and has much lower startup time <robertk> yeah but you're memcache doesn't follow. it will be reset too in that case <Google_Wesley> kevlar> people are too young here to remember that song (except for the occasional commercials) <marzia_google> Keep in mind that you can't just charge for memory, apps also use CPU <peper> this was probably asked before, but any plans on streamlinging the M/S -> HR migration? <ikai_google> peper: yes <Google_Wesley> peper> yes, we are working on it <marzia_google> hence the move to instance billing <peper> any ETA? <marzia_google> vs just CPU or memory <ikai_google> peper: no <nubbius> we have an application that treat personal data (nubbius.com), and we would like to know when SSL will be avalaible for GAE custom domain apps <peper> ok <ikai_google> peper: soon <peper> thanks for info :) <ikai_google> nubbius: soon, ideally <ikai_google> as a general rule we rarely give ETAs <itsPhilip> So what are the consequences if the latency increases by a factor of 3 on all apps? Do we have to pay that or will this be covered by the SLA? <ikai_google> the exception is if a prerelease goes out <peper> yeah, i noticed <ikai_google> we can tell you that it'lll probably be a week-a month, and we can also tell you if we find too many bugs with a feature and it probably *won't* go out <Google_Wesley> "we are working on it." in general, anything on our public roadmap means there is at least one engineer actively working on it. <uriel> marzia_google: cpu-based billing worked great :( <robertk> hi Google_Wesley. didn't notice you lurking in here ;) <marzia_google> but you didn't pay for memory :) <proppy> guido is having an hangout on G+ <proppy> feel free to pm me your email if you want to join <ikai_google> itsPhilip: latency is often the result of multiple factors. the SLA currently only covers uptime. We are not planning on per API SLAs <dunker> proppy: get guido over here would be better ;] <uriel> marzia_google: I wouldnt mind paying for memory too, if I'm going to be using Go I will use very little memory ;P (but I understand that it is tricky to account for everything) <itsPhilip> marzia_google: Fun question: Where does google buys its memory? I can rent a server with 6core amd, 32gig ram and 20TB gbit traffic for 130/month? Maybe you should outsource your hosting ;-) <mbw> marzia_google: We are lobbying for more ram on front ends... we need it <marzia_google> i'm fairly certain we get it from Best Buy, right? <marzia_google> (that was clearly a joke, to be explicit) <itsPhilip> (my offer is from a huge hoster with 4 datacenters; they will expand to the usa soon) <proppy> dunker: he seems to really prefer the hangout format <dunker> yeah just kidding <peper> marzia_google: otoh allowing one instance to serve multiple requests in parallel would lower the memory usage :) <proppy> let me know if you want to join, so I can invite you <Wooble> itsPhilip: it seems unlikely many hosting companies would meet google's needs :P <ikai_google> itsPhilip: those are good prices. You should do what's best for you. Me? I've spent enough time in my life driving to data centers kicking servers back to life <dunker> thats why IRC needs to be integrated in Google+ ;] <dunker> proppy: i dont have a cam so i cant speak in hangouts only type chat <Google_Wesley> proppy> can you IM us? jfmontesdeoca (480ee551@gateway/web/freenode/ip.72.14.229.81) has joined #appengine <itsPhilip> ikai_google: Yeah thats the problem ;-) <ikai_google> and yes, sometimes, you physically need to kick a server for it to start working again (my guess is weird fan stuff) <marzia_google> yes we are working on multiple requests/instance for languages, which java always have <dunker> which sucks cause i dont have a mic as well <marzia_google> random comment: you should definitely turn on threadsafe flag for java <proppy> Google_Wesley: give me you google+ email can I can invite you <peper> Can i just join to listen? no webcam <Wooble> ikai_google: reminds me of the floppy drives on my commodore 64 :) <Google_Wesley> geoff> i loved my C-64! :) <itsPhilip> A few months ago I asked Gregory D.. if prospective search will be free of charge, he wanted to check it but I've missed the answer: Will it be free of charge? <proppy> if you have question about App Engine billing, Appstats, domain console feel free to join guido hangout <marzia_google> @itsPhilip we are currently not charging for PS, since it's experimental. But in the future we will, yes. <ikai_google> lol billing? <ikai_google> you mean the billing console <ikai_google> guido can't answer any questions about billing *plans* <proppy> ikai_google: yep sorry :) <Google_Wesley> yes <proppy> thanks for watching my words out <ikai_google> =) <ikai_google> I would feel bad if he got a bunch of "what if this" "what if that" questions <ikai_google> the man isn't an insurance salesman. <Google_Wesley> he would get mad <Google_Wesley> don't make him mad... you woudln't like him when he's angry <dunker> hehe <dunker> would be fun to see Guido go mad ;] <Google_Wesley> although he isn't "cloud hulk" <Google_Wesley> someone else is... or at least acts like it <Wooble> can we ask Guido about Python's uncertain future? ;) <ikai_google> is it uncertain? <Wooble> I read on the mailing list that it is, so it must be. <uriel> if you read it on the internets, it is a fact jfmontesdeoca (480ee551@gateway/web/freenode/ip.72.14.229.81) has joined #appengine <Google_Wesley> geoff> what is uncertain? <Google_Wesley> oh, i need to find that... is that in the Python forum? <itsPhilip> There was a discussion in the mailing list regarding the complex TOS. Do you plan to simplify it? The Azure TOS are a good example of easy to understand TOS. <Wooble> I don't know; apparently having the BDFL work on a 2.7 runtime isn't an indication Google's going to stick with python. <Google_Wesley> itsPhilip> we also have a PP paired with the ToS <ikai_google> itsPhilip: yes, I believe we are <dunker> real questions is when will 2.7 be here .. but such questions cannot be answered ;] <Google_Wesley> Wooble> people don't get it <ikai_google> itsPhilip: changing legalese though is non trivial. have you ever heard of "sh*t's easy syndrome"? http://steve-yegge.blogspot.com/2009/04/have-you-ever-legalized-marijuana.html <marzia_google> @dunker - does the answer "we're actively working on it" make you happy <marzia_google> :) <dunker> ikai_google: i liked that article <itsPhilip> PP <ikai_google> so did I. I'm throwing that term out a lot nowadays. <dunker> marzia_google: havent heared that one before ;] <itsPhilip> Yes, the azure pp is: "Don't host anything that is illegal on azure". I've talked to one of their managers they won't even judge by theirself what has to be considered illegal. Thats the way to go IMHO. <dunker> ikai_google: its indeed all about complexity besides the weed thing <Wooble> so can I host a website about tiananmen square on Azure? <itsPhilip> ikai_google: Wow that article is too long to read ;-D <dunker> itsPhilip: its a nice read though <Google_Wesley> wooble> yes, just choose to host it in asia... china if possible. <dunker> 18:41 < jsonbot> [jsb-debian] - jsonbot 0.7.1.2-1 MIGRATED to testing (Britney) - http://tinyurl.com/44zntcv <dunker> yess ;] <dunker> bit offtopic sorry bout that <itsPhilip> Google_Wesley: Thats a mean comment ;-) rsaccon ([email protected]) has joined #appengine <itsPhilip> Sooo anything new about nextgen queries? =) <rwilhelm23> Question about the differences in time between the appstats and the dashboard: appstats will show a consistent "real" time for a bunch of requests, but the dashboard will show some took twice as long as others - what are the sources of that "extra" time? <Google_Wesley> itsPhilip> more "in-progress" now than in the past year... <ikai_google> itsPhilip: you are getting no exploding indexes required <ikai_google> that's part of it. <itsPhilip> Ah nice <ikai_google> it's in the next release <Wooble> you should just call that "nextgen", then add "third generation queries" to the roadmap. :) <ikai_google> see prerelease notes <proppy> rwilhelm23: Google_Wesley is asking guido in the hangouts <Google_Wesley> Guido says> if they're loading requests, Appstats doesn't start its clock until after the app has initialized, i.e., it doesn't count imports, etc. <Google_Wesley> Guido also says> Appstats takes measurements from point A to point B, but after that, the request isn't quite sent back to the Appserver because Appstats is pure userspace and just wraps low-level calls <Google_Wesley> that's all he can think of at this time itsPhilip (~Philip@unaffiliated/itsphilip) has joined #appengine <rwilhelm23> thanks Leemp ([email protected]) has joined #appengine <proppy> robertk> asking on the hangouts asking about C/extension on Python 2.7 <itsPhilip> Why doesn't Guido join the chat? <ikai_google> I think he said something about preferring the hangouts format <Google_Wesley> itsPhilip> he and IRC don't get along very well <proppy> guido> said he couldn't comment publicly yet, but that it will be open for testing internally first <dunker> IRC is too nerdy for guido ;] <proppy> and then to truster tester <Wooble> having visited #python, I don't blame him :) <dunker> so it *is* going to be there <Google_Wesley> can you imagine if you were him, get onto IM, and see "OHHHHH, it's GUIDO... HIIIIIIII GUIDO!!! I LOVE MY PYTHONZ" <dunker> didnt know that <proppy> guido> no plan for allowing people to upload their own C extension in the first phase of the project <dunker> Google_Wesley: its tough being a hero ;] <proppy> guido> but plan to provide access to popular C extension <rwilhelm23> Cpickle would be nice, but I'm not holding my breath <Google_Wesley> basically, it needs a LOT of testing to make sure things don't break before 2.7 support goes out <mbw> proppy: how about providing lxml? cpickle? etc etc out of the box? <dunker> do want to say that to Guido someday <Wooble> sandboxing user-contributed C extensions seems... fairly difficult. <itsPhilip> If he also has hot groupies? lol <mbw> Wooble: thus NACL <proppy> mbw: that I think that's the provide access to popular C extension part <mbw> I am still amazed that the community hasn't been a lot more vocal about library support and fast serialization support... it's a constant pain in my ass <moraes> i've met guido editting a google doc. i said "OHHHHH, it's GUIDO... HIIIIIIII GUIDO!!! I LOVE MY PYTHONZ" but he ignored me. <dunker> moraes: ;] <dunker> you need to think of a clever question <proppy> mbw: guido> cpickle is on the list, dunno about lxml he will ask <dunker> like when is the GIL going to be removed <dunker> hehe soyapi ([email protected]) has joined #appengine <rwilhelm23> Did you see Brett's post on that? <tmatsuo_google> if I remember correctly, lxml is on the list <dunker> proppy: simplejson with accel ? <proppy> mbw: guido> lxml would probably be on the list too <marzia_google> ok, i'm off <moraes> or "why is max a built in function?" <marzia_google> thanks for coming <dunker> marzia_google: thanks ! <dunker> ow <dunker> thats quick <dunker> learned a lot today though it was usefull <moraes> damn lxml would be awesome <Google_Wesley> cPickle is very likely --------------------------------------------------------------------------------------------- 2011-07-20 19:00:25 CEST App Engine IRC Office Hours closes --------------------------------------------------------------------------------------------- -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
