Hi Stephen, I'll start a separate thread with you so we can discuss the specifics.
Thanks, Christina, Google App Engine Product Manager On Fri, Jan 4, 2013 at 1:02 PM, Stephen Blum <[email protected]> wrote: > Hi Takashi and Google Team, > > We made the transition successfully from *Master/Slave* to the new *High > Replication *datastore. However it cost us just under *$2,000.00 and it > was supposed to be free*. Please let us know how we fix this billing > error. > > > On Wednesday, January 2, 2013 11:26:23 PM UTC-8, Stephen Blum wrote: >> >> Hi Takashi, >> >> We tried using the *Migration Buttons* in the Datastore Admin >> section labeled "*Copy to Another App*" and "Delete Entities". The >> button labeled "Copy to Another App" is totally related and relevant to >> migration and the way it is described in the Migration Document. The two >> important notes I took from the Migration Doc were: >> >> 1. You do not incur any billing costs when you do this migration. >> 2. you will make your old application ID an "alias" for the new one; >> thus, if users browse your application at *old-name*.appspot.com, >> they will instead see your new application (which you might think of as >> *new-name*.appspot.com). >> >> That made it totally obvious what we needed to do. Basically create a >> new App Name. Then Copy the Data over to the new App. This is the >> direction we followed and yet apparently we were surprised by a $2,000.000 >> bill in only a few moments. Please help us fix this billing error! :-) >> >> On Wednesday, January 2, 2013 11:17:24 PM UTC-8, Takashi Matsuo (Google) >> wrote: >>> >>> >>> Hi Stephen, >>> >>> On Wed, Jan 2, 2013 at 10:49 PM, Stephen Blum <[email protected]>wrote: >>> >>>> Hi Takashi, >>>> >>>> Thank you for the update. How do we properly apply for errors in >>>> billing? Not allowing us Billing Credit for a Migration Path that is * >>>> Required* does not makes sense! *:-)* There must be a way to correct >>>> this error. Basically GAE Dashboard is currently: >>>> >>>> 1. Offering a Migration Button that will instantly charge $2,000.00 >>>> (why why why?) and >>>> >>>> Can you tell me which button you're talking about? >>> >>>> >>>> 1. Warning loudly in a big yellow *warning* banner that told us to >>>> Migrate Now! and it *would actually be free*. >>>> >>>> As long as you use our migration tool, the migration process itself >>> should be free. >>> >>> >>>> It is my understanding that we are suposed to migrate. Yet it isn't >>>> free...? No worries! We won't migrate then. * :-) * That solves >>>> that part... now How may we make it so we are not charged for this FAKE OUT >>>> migration process? Please Help! >>>> >>>> On Wednesday, January 2, 2013 10:28:01 PM UTC-8, Takashi Matsuo >>>> (Google) wrote: >>>> >>>>> >>>>> Hi Stephen, >>>>> >>>>> Unfortunately, you did it wrong. Please refer to the >>>>> doc<https://developers.google.com/appengine/docs/adminconsole/migration?hl=en>and >>>>> migrate with our migration tool. >>>>> >>>>> Re: refund >>>>> I don't think we can refund for it, sorry. >>>>> >>>>> -- Takashi >>>>> >>>>> >>>>> On Wed, Jan 2, 2013 at 10:18 PM, Stephen Blum <[email protected]>wrote: >>>>> >>>>>> Hi Takashi, >>>>>> >>>>>> >>>>>> <https://lh5.googleusercontent.com/-6hxKP1JDfeU/UOUifTEC7KI/AAAAAAAAAYE/KFldMXpMZ1g/s1600/Screen+Shot+2013-01-02+at+10.16.01+PM.png> >>>>>> Thank you again for the fast response and direct details. You are >>>>>> being very helpful. There are Datastore buttons and controls which we >>>>>> are >>>>>> using in the *GAE Dashboard* under *Datastore Admin* section. Are >>>>>> we to understand that those are not honored with the Migration Path? If >>>>>> this is the case it was a massive mistake that we made: May ask for a >>>>>> refund? We are using these buttons (as shown in the screenshot). >>>>>> >>>>>> >>>>>> On Wednesday, January 2, 2013 10:05:38 PM UTC-8, Takashi Matsuo >>>>>> (Google) wrote: >>>>>> >>>>>>> >>>>>>> On Wed, Jan 2, 2013 at 9:48 PM, Stephen Blum <[email protected]>wrote: >>>>>>> >>>>>>>> Hi Takashi, >>>>>>>> >>>>>>>> Thank you for the details! We are in the process of migrating now, >>>>>>>> and I expect that is cause. However it was my understanding that the >>>>>>>> migration was a free process as per the docs. Why is it incurring >>>>>>>> such a >>>>>>>> high charge? Those tasks queues are old and from long ago that has not >>>>>>>> changed for about a year. However now that we are starting the >>>>>>>> migration >>>>>>>> the billing has overblown! It's almost at *$2,000.00* for just a >>>>>>>> few hours. >>>>>>>> >>>>>>> >>>>>>> Yes, our migration >>>>>>> tool<https://developers.google.com/appengine/docs/adminconsole/migration?hl=en>doesn't >>>>>>> cost you for the migration process itself. If you're using our >>>>>>> migration tool, I believe that the high usage of the datastore is >>>>>>> caused by >>>>>>> other reasons(it is very likely the infinite retry loop I mentioned >>>>>>> before). >>>>>>> >>>>>>> Can you tell me how you're migrating your app? If you're using any >>>>>>> other tools for the migration, it will very likely cost you for the >>>>>>> migration process itself. >>>>>>> >>>>>>> -- Takashi >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Wednesday, January 2, 2013 9:37:16 PM UTC-8, Takashi Matsuo >>>>>>>> (Google) wrote: >>>>>>>> >>>>>>>>> >>>>>>>>> Hi Stephen, >>>>>>>>> >>>>>>>>> All of your tasks in the 'default' task queue are retrying more >>>>>>>>> than 17 times. I suspect these tasks contribute to the high usage of >>>>>>>>> the >>>>>>>>> datastore. >>>>>>>>> >>>>>>>>> If that's the case, maybe you can stop your 'default' queue first, >>>>>>>>> and fix the settings for your 'default' queue to prevent the infinite >>>>>>>>> loop(see the taskqueue settings >>>>>>>>> doc[Python<https://developers.google.com/appengine/docs/python/config/queue#Configuring_Retry_Attempts_for_Failed_Tasks> >>>>>>>>> |Java<https://developers.google.com/appengine/docs/java/config/queue#Configuring_Retry_Attempts_for_Failed_Tasks>]), >>>>>>>>> and then fix the root cause, and then finally you can restart the >>>>>>>>> default >>>>>>>>> queue. >>>>>>>>> >>>>>>>>> I also noticed that you're using deprecated M/S datastore, which >>>>>>>>> is very susceptible to just a small failure in a single datacenter. I >>>>>>>>> strongly recommend migrating your app to HRD. >>>>>>>>> >>>>>>>>> -- Takashi >>>>>>>>> >>>>>>>>> >>>>>>>>> On Wed, Jan 2, 2013 at 9:06 PM, Stephen Blum >>>>>>>>> <[email protected]>wrote: >>>>>>>>> >>>>>>>>>> Hi Takashi! >>>>>>>>>> >>>>>>>>>> App ID: See Screenshot Below >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> <https://lh6.googleusercontent.com/-SvGzC9LkOVc/UOURu-OdYqI/AAAAAAAAAXs/qZydTwPBjss/s1600/Screen+Shot+2013-01-02+at+9.05.47+PM.png> >>>>>>>>>> Full Usage Stats: See Screenshot Below >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> <https://lh6.googleusercontent.com/-OjhWSnMFZgo/UOUR3x01BJI/AAAAAAAAAX0/yAaPAEHMGKY/s1600/Screen+Shot+2013-01-02+at+9.05.10+PM.png> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Wednesday, January 2, 2013 8:47:20 PM UTC-8, Takashi Matsuo >>>>>>>>>> (Google) wrote: >>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Hi Stephen, >>>>>>>>>>> >>>>>>>>>>> What is your app-id? Can you share the error message? I don't >>>>>>>>>>> think anyone can help you without knowing more. >>>>>>>>>>> >>>>>>>>>>> Few things came to mind though: >>>>>>>>>>> How big is the traffic to your app now? Doesn't your app just >>>>>>>>>>> have a big spike? >>>>>>>>>>> If you're running erroneous tasks on the task queue, you can >>>>>>>>>>> pause the queue from the admin console for a band-aid purpose. >>>>>>>>>>> If you have some error-handling implementations which can lead >>>>>>>>>>> to an infinite retry loop(with or without task queue), you need to >>>>>>>>>>> fix the >>>>>>>>>>> implementation. >>>>>>>>>>> >>>>>>>>>>> -- Takashi >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> On Wed, Jan 2, 2013 at 8:03 PM, Stephen Blum < >>>>>>>>>>> [email protected]> wrote: >>>>>>>>>>> >>>>>>>>>>>> Help! We are bing charged *$30/per minute*! *A major >>>>>>>>>>>> datastore error. * Help! Keeps increasing! Help! >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> <https://lh3.googleusercontent.com/-7aDiJuaWEQM/UOUCx3GKmCI/AAAAAAAAAXc/DFYTgLtqSHI/s1600/Screen+Shot+2013-01-02+at+8.01.41+PM.png> >>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>>>> Google Groups "Google App Engine" group. >>>>>>>>>>>> To view this discussion on the web visit >>>>>>>>>>>> https://groups.google.com/d/**ms********g/google-appengine/-/** >>>>>>>>>>>> mCpqsBgvS********kcJ<https://groups.google.com/d/msg/google-appengine/-/mCpqsBgvSkcJ> >>>>>>>>>>>> . >>>>>>>>>>>> To post to this group, send email to google-a...@googlegroups. >>>>>>>>>>>> **com. >>>>>>>>>>>> To unsubscribe from this group, send email to >>>>>>>>>>>> google-appengi...@**googlegroups********.com. >>>>>>>>>>>> >>>>>>>>>>>> For more options, visit this group at http://groups.google.com/ >>>>>>>>>>>> **group********/google-appengine?hl=en<http://groups.google.com/group/google-appengine?hl=en> >>>>>>>>>>>> . >>>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Takashi Matsuo | Developers Advocate | [email protected] >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> You received this message because you are subscribed to the >>>>>>>>>> Google Groups "Google App Engine" group. >>>>>>>>>> To view this discussion on the web visit >>>>>>>>>> https://groups.google.com/d/**ms******g/google-appengine/-/** >>>>>>>>>> HO4kpXH_**A****AQJ<https://groups.google.com/d/msg/google-appengine/-/HO4kpXH_AAQJ> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>>> To post to this group, send email to google-a...@googlegroups.** >>>>>>>>>> com. >>>>>>>>>> To unsubscribe from this group, send email to google-appengi...@* >>>>>>>>>> *googlegroups******.com. >>>>>>>>>> For more options, visit this group at http://groups.google.com/** >>>>>>>>>> group******/google-appengine?hl=en<http://groups.google.com/group/google-appengine?hl=en> >>>>>>>>>> . >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Takashi Matsuo | Developers Advocate | [email protected] >>>>>>>>> >>>>>>>> -- >>>>>>>> You received this message because you are subscribed to the Google >>>>>>>> Groups "Google App Engine" group. >>>>>>>> To view this discussion on the web visit >>>>>>>> https://groups.google.com/d/**ms****g/google-appengine/-/** >>>>>>>> nIwnzB7Dl****QIJ<https://groups.google.com/d/msg/google-appengine/-/nIwnzB7DlQIJ> >>>>>>>> . >>>>>>>> >>>>>>>> To post to this group, send email to google-a...@googlegroups.**com >>>>>>>> . >>>>>>>> To unsubscribe from this group, send email to google-appengi...@** >>>>>>>> googlegroups****.com. >>>>>>>> For more options, visit this group at http://groups.google.com/** >>>>>>>> group****/google-appengine?hl=en<http://groups.google.com/group/google-appengine?hl=en> >>>>>>>> . >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Takashi Matsuo | Developers Advocate | [email protected] >>>>>>> >>>>>> -- >>>>>> You received this message because you are subscribed to the Google >>>>>> Groups "Google App Engine" group. >>>>>> To view this discussion on the web visit https://groups.google.com/d/ >>>>>> **ms**g/google-appengine/-/**AVBE4004J**_YJ<https://groups.google.com/d/msg/google-appengine/-/AVBE4004J_YJ> >>>>>> . >>>>>> >>>>>> To post to this group, send email to google-a...@googlegroups.**com. >>>>>> To unsubscribe from this group, send email to google-appengi...@** >>>>>> googlegroups**.com. >>>>>> For more options, visit this group at http://groups.google.com/** >>>>>> group**/google-appengine?hl=en<http://groups.google.com/group/google-appengine?hl=en> >>>>>> . >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Takashi Matsuo | Developers Advocate | [email protected] >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Google App Engine" group. >>>> To view this discussion on the web visit https://groups.google.com/d/** >>>> msg/google-appengine/-/**uHtKkpji1RYJ<https://groups.google.com/d/msg/google-appengine/-/uHtKkpji1RYJ> >>>> . >>>> >>>> To post to this group, send email to [email protected]. >>>> To unsubscribe from this group, send email to google-appengi...@** >>>> googlegroups.com. >>>> For more options, visit this group at http://groups.google.com/** >>>> group/google-appengine?hl=en<http://groups.google.com/group/google-appengine?hl=en> >>>> . >>>> >>> >>> >>> >>> -- >>> Takashi Matsuo | Developers Advocate | [email protected] >>> >> -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/jp1qYT_WrokJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
