date:20120803

Hi Garima,

Please fill out the form bellow:
https://appengine.google.com/waitlist/sms_issues

and wait a bit.

Thanks,

On Fri, Aug 3, 2012 at 1:45 PM, Garima Amit Agarwal amitgar...@gmail.comwrote:

Hello Amy,
I have tried many times to send sms to my mobile number but haven't
received any yet and my phone is receiving other sms perfectly. Now as I
have tried multiple times so now I am getting error 'too many sms smg
sent'. I also tried to use 'sms_issues' link but no reply on that so far.
Kindly help.

Thanks,
Amit

On Thursday, April 26, 2012 3:10:10 AM UTC-4, Amy Unruh wrote:

We often get questions on this list about how to sign up for an App
Engine Account in the case where you are not able to receive SMS messages,
or if you don't have a mobile phone. If you have those or similar issues,
then as described here:

https://developers.google.com/**appengine/kb/sms#errorhttps://developers.google.com/appengine/kb/sms#error
you can fill out a form:

https://appengine.google.com/**waitlist/sms_issueshttps://appengine.google.com/waitlist/sms_issues
which will put you on a list to receive an invitation. (Note that this
list is not processed automatically, so there can sometimes be a few days'
delay).

-Amy

To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

--
Takashi Matsuo

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-03 Thread Richard Watson

What are the performance characteristics of connecting to Google Compute 
Engine?  Maybe slap the in-memory app onto that.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/pTFagZqQkx4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Boat Load of Questions (which might turn in to feature requests)



 Why can’t I configure how many requests  Frontends handle? F1’s serving 8 
 requests is silly. F4’s handling 8 requests is often equally silly in the 
 opposite direction.


Good idea. I have created a feature request for it, Please star it:
http://code.google.com/p/googleappengine/issues/detail?id=7927

Do we always have to go through Apps For Domains? How much money do I have 
 to send you to never have to talk to their tech support again? Similar 
 question Naked Domians. I want one. Building a URL Shortener that has to 
 have www. On the front is breaking my heart. 

Would the VIP SSL allow you to circumvent Apps for Domains? 

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/q8pIZi4OdZQJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Boat Load of Questions (which might turn in to feature requests)

2012-08-03 Thread Kristopher Giesing



On Sunday, July 29, 2012 12:43:31 PM UTC-7, Brandon Wirtz wrote:

 Why can’t I configure how many requests a backend can handle? Or have a 
 back end act like a front end and handle 8?

The Java version has an experimental field in backends.xml called 
max-concurrent-requests.  I would assume Python will get the same at some 
point.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/KOvXcx3NZBgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-03 Thread Mauricio Aristizabal

Takashi, is there some more detailed information on why Google doesn't
encourage using a connection pool?  Is it simply to encourage allowing the
db instance to wind down instead of being kept alive only by pool
connection health checks?  If so I'm sure it could be configured to avoid
this.

It does seem to me that it could reduce Richard's costs drastically, by 2/3
just on the writes to the db by his own numbers.

I've been using pooling without issue for several months now, though
admittedly with very little traffic so far, so if you think this is going
to get me in trouble later I'm very eager to hear why.



On Fri, Aug 3, 2012 at 12:28 AM, Richard Watson richard.wat...@gmail.comwrote:

 What are the performance characteristics of connecting to Google Compute
 Engine?  Maybe slap the in-memory app onto that.

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/pTFagZqQkx4J.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Custom SSL certificate with more than two intermediate CAs

2012-08-03 Thread Nacho Coloma

Hi Cayden,

Thanks for your reply.

You appear to have the incorrect CNAME for your domain. This is most
probably what is causing android browsers to fail to connect. The correct
CNAME can be found in your Google Apps control panel. The uploading and
configuring certificates section of the SSL for Custom Domains
documentation https://developers.google.com/appengine/docs/ssl may
prove helpful if you have any issues.

Yep, I saw the change of ghs name but since neither certificate was working
we are just stopping this (with this working configuration) until our new
certificate arrives.

We just purchased a new one with DigiCert that includes EV validation and
uses (supposedly, as far as we could check) a single intermediate authority.

On the topic of intermediate certificates you should be able to download a
single intermediate certificate from Comodo
herehttps://support.comodo.com/index.php?_m=downloads_a=viewparentcategoryid=1pcid=0nav=0.
Usually certificate authorities provide a bundle file which contains the
full chain, all the certificates in the bundle are often not required.

Ours is (was) a Comodo EssentialSSL. It comes with 5 CAs in the bundle, and
AFAIK most browsers require the chain up to the root CA.

Don't worry about this, the change of certificate should fix it up. Anyway,
I would reconsider the limitation of two CAs in the PEM bundle, if that's
an option. Anyway, it's just my fault for not fully understanding the
limitations before choosing the certificate provider. Thank God for the
15-days refund policy.

Thanks for your support.

On 2 August 2012 04:03, Nacho Coloma icol...@gmail.com wrote:

Hi, I have just configured a certificate for our own custom domain (VIP)
and it is working fine, but Android browsers are rejecting to connect.

Investigating, it seems that I should include the full chain of
intermediate CAs to the uploaded PEM file, but that's not possible since
AppEngine only allows at most two certificates in the PEM file. Our Comodo
certificate has a chain composed of five CAs. If I try to upload the full
PEM file, AppEngine complains that the format is not supported.

The working certificate can be seen at https://koliseo.com. You can test
it with:

openssl s_client -showcerts -connect www.koliseo.com:443

Desktop browsers are OK with it, but Android (Froyo and Honeycomb) will
just refuse to connect. Any ideas?

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/AvvSXY6BrugJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Suggestion: Python Requests library

2012-08-03 Thread Helge Tesdal

If you use buildout and a recipe like
http://pypi.python.org/pypi/rod.recipe.appengine/ you add requests as
a dependency in the setup.py of your package, and add it to the
packages list in the buildout configuration to make it part of a
packages.zip. You then include packages.zip to sys.path and it just
works.

I find that to be a convenient way of including 3rd party libraries,
actually preferable to having it included in GAE, as I'm in charge of
choosing explicit versions in my buildout and can test, switch and
upgrade exactly when I want, independent of GAE release cycle.

On Wed, Jul 25, 2012 at 6:24 PM, Robert Schuppenies schu...@google.com wrote:
 You can file a feature request, but rather unlikely IMO. Two reasons that
 come to mind:
 - requests is pure Python so bundling it with your app is not hard (as for
 example numpy)
 - we don't want to commit to supporting libraries unless the previous reason
 is true or a lot of people ask for it (avoiding maintenance being the top
 reason)

 Also, both urllib and urllib2 are part of Python's stdlib, so using them as
 an argument doesn't count :P


 cheers,
 robert




 On Tue, Jul 24, 2012 at 11:02 AM, rdodev ami...@gmail.com wrote:

 Would it be possible to add Python Requests library (
 http://docs.python-requests.org/en/latest/index.html ) to AppEngine? urllib
 and urllib2 are rather cumbersome for complex/custom http requests to other
 web services. Your attention is much appreciated.

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/gMHme7EfuYwJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.



-- 
Helge Tesdal
CTO - mCASH Norge AS
+47 815 10 150
http://mCA.SH

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Problem with ctypes

2012-08-03 Thread Helge Tesdal

I think the page could be made easier to understand by tweaking the text.

It says C Modules Supported in Python 2.7 and the text goes on to
say the list contains modules NOT allowed.

The next heading is for Python 2.5 and lists modules that ARE allowed.

If the top header says something like C modules disallowed in Python
2.7 it might be clearer, and you can change it back if you add lists
of both allowed and disallowed for 2.7.

On Wed, Jul 25, 2012 at 4:44 AM, Brian Quinlan bquin...@google.com wrote:
 Hi Yuval,

 Take a look at this link:
 https://developers.google.com/appengine/kb/libraries

 Cheers,
 Brian

 On Mon, Jul 23, 2012 at 3:11 AM, Yuval Kalev yuvalka...@gmail.com wrote:
 I seem to be having problems with importing ctypes to use for an MGRS module
 I want to load.
 I keep getting this error:

 ImportError: No module named _ctypes


 Reading posts I read that version 2.7.1-2 might have fixed this issue. Still
 I cannot get it to work.

 Any ideas?


 Thanks,

 - Yuval

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/S7pu8and_AEJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Helge Tesdal
CTO - mCASH Norge AS
+47 815 10 150
http://mCA.SH

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

Connection pooling might be a good idea.  Since there are people in every 
game round and each round is 3 minutes, the SQL db will always be up.  I 
did try it, but I think my connection from home was limited.  

RE: SQL solution:   Can some of you with LOTS of bandwidth (from a *nix 
machine), please AB the following URL:

 http://sven-anagramhero.appspot.com/client/loadtest

Try at least 1000 connections with 250-500 concurrent and report back here 
please.

WRT costs:  DB read/writes are around $3/day.  Whereas 10 B1 backends would 
be almost $20/day.

In addition, the B1 solution does not scale.  Lets say the app suddenly 
gets a lot of new users.  Now I need to update the backend.  Additionally, 
peak load is approximately 4x the lowest value.  Now, for part of each day, 
I need to have enough instances to handle the peak load just doing minimal 
work.  This is not scaling automatically.  I always need to pay the maximum 
of whatever is needed to handle peak load or else update the backends 
every few hours to add/remove B1's.  Not exactly fulfilling the automatic 
scaling promise!



On Friday, August 3, 2012 3:49:24 AM UTC-4, Mauricio Aristizabal wrote:

 Takashi, is there some more detailed information on why Google doesn't 
 encourage using a connection pool?  Is it simply to encourage allowing the 
 db instance to wind down instead of being kept alive only by pool 
 connection health checks?  If so I'm sure it could be configured to avoid 
 this.

 It does seem to me that it could reduce Richard's costs drastically, by 
 2/3 just on the writes to the db by his own numbers.

 I've been using pooling without issue for several months now, though 
 admittedly with very little traffic so far, so if you think this is going 
 to get me in trouble later I'm very eager to hear why.



 On Fri, Aug 3, 2012 at 12:28 AM, Richard Watson 
 richard.wat...@gmail.comwrote:

 What are the performance characteristics of connecting to Google Compute 
 Engine?  Maybe slap the in-memory app onto that.

  -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/pTFagZqQkx4J.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/-NIqVbhGPw4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] How can I block curl requests

2012-08-03 Thread Joshua Smith

What might be helpful would be:

1. Add some logging. Up top:

import logging

then in the __call__ method:

def __call__(self, environ, start_response):
logging.info('__call__ sees UA: %s', environ['HTTP_USER_AGENT'])
if environ['HTTP_USER_AGENT'].startswith('curl'):

2. Deploy that

3. Check your logs. If you are being hit with lots of inbound curl calls, you 
should get some clues right away. But you can also try hitting it with curl 
yourself.



On Aug 2, 2012, at 8:39 PM, Kate mss.ka...@gmail.com wrote:

 I restarted the instance. I also got a copy of curl.exe for windows and it 
 lets me through! :-(
 
 My code is exactly this (below) and is in the file appengine_config.py in the 
 root directory.
 
 I also tested your example from my machine and got blocked.
 
 I can't think of why this could be! I even altered appengine_config.py to 
 have a syntax error and it picked it up, so I know it is loading.
 
 
 from webob import Response
 
 class AntiCurlMiddleware(object):
 def __init__(self, app):
 self.app = app
 
 def __call__(self, environ, start_response):
 if environ['HTTP_USER_AGENT'].startswith('curl'):
 resp = Response('Too many requests!')
 resp.status = 423
 return resp(environ, start_response)
 return self.app(environ, start_response)
 
 def webapp_add_wsgi_middleware(app):
 return AntiCurlMiddleware(app)
 
 
 On Thursday, August 2, 2012 6:55:37 PM UTC-4, Kyle Finley wrote:
 How do I restart the instance?
 at appengine.google.com in the instance section you should see a list of 
 instance. They each have a Shutdown button
 
 Yes it is in the root directory. I am stumped! I didn't think it was working 
 as I tried testing for different browsers and it didn't catch them.
 
 I don't know, that's strange. I'ts working here:
 http://anticurl.scotch-media.appspot.com/
 
 And if you have included the file in your project, it should at the very 
 least stop me from using curl to access your site.
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/1jEROOXORQMJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

On Fri, Aug 3, 2012 at 10:04 PM, Richard steven...@gmail.com wrote:

Connection pooling might be a good idea. Since there are people in every
game round and each round is 3 minutes, the SQL db will always be up. I
did try it, but I think my connection from home was limited.

RE: SQL solution: Can some of you with LOTS of bandwidth (from a *nix
machine), please AB the following URL:

http://sven-anagramhero.appspot.com/client/loadtest

Try at least 1000 connections with 250-500 concurrent and report back here
please.

WRT costs: DB read/writes are around $3/day. Whereas 10 B1 backends
would be almost $20/day.

In addition, the B1 solution does not scale. Lets say the app suddenly
gets a lot of new users. Now I need to update the backend. Additionally,
peak load is approximately 4x the lowest value. Now, for part of each day,
I need to have enough instances to handle the peak load just doing minimal
work. This is not scaling automatically. I always need to pay the maximum
of whatever is needed to handle peak load or else update the backends
every few hours to add/remove B1's. Not exactly fulfilling the automatic
scaling promise!

Good point, but I think you can workaround with dynamic backend instances,
although you need to care about shutdown scenario. Here is just a
theoretical implementation.

* Define one resident backend instance for handling the scores from backend
instances which are shutting down. Let's call this instance as a 'master'.
* Define dynamic backend with appropriate max instances according your
needs, setup a shutdown hook which will pass the scores to the master
instance.
* When reaping, you can ask dynamic instances as well as the master
instance.

That way, you can have the autoscale capability with minimum cost.

Maybe I can create a prototype implementation hopefully early next week. Of
course, you can create your own.

-- Takashi

On Friday, August 3, 2012 3:49:24 AM UTC-4, Mauricio Aristizabal wrote:

Takashi, is there some more detailed information on why Google doesn't
encourage using a connection pool? Is it simply to encourage allowing the
db instance to wind down instead of being kept alive only by pool
connection health checks? If so I'm sure it could be configured to avoid
this.

It does seem to me that it could reduce Richard's costs drastically, by
2/3 just on the writes to the db by his own numbers.

I've been using pooling without issue for several months now, though
admittedly with very little traffic so far, so if you think this is going
to get me in trouble later I'm very eager to hear why.

On Fri, Aug 3, 2012 at 12:28 AM, Richard Watson richard.wat...@gmail.com
wrote:

What are the performance characteristics of connecting to Google Compute
Engine? Maybe slap the in-memory app onto that.

--
You received this message because you are subscribed to the Google
Groups Google App Engine group.
To view this discussion on the web visit https://groups.google.com/d/**
msg/google-appengine/-/**pTFagZqQkx4Jhttps://groups.google.com/d/msg/google-appengine/-/pTFagZqQkx4J
.

To post to this group, send email to
google-appengine@googlegroups.**comgoogle-appengine@googlegroups.com
.
To unsubscribe from this group, send email to
google-appengine+unsubscribe@**googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com
.
For more options, visit this group at http://groups.google.com/**
group/google-appengine?hl=enhttp://groups.google.com/group/google-appengine?hl=en
.

--
Takashi Matsuo

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-03 Thread alex

From Rackspace (London):

ab -n 1000 -c 200  http://sven-anagramhero.appspot.com/client/loadtest
This is ApacheBench, Version 2.3 $Revision: 655654 $

Server Software:Google
Server Hostname:sven-anagramhero.appspot.com
Server Port:80

Document Path:  /client/loadtest
Document Length:2 bytes

Concurrency Level:  200
Time taken for tests:   15.694 seconds
Complete requests:  1000
Failed requests:0
Write errors:   0
Total transferred:  171000 bytes
HTML transferred:   2000 bytes
Requests per second:63.72 [#/sec] (mean)
Time per request:   3138.712 [ms] (mean)
Time per request:   15.694 [ms] (mean, across all concurrent requests)
Transfer rate:  10.64 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:88   1.5  8  22
Processing:   139 2827 1197.6   29108487
Waiting:  139 2827 1197.6   29108487
Total:147 2835 1197.6   29188494

Percentage of the requests served within a certain time (ms)
  50%   2918
  66%   3341
  75%   3620
  80%   3874
  90%   4257
  95%   4700
  98%   5900
  99%   6131
 100%   8494 (longest request)


ab -n 1000 -c 500  http://sven-anagramhero.appspot.com/client/loadtest
This is ApacheBench, Version 2.3 $Revision: 655654 $

Server Software:Google
Server Hostname:sven-anagramhero.appspot.com
Server Port:80

Document Path:  /client/loadtest
Document Length:2 bytes

Concurrency Level:  500
Time taken for tests:   6.879 seconds
Complete requests:  1000
Failed requests:0
Write errors:   0
Total transferred:  171000 bytes
HTML transferred:   2000 bytes
Requests per second:145.37 [#/sec] (mean)
Time per request:   3439.463 [ms] (mean)
Time per request:   6.879 [ms] (mean, across all concurrent requests)
Transfer rate:  24.28 [Kbytes/sec] received

Connection Times (ms)
  min  mean[+/-sd] median   max
Connect:8   17   9.0 11  28
Processing:   144 2210 1535.1   18856831
Waiting:  144 2210 1535.2   18856831
Total:152 2227 1539.7   18946853

Percentage of the requests served within a certain time (ms)
  50%   1894
  66%   2410
  75%   3100
  80%   3225
  90%   4492
  95%   5628
  98%   6418
  99%   6484
 100%   6853 (longest request)


On Friday, August 3, 2012 3:04:55 PM UTC+2, Richard wrote:

 Connection pooling might be a good idea.  Since there are people in every 
 game round and each round is 3 minutes, the SQL db will always be up.  I 
 did try it, but I think my connection from home was limited.  

 RE: SQL solution:   Can some of you with LOTS of bandwidth (from a *nix 
 machine), please AB the following URL:

  http://sven-anagramhero.appspot.com/client/loadtest

 Try at least 1000 connections with 250-500 concurrent and report back here 
 please.

 WRT costs:  DB read/writes are around $3/day.  Whereas 10 B1 backends 
 would be almost $20/day.

 In addition, the B1 solution does not scale.  Lets say the app suddenly 
 gets a lot of new users.  Now I need to update the backend.  Additionally, 
 peak load is approximately 4x the lowest value.  Now, for part of each day, 
 I need to have enough instances to handle the peak load just doing minimal 
 work.  This is not scaling automatically.  I always need to pay the maximum 
 of whatever is needed to handle peak load or else update the backends 
 every few hours to add/remove B1's.  Not exactly fulfilling the automatic 
 scaling promise!



 On Friday, August 3, 2012 3:49:24 AM UTC-4, Mauricio Aristizabal wrote:

 Takashi, is there some more detailed information on why Google doesn't 
 encourage using a connection pool?  Is it simply to encourage allowing the 
 db instance to wind down instead of being kept alive only by pool 
 connection health checks?  If so I'm sure it could be configured to avoid 
 this.

 It does seem to me that it could reduce Richard's costs drastically, by 
 2/3 just on the writes to the db by his own numbers.

 I've been using pooling without issue for several months now, though 
 admittedly with very little traffic so far, so if you think this is going 
 to get me in trouble later I'm very eager to hear why.



 On Fri, Aug 3, 2012 at 12:28 AM, Richard Watson richard.wat...@gmail.com
  wrote:

 What are the performance characteristics of connecting to Google Compute 
 Engine?  Maybe slap the in-memory app onto that.

  -- 
 You received this message because you are subscribed to the Google 
 Groups Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/pTFagZqQkx4J.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at

Re: [google-appengine] How can I block curl requests

Hi Joshua, 

Thank you, that's a good thought. 

Kate sent me some files offline, and I believe we've figured out the 
problem. For the middleware to work you must be using WSGI not CGI. Someone 
please correct me if I'm wrong, but I believe she would have to upgrade 
here App to python27 to use it. The alternative is to do the check in the 
webapp request handler:

def check_for_curl(self):
if self.request.environ['HTTP_USER_AGENT'].startswith('curl'):
return self.error(401)

class MainHandler(webapp.RequestHandler):
def get(self):
check_for_curl(self)
# handle request

The problem is that webapp doesn't recognize error code 429 so we have to 
use something else. Unless there's a simple way to make it write 429?

- Kyle

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/TQuZYYR0wrAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] How can I block curl requests

2012-08-03 Thread Joshua Smith

There are couple problems with your snippet.

First, she's getting HEAD not GET requests, so you need to use different 
handler.

Also, you aren't returning, so if you were in a GET request, it would proceed 
to handle the request regardless.

Something more like this (untested):

class MainHandler(webapp.RequestHandler):
  def head(self):
self.error(401)

  def get(self):
if (self.request.headers['User-Agent'].startswith('curl'))
  self.error(401)
  return
# rest of the get handler

On Aug 3, 2012, at 10:34 AM, Kyle Finley kylefin...@gmail.com wrote:

 Hi Joshua, 
 
 Thank you, that's a good thought. 
 
 Kate sent me some files offline, and I believe we've figured out the problem. 
 For the middleware to work you must be using WSGI not CGI. Someone please 
 correct me if I'm wrong, but I believe she would have to upgrade here App to 
 python27 to use it. The alternative is to do the check in the webapp request 
 handler:
 
 def check_for_curl(self):
 if self.request.environ['HTTP_USER_AGENT'].startswith('curl'):
 return self.error(401)
 
 class MainHandler(webapp.RequestHandler):
 def get(self):
 check_for_curl(self)
 # handle request
 
 The problem is that webapp doesn't recognize error code 429 so we have to use 
 something else. Unless there's a simple way to make it write 429?
 
 - Kyle
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/TQuZYYR0wrAJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] How can I block curl requests

Yes, thank you.  Do you have any thoughts on how to return error code 429?

On Aug 3, 2012, at 9:51 AM, Joshua Smith wrote:

 There are couple problems with your snippet.
 
 First, she's getting HEAD not GET requests, so you need to use different 
 handler.
 
 Also, you aren't returning, so if you were in a GET request, it would proceed 
 to handle the request regardless.
 
 Something more like this (untested):
 
 class MainHandler(webapp.RequestHandler):
   def head(self):
 self.error(401)
 
   def get(self):
 if (self.request.headers['User-Agent'].startswith('curl'))
   self.error(401)
   return
 # rest of the get handler
 
 On Aug 3, 2012, at 10:34 AM, Kyle Finley kylefin...@gmail.com wrote:
 
 Hi Joshua, 
 
 Thank you, that's a good thought. 
 
 Kate sent me some files offline, and I believe we've figured out the 
 problem. For the middleware to work you must be using WSGI not CGI. Someone 
 please correct me if I'm wrong, but I believe she would have to upgrade here 
 App to python27 to use it. The alternative is to do the check in the webapp 
 request handler:
 
 def check_for_curl(self):
 if self.request.environ['HTTP_USER_AGENT'].startswith('curl'):
 return self.error(401)
 
 class MainHandler(webapp.RequestHandler):
 def get(self):
 check_for_curl(self)
 # handle request
 
 The problem is that webapp doesn't recognize error code 429 so we have to 
 use something else. Unless there's a simple way to make it write 429?
 
 - Kyle
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/TQuZYYR0wrAJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.
 
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Startup time exceeded...on F4?!

While startup times appears to be a contentious topic. I think that we can 
all agree that:

*Fewer Instance *==* Few Cold Starts *==* Happy Customers *==* Smaller Bills
*

Currently Apps of all sizes are capped at 10 concurrent requests. F4 
instance can handle no more concurrent request then a F1. I have create a 
feature request to make this option configurable. 

If you believe that your App can handle Greater than 10 concurrent requests 
please star this issue: 

http://code.google.com/p/googleappengine/issues/detail?id=7927

Thank you

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/MM5pHzrCPVcJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] How can I block curl requests


 I think you change 401 in this code to 429

I wish it was that easy. Webapp2 uses dictionary to return the status code / 
message and 429 didn't make the list.

preTraceback (most recent call last):
  File 
quot;/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/_webapp25.pyquot;,
 line 701, in __call__
handler.get(*groups)
  File quot;/Users/finley/dev/scotch/operation_curl_block/main.pyquot;, line 
7, in get
return self.error(429)
  File 
quot;/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/_webapp25.pyquot;,
 line 435, in error
self.response.set_status(code)
  File 
quot;/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/_webapp25.pyquot;,
 line 279, in set_status
message = Response.http_status_message(code)
  File 
quot;/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/webapp/_webapp25.pyquot;,
 line 341, in http_status_message
raise Error('Invalid HTTP status code: %d' % code)
Error: Invalid HTTP status code: 429

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] How can I block curl requests

2012-08-03 Thread Drake

Ah, I hadn't checked. I usually return a permission denied Error, or a Busy
Error, 503 I think (sorry not at my desk)

 

 

From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Kyle Finley
Sent: Friday, August 03, 2012 8:43 AM
To: google-appengine@googlegroups.com
Subject: Re: [google-appengine] How can I block curl requests

 





I think you change 401 in this code to 429

 

I wish it was that easy. Webapp2 uses dictionary to return the status code /
message and 429 didn't make the list.

 

preTraceback (most recent call last):

  File
quot;/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleApp
Engine-default.bundle/Contents/Resources/google_appengine/google/appengine/e
xt/webapp/_webapp25.pyquot;, line 701, in __call__

handler.get(*groups)

  File quot;/Users/finley/dev/scotch/operation_curl_block/main.pyquot;,
line 7, in get

return self.error(429)

  File
quot;/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleApp
Engine-default.bundle/Contents/Resources/google_appengine/google/appengine/e
xt/webapp/_webapp25.pyquot;, line 435, in error

self.response.set_status(code)

  File
quot;/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleApp
Engine-default.bundle/Contents/Resources/google_appengine/google/appengine/e
xt/webapp/_webapp25.pyquot;, line 279, in set_status

message = Response.http_status_message(code)

  File
quot;/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleApp
Engine-default.bundle/Contents/Resources/google_appengine/google/appengine/e
xt/webapp/_webapp25.pyquot;, line 341, in http_status_message

raise Error('Invalid HTTP status code: %d' % code)

Error: Invalid HTTP status code: 429

 

-- 
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] How can I block curl requests

2012-08-03 Thread Joshua Smith

I would have thought self.error(429). That doesn't work? Is there a doc that 
says what codes are are allowed to return?

On Aug 3, 2012, at 11:02 AM, Kyle Finley kylefin...@gmail.com wrote:

 Yes, thank you.  Do you have any thoughts on how to return error code 429?
 
 On Aug 3, 2012, at 9:51 AM, Joshua Smith wrote:
 
 There are couple problems with your snippet.
 
 First, she's getting HEAD not GET requests, so you need to use different 
 handler.
 
 Also, you aren't returning, so if you were in a GET request, it would 
 proceed to handle the request regardless.
 
 Something more like this (untested):
 
 class MainHandler(webapp.RequestHandler):
   def head(self):
 self.error(401)
 
   def get(self):
 if (self.request.headers['User-Agent'].startswith('curl'))
   self.error(401)
   return
 # rest of the get handler
 
 On Aug 3, 2012, at 10:34 AM, Kyle Finley kylefin...@gmail.com wrote:
 
 Hi Joshua, 
 
 Thank you, that's a good thought. 
 
 Kate sent me some files offline, and I believe we've figured out the 
 problem. For the middleware to work you must be using WSGI not CGI. Someone 
 please correct me if I'm wrong, but I believe she would have to upgrade 
 here App to python27 to use it. The alternative is to do the check in the 
 webapp request handler:
 
 def check_for_curl(self):
 if self.request.environ['HTTP_USER_AGENT'].startswith('curl'):
 return self.error(401)
 
 class MainHandler(webapp.RequestHandler):
 def get(self):
 check_for_curl(self)
 # handle request
 
 The problem is that webapp doesn't recognize error code 429 so we have to 
 use something else. Unless there's a simple way to make it write 429?
 
 - Kyle
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/TQuZYYR0wrAJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.
 
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.
 
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] How can I block curl requests

@Brandon
Yes, 503 would probably be better then 401.

@Joshua
No 429 doesn't work. I don't know if the allowed return values are documented, 
but here's the source:
http://code.google.com/p/googleappengine/source/browse/trunk/python/google/appengine/ext/webapp/_webapp25.py#345


On Aug 3, 2012, at 10:45 AM, Joshua Smith wrote:

 I would have thought self.error(429). That doesn't work? Is there a doc that 
 says what codes are are allowed to return?
 
 On Aug 3, 2012, at 11:02 AM, Kyle Finley kylefin...@gmail.com wrote:
 
 Yes, thank you.  Do you have any thoughts on how to return error code 429?
 
 On Aug 3, 2012, at 9:51 AM, Joshua Smith wrote:
 
 There are couple problems with your snippet.
 
 First, she's getting HEAD not GET requests, so you need to use different 
 handler.
 
 Also, you aren't returning, so if you were in a GET request, it would 
 proceed to handle the request regardless.
 
 Something more like this (untested):
 
 class MainHandler(webapp.RequestHandler):
   def head(self):
 self.error(401)
 
   def get(self):
 if (self.request.headers['User-Agent'].startswith('curl'))
   self.error(401)
   return
 # rest of the get handler
 
 On Aug 3, 2012, at 10:34 AM, Kyle Finley kylefin...@gmail.com wrote:
 
 Hi Joshua, 
 
 Thank you, that's a good thought. 
 
 Kate sent me some files offline, and I believe we've figured out the 
 problem. For the middleware to work you must be using WSGI not CGI. 
 Someone please correct me if I'm wrong, but I believe she would have to 
 upgrade here App to python27 to use it. The alternative is to do the check 
 in the webapp request handler:
 
 def check_for_curl(self):
 if self.request.environ['HTTP_USER_AGENT'].startswith('curl'):
 return self.error(401)
 
 class MainHandler(webapp.RequestHandler):
 def get(self):
 check_for_curl(self)
 # handle request
 
 The problem is that webapp doesn't recognize error code 429 so we have to 
 use something else. Unless there's a simple way to make it write 429?
 
 - Kyle
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To view this discussion on the web visit 
 https://groups.google.com/d/msg/google-appengine/-/TQuZYYR0wrAJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.
 
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.
 
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.
 
 
 -- 
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

Thanks Alex, VERY much appreciated, since I can't test this myself without 
buying a shell account somewhere.

Luckily, the backend crashed due to being unable to reuse the connection 
for the delete.  So I added some exception handling :)

Can I ask some more people to try this link:  
http://sven-anagramhero.appspot.com/client/loadtest

Please ping it once from a web browser just before you hit it.  This will 
ensure the DB is up :)

I would like to see results for loads of n  1000 with c = 500.

The server clears out results every 3 minutes (synchronized to NTP time) on 
the minute boundary, so please try to avoid doing it exactly on that 
boundary (in which case the results will be spread and it makes it more 
difficult to ensure we did not 'lose' any).

NOTE:  It seems we can store at least 1k users within 10 seconds . I 
really don't like the 6.8 second response (I would prefer 300 msec). 
viable ?  y/n ?

Thanks !

-R

On Friday, August 3, 2012 10:02:09 AM UTC-4, alex wrote:

 From Rackspace (London):

 ab -n 1000 -c 200  http://sven-anagramhero.appspot.com/client/loadtest
 This is ApacheBench, Version 2.3 $Revision: 655654 $

 Server Software:Google
 Server Hostname:sven-anagramhero.appspot.com
 Server Port:80

 Document Path:  /client/loadtest
 Document Length:2 bytes

 Concurrency Level:  200
 Time taken for tests:   15.694 seconds
 Complete requests:  1000
 Failed requests:0
 Write errors:   0
 Total transferred:  171000 bytes
 HTML transferred:   2000 bytes
 Requests per second:63.72 [#/sec] (mean)
 Time per request:   3138.712 [ms] (mean)
 Time per request:   15.694 [ms] (mean, across all concurrent requests)
 Transfer rate:  10.64 [Kbytes/sec] received

 Connection Times (ms)
   min  mean[+/-sd] median   max
 Connect:88   1.5  8  22
 Processing:   139 2827 1197.6   29108487
 Waiting:  139 2827 1197.6   29108487
 Total:147 2835 1197.6   29188494

 Percentage of the requests served within a certain time (ms)
   50%   2918
   66%   3341
   75%   3620
   80%   3874
   90%   4257
   95%   4700
   98%   5900
   99%   6131
  100%   8494 (longest request)


 ab -n 1000 -c 500  http://sven-anagramhero.appspot.com/client/loadtest
 This is ApacheBench, Version 2.3 $Revision: 655654 $

 Server Software:Google
 Server Hostname:sven-anagramhero.appspot.com
 Server Port:80

 Document Path:  /client/loadtest
 Document Length:2 bytes

 Concurrency Level:  500
 Time taken for tests:   6.879 seconds
 Complete requests:  1000
 Failed requests:0
 Write errors:   0
 Total transferred:  171000 bytes
 HTML transferred:   2000 bytes
 Requests per second:145.37 [#/sec] (mean)
 Time per request:   3439.463 [ms] (mean)
 Time per request:   6.879 [ms] (mean, across all concurrent requests)
 Transfer rate:  24.28 [Kbytes/sec] received

 Connection Times (ms)
   min  mean[+/-sd] median   max
 Connect:8   17   9.0 11  28
 Processing:   144 2210 1535.1   18856831
 Waiting:  144 2210 1535.2   18856831
 Total:152 2227 1539.7   18946853

 Percentage of the requests served within a certain time (ms)
   50%   1894
   66%   2410
   75%   3100
   80%   3225
   90%   4492
   95%   5628
   98%   6418
   99%   6484
  100%   6853 (longest request)


 On Friday, August 3, 2012 3:04:55 PM UTC+2, Richard wrote:

 Connection pooling might be a good idea.  Since there are people in every 
 game round and each round is 3 minutes, the SQL db will always be up.  I 
 did try it, but I think my connection from home was limited.  

 RE: SQL solution:   Can some of you with LOTS of bandwidth (from a *nix 
 machine), please AB the following URL:

  http://sven-anagramhero.appspot.com/client/loadtest

 Try at least 1000 connections with 250-500 concurrent and report back 
 here please.

 WRT costs:  DB read/writes are around $3/day.  Whereas 10 B1 backends 
 would be almost $20/day.

 In addition, the B1 solution does not scale.  Lets say the app suddenly 
 gets a lot of new users.  Now I need to update the backend.  Additionally, 
 peak load is approximately 4x the lowest value.  Now, for part of each day, 
 I need to have enough instances to handle the peak load just doing minimal 
 work.  This is not scaling automatically.  I always need to pay the maximum 
 of whatever is needed to handle peak load or else update the backends 
 every few hours to add/remove B1's.  Not exactly fulfilling the automatic 
 scaling promise!



 On Friday, August 3, 2012 3:49:24 AM UTC-4, Mauricio Aristizabal wrote:

 Takashi, is there some more detailed information on why Google doesn't 
 encourage using a connection pool?  Is it simply to encourage allowing the 
 db instance to wind down instead of being kept alive only by pool 
 connection health checks?

[google-appengine] Conversion-API (Java) failure in convert into PDF file

2012-08-03 Thread satyarao

Helo to allI,

  I am trying to convert text/HTML file into Pdf file.
  I am using AppEngine Conversion-API (for java). 
  I am getting proper output on browser(Jsp page). But the created pdf 
page, from the HTML is not good.
  It looks like without proper alignment. Attached is the  HTML file 
and the  PDF created. 
  Please send me the solution for obtaining a proper PDF.


1) Using Below code,  I am trying to converting text/html file into PDF 
file.

   ListAsset listAssets = new ArrayListAsset();
  Asset asset = new Asset(text/html, wrapper.getOutput().getBytes(), 
testfile.html);
  listAssets.add(asset);

  Document document = new Document(listAssets);
  Conversion conversion = new Conversion(document, application/pdf);

  ConversionService service = 
ConversionServiceFactory.getConversionService();
  ConversionResult result = service.convert(conversion);

  if (result.success()) {
res.setStatus(HttpServletResponse.SC_OK);
res.setHeader(Content-Disposition, inline;);
res.setContentType(application/pdf);
for (Asset resAsset : result.getOutputDoc().getAssets()) {
  res.getOutputStream().write(resAsset.getData());
}
  } else {
ConversionErrorCode errorcode = result.getErrorCode();
log.severe(errorcode.name());
log.severe(errorcode.toString());
System.out.println(errorcode);
System.out.println(errorcode.toString());
res.getWriter().write(Error creating pdf);
  }
 

2) The Required Page is clean on Browser, But in pdf ,its very bad .

Please send  me the solution for this conversion failures. 

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Xa5hlNlzDOAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.



template1.pdf
Description: Adobe PDF document

[google-appengine] Problem in authorizing my App Engine app to access Big Query Cloud Storage

2012-08-03 Thread Vivek Madurai

Hi ,
I am a Google Apps account user of mydomain.com. I have my application 
running at myappspot.  I created a Big Query project in Google apis 
console, as I wanted to analyse data from myappspot using Big Query. 

As per the 'Authorize App Engine app' step in this tutorial 
https://developers.google.com/bigquery/articles/datastoretobigquery#auth, I 
tried adding myappspotid.@appspot.gserviceaccount.com under the Team tab. 
It doesn't allow me to do that and shows an error saying *User 
myappspotid@appspot.gserviceaccount.com may not be added to project My 
Project Only members from domain xxx.com may be added* 

When I tried creating another Google apis project, using gmail (instead of 
my mydomain.com Google Apps id), I was able to succeed. Can somebody help 
me with whats going wrong with this step, when using a Google Apps account?


Thanks
Vivek

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/wI7qtTyiZVEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: API serving not allowed for this application

2012-08-03 Thread Albert

Hi,

I think this happens because you need to be Trusted Tester to be able to 
use this feature https://endpoints-trusted-tester.appspot.com/

четверг, 19 июля 2012 г., 16:08:28 UTC+3 пользователь adt написал:

 Hello.

 I am attempting to follow the tutorial at: 
 http://www.youtube.com/watch?v=v9TG7OzsZqQ

 My Cloud Endpoint REST API works well on my local development machine, but 
 when 
 I deploy to App Engine, I receive errors in my Admin Log that API serving 
 not allowed for this application.

 Is this a paid feature? If not, is there documentation that explains this 
 issue?

 Thanks,
 Angelo Torres 


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/2O99FRSKHbgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] What are the pros and cons of using Google App engine for my startup?

2012-08-03 Thread Levi Campbell

I'm building a startup, and I'm considering GAE as the platform, however
I've been having a hard time finding information on why a startup might
consider GAE instead of the many cloud providers out there. Let me explain
what I'm working on.

I'm a big fan of David Allen's Getting Things Done: The Art of Stress-Free
Productivityhttp://www.amazon.com/Getting-Things-Done-Stress-Free-Productivity/dp/0142000280/ref=sr_1_1?ie=UTF8qid=1343984264sr=8-1keywords=Getting+things+done,

and after trying several tools and online services that (claim to)
implement the GTD methodology, I couldn't find anything that I loved, so I
decided to build my own and make it available as a SaaS offering. This app
will allow users to pull in their info_crap from email, facebook, twitter
(and yes, I do have plans to add support for more social networks.), and
RSS feeds and organize it by relationship to the sender (i.e. family,
work colleague, vendor, and the like), project (i.e. planning a family
vacation.), and context (Either the when or where something should
happen.).\

Would GAE be a good fit for the application I'm developing? Why?

--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/mncmD6uYBooJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: What are the pros and cons of using Google App engine for my startup?

2012-08-03 Thread Andrei Volgin

Pros:
You don't need your own servers.
You don't need system admins.
You don't need database admins.
You don't worry about replication and backups.
Many useful built-in services (Users, Blobstore, etc.)
Appengine is free until your traffic picks up.
Seamless (almost) deployment of new versions.

Cons: None.

Some people believe that Appengine has certain shortcomings. Some services 
can be a bit pricey. None of this matters to you. If your startup becomes a 
huge success, you can always look at your Appengine bill and optimize parts 
of your service, or move some services (e.g. email, file storage) to other 
platforms. Appengine is the ideal platform for startups.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/pV_fhfS9bIwJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] What are the pros and cons of using Google App engine for my startup?

2012-08-03 Thread Drake

I have a video on the topic coming shortly.  Boils down to GAE is pretty
much the only PAAS, and IAAS still requires an I Manager. So you save a head
count out the get go.

 

Plus GAE is the most elastic of the Rapidly Elastic, so when
CrunchMashSlashScobleForbesModo writes about you all on the same day you
won't be down.  And when you have crickets. You aren't going broke.

 

From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Levi Campbell
Sent: Friday, August 03, 2012 2:38 AM
To: google-appengine@googlegroups.com
Subject: [google-appengine] What are the pros and cons of using Google App
engine for my startup?

 

I'm building a startup, and I'm considering GAE as the platform, however
I've been having a hard time finding information on why a startup might
consider GAE instead of the many cloud providers out there. Let me explain
what I'm working on.

 

I'm a big fan of David Allen's Getting Things Done: The Art of Stress-Free
Productivity
http://www.amazon.com/Getting-Things-Done-Stress-Free-Productivity/dp/01420
00280/ref=sr_1_1?ie=UTF8qid=1343984264sr=8-1keywords=Getting+things+done
, and after trying several tools and online services that (claim to)
implement the GTD methodology, I couldn't find anything that I loved, so I
decided to build my own and make it available as a SaaS offering. This app
will allow users to pull in their info_crap from email, facebook, twitter
(and yes, I do have plans to add support for more social networks.), and RSS
feeds and organize it by relationship to the sender (i.e. family, work
colleague, vendor, and the like), project (i.e. planning a family
vacation.), and context (Either the when or where something should
happen.).\

 

Would GAE be a good fit for the application I'm developing? Why?

 

-- 
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/mncmD6uYBooJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-03 Thread Jon Stevens

If you are upset by Google's pricing for SSL, please star this issue:

http://code.google.com/p/googleappengine/issues/detail?id=7932

jon


On Thursday, August 2, 2012 10:46:14 AM UTC-7, Chris Ramsdale wrote:

 Jeff, et al.--

 We have verified that a configuration change on our side led to certain 
 requests being denied / redirected.  The rollback of this change started 
 earlier this morning and should be completed shortly.  We are actively 
 looking into measures that we can take to ensure that issues like this are 
 caught prior to rolling out to production.

 If your application continues to be impacted please contact me directly. 

 -- Chris 

 Product Manager, Google App Engine


 On Wed, Aug 1, 2012 at 11:13 AM, Jeff Schnitzer j...@infohazard.orgwrote:

 Ok, this is fucked up.  Visit http://www.voo.st/, and get this:

 https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg

 Something is triggering a false positive from a totally undocumented
 Google security system.  This is really, REALLY not ok.  We are losing
 sales and looking like total idiots to our customers:

 -
 Our systems have detected unusual traffic from your computer network.
 Please try your request again later. Why did this happen?

 This page appears when Google automatically detects requests coming
 from your computer network which appear to be in violation of the
 Terms of Service. The block will expire shortly after those requests
 stop.

 This traffic may have been sent by malicious software, a browser
 plug-in, or a script that sends automated requests. If you share your
 network connection, ask your administrator for help — a different
 computer using the same IP address may be responsible. Learn more

 Sometimes you may see this page if you are using advanced terms that
 robots are known to use, or sending requests very quickly.

 IP address: 208.90.212.26
 Time: 2012-08-01T18:02:00Z
 URL: http://www.voo.st/
 -

 We use CloudFlare as a reverse proxy.  Wild guess is that some sort of
 automated security system is cutting in and detecting CF's proxy as an
 attack.

 PLEASE TURN THIS OFF NOW.

 Jeff

 --
 You received this message because you are subscribed to the Google Groups 
 Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/d4QYLtkKNA0J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-03 Thread Jon Stevens

Here is another issue to star... SSL is too difficult to setup.

http://code.google.com/p/googleappengine/issues/detail?id=7933

jon


On Friday, August 3, 2012 10:16:36 AM UTC-7, Jon Stevens wrote:

 If you are upset by Google's pricing for SSL, please star this issue:

 http://code.google.com/p/googleappengine/issues/detail?id=7932

 jon


 On Thursday, August 2, 2012 10:46:14 AM UTC-7, Chris Ramsdale wrote:

 Jeff, et al.--

 We have verified that a configuration change on our side led to certain 
 requests being denied / redirected.  The rollback of this change started 
 earlier this morning and should be completed shortly.  We are actively 
 looking into measures that we can take to ensure that issues like this are 
 caught prior to rolling out to production.

 If your application continues to be impacted please contact me directly. 

 -- Chris 

 Product Manager, Google App Engine


 On Wed, Aug 1, 2012 at 11:13 AM, Jeff Schnitzer j...@infohazard.orgwrote:

 Ok, this is fucked up.  Visit http://www.voo.st/, and get this:

 https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg

 Something is triggering a false positive from a totally undocumented
 Google security system.  This is really, REALLY not ok.  We are losing
 sales and looking like total idiots to our customers:

 -
 Our systems have detected unusual traffic from your computer network.
 Please try your request again later. Why did this happen?

 This page appears when Google automatically detects requests coming
 from your computer network which appear to be in violation of the
 Terms of Service. The block will expire shortly after those requests
 stop.

 This traffic may have been sent by malicious software, a browser
 plug-in, or a script that sends automated requests. If you share your
 network connection, ask your administrator for help — a different
 computer using the same IP address may be responsible. Learn more

 Sometimes you may see this page if you are using advanced terms that
 robots are known to use, or sending requests very quickly.

 IP address: 208.90.212.26
 Time: 2012-08-01T18:02:00Z
 URL: http://www.voo.st/
 -

 We use CloudFlare as a reverse proxy.  Wild guess is that some sort of
 automated security system is cutting in and detecting CF's proxy as an
 attack.

 PLEASE TURN THIS OFF NOW.

 Jeff

 --
 You received this message because you are subscribed to the Google 
 Groups Google App Engine group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/vraCphOvVlcJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to move data reliably from front to backends ?

Richard,

I did some testing overnight, and I have some good news, and some bad
news.

Good news, I can give you a system that stores 1,000 users and scores
in roughly 1 second. In less than a second, I can pull out all 1,000
scores, sort the scores numerically, and print out the score list.
Bad news: It depends on memcache.

Details: Last night, I wrote an application to generate 1000 users and
1000 scores randomly, and store them in memcache. On average, this
operation takes roughly 1 - 1.3 seconds, although I suspect the
slowness is due to the random number generator, not the memcache. I'll
test this more.

Then a task is enqueued, to call another F1 instance in three seconds.

The next instance pulls out all 1000 scores, sorts them using a
treemap, and prints out the sorted data into GAE logging in less than
a second. Then, the memcache is cleared for the next iteration of the
test, so we don't get old data.

A cron job repeats this test every 2 minutes.

Here is my memcache viewer screen: http://i.imgur.com/oypAm.png . As
you can see, this service ran overnight, and didn't drop a single user/
score. Over 330,000 scores were posted and accessed in total. Is this
good enough performance for your game?



On Aug 3, 11:19 am, Richard steven...@gmail.com wrote:
 Thanks Alex, VERY much appreciated, since I can't test this myself without
 buying a shell account somewhere.

 Luckily, the backend crashed due to being unable to reuse the connection
 for the delete.  So I added some exception handling :)

 Can I ask some more people to try this link:  
 http://sven-anagramhero.appspot.com/client/loadtest

 Please ping it once from a web browser just before you hit it.  This will
 ensure the DB is up :)

 I would like to see results for loads of n  1000 with c = 500.

 The server clears out results every 3 minutes (synchronized to NTP time) on
 the minute boundary, so please try to avoid doing it exactly on that
 boundary (in which case the results will be spread and it makes it more
 difficult to ensure we did not 'lose' any).

 NOTE:  It seems we can store at least 1k users within 10 seconds . I
 really don't like the 6.8 second response (I would prefer 300 msec).
 viable ?  y/n ?

 Thanks !

 -R







 On Friday, August 3, 2012 10:02:09 AM UTC-4, alex wrote:

  From Rackspace (London):

  ab -n 1000 -c 200  http://sven-anagramhero.appspot.com/client/loadtest
  This is ApacheBench, Version 2.3 $Revision: 655654 $

  Server Software:        Google
  Server Hostname:        sven-anagramhero.appspot.com
  Server Port:            80

  Document Path:          /client/loadtest
  Document Length:        2 bytes

  Concurrency Level:      200
  Time taken for tests:   15.694 seconds
  Complete requests:      1000
  Failed requests:        0
  Write errors:           0
  Total transferred:      171000 bytes
  HTML transferred:       2000 bytes
  Requests per second:    63.72 [#/sec] (mean)
  Time per request:       3138.712 [ms] (mean)
  Time per request:       15.694 [ms] (mean, across all concurrent requests)
  Transfer rate:          10.64 [Kbytes/sec] received

  Connection Times (ms)
                min  mean[+/-sd] median   max
  Connect:        8    8   1.5      8      22
  Processing:   139 2827 1197.6   2910    8487
  Waiting:      139 2827 1197.6   2910    8487
  Total:        147 2835 1197.6   2918    8494

  Percentage of the requests served within a certain time (ms)
    50%   2918
    66%   3341
    75%   3620
    80%   3874
    90%   4257
    95%   4700
    98%   5900
    99%   6131
   100%   8494 (longest request)

  ab -n 1000 -c 500  http://sven-anagramhero.appspot.com/client/loadtest
  This is ApacheBench, Version 2.3 $Revision: 655654 $

  Server Software:        Google
  Server Hostname:        sven-anagramhero.appspot.com
  Server Port:            80

  Document Path:          /client/loadtest
  Document Length:        2 bytes

  Concurrency Level:      500
  Time taken for tests:   6.879 seconds
  Complete requests:      1000
  Failed requests:        0
  Write errors:           0
  Total transferred:      171000 bytes
  HTML transferred:       2000 bytes
  Requests per second:    145.37 [#/sec] (mean)
  Time per request:       3439.463 [ms] (mean)
  Time per request:       6.879 [ms] (mean, across all concurrent requests)
  Transfer rate:          24.28 [Kbytes/sec] received

  Connection Times (ms)
                min  mean[+/-sd] median   max
  Connect:        8   17   9.0     11      28
  Processing:   144 2210 1535.1   1885    6831
  Waiting:      144 2210 1535.2   1885    6831
  Total:        152 2227 1539.7   1894    6853

  Percentage of the requests served within a certain time (ms)
    50%   1894
    66%   2410
    75%   3100
    80%   3225
    90%   4492
    95%   5628
    98%   6418
    99%   6484
   100%   6853 (longest request)

  On Friday, August 3, 2012 3:04:55 PM UTC+2, Richard wrote:

  Connection pooling might be a good idea.  Since there are people in every
  game

[google-appengine] Re: How to move data reliably from front to backends ?

Sounds interesting. but how do you handle write contention to the 
memcache datastorage structure from multiple F1's serving client side score 
submissions ?

Also, I thought memcache had a size limit ?  I store a lot more than just 
username + score (including a full stream of all actions the user takes in 
the UI to prevent cheating).

-R

On Friday, August 3, 2012 2:05:34 PM UTC-4, hyperflame wrote:

 Richard, 

 I did some testing overnight, and I have some good news, and some bad 
 news. 

 Good news, I can give you a system that stores 1,000 users and scores 
 in roughly 1 second. In less than a second, I can pull out all 1,000 
 scores, sort the scores numerically, and print out the score list. 
 Bad news: It depends on memcache. 

 Details: Last night, I wrote an application to generate 1000 users and 
 1000 scores randomly, and store them in memcache. On average, this 
 operation takes roughly 1 - 1.3 seconds, although I suspect the 
 slowness is due to the random number generator, not the memcache. I'll 
 test this more. 

 Then a task is enqueued, to call another F1 instance in three seconds. 

 The next instance pulls out all 1000 scores, sorts them using a 
 treemap, and prints out the sorted data into GAE logging in less than 
 a second. Then, the memcache is cleared for the next iteration of the 
 test, so we don't get old data. 

 A cron job repeats this test every 2 minutes. 

 Here is my memcache viewer screen: http://i.imgur.com/oypAm.png . As 
 you can see, this service ran overnight, and didn't drop a single user/ 
 score. Over 330,000 scores were posted and accessed in total. Is this 
 good enough performance for your game? 



 On Aug 3, 11:19 am, Richard steven...@gmail.com wrote: 
  Thanks Alex, VERY much appreciated, since I can't test this myself 
 without 
  buying a shell account somewhere. 
  
  Luckily, the backend crashed due to being unable to reuse the connection 
  for the delete.  So I added some exception handling :) 
  
  Can I ask some more people to try this link:  
 http://sven-anagramhero.appspot.com/client/loadtest 
  
  Please ping it once from a web browser just before you hit it.  This 
 will 
  ensure the DB is up :) 
  
  I would like to see results for loads of n  1000 with c = 500. 
  
  The server clears out results every 3 minutes (synchronized to NTP time) 
 on 
  the minute boundary, so please try to avoid doing it exactly on that 
  boundary (in which case the results will be spread and it makes it more 
  difficult to ensure we did not 'lose' any). 
  
  NOTE:  It seems we can store at least 1k users within 10 seconds . I 
  really don't like the 6.8 second response (I would prefer 300 msec). 
  viable ?  y/n ? 
  
  Thanks ! 
  
  -R 
  
  
  
  
  
  
  
  On Friday, August 3, 2012 10:02:09 AM UTC-4, alex wrote: 
  
   From Rackspace (London): 
  
   ab -n 1000 -c 200  http://sven-anagramhero.appspot.com/client/loadtest 
   This is ApacheBench, Version 2.3 $Revision: 655654 $ 
  
   Server Software:Google 
   Server Hostname:sven-anagramhero.appspot.com 
   Server Port:80 
  
   Document Path:  /client/loadtest 
   Document Length:2 bytes 
  
   Concurrency Level:  200 
   Time taken for tests:   15.694 seconds 
   Complete requests:  1000 
   Failed requests:0 
   Write errors:   0 
   Total transferred:  171000 bytes 
   HTML transferred:   2000 bytes 
   Requests per second:63.72 [#/sec] (mean) 
   Time per request:   3138.712 [ms] (mean) 
   Time per request:   15.694 [ms] (mean, across all concurrent 
 requests) 
   Transfer rate:  10.64 [Kbytes/sec] received 
  
   Connection Times (ms) 
 min  mean[+/-sd] median   max 
   Connect:88   1.5  8  22 
   Processing:   139 2827 1197.6   29108487 
   Waiting:  139 2827 1197.6   29108487 
   Total:147 2835 1197.6   29188494 
  
   Percentage of the requests served within a certain time (ms) 
 50%   2918 
 66%   3341 
 75%   3620 
 80%   3874 
 90%   4257 
 95%   4700 
 98%   5900 
 99%   6131 
100%   8494 (longest request) 
  
   ab -n 1000 -c 500  http://sven-anagramhero.appspot.com/client/loadtest 
   This is ApacheBench, Version 2.3 $Revision: 655654 $ 
  
   Server Software:Google 
   Server Hostname:sven-anagramhero.appspot.com 
   Server Port:80 
  
   Document Path:  /client/loadtest 
   Document Length:2 bytes 
  
   Concurrency Level:  500 
   Time taken for tests:   6.879 seconds 
   Complete requests:  1000 
   Failed requests:0 
   Write errors:   0 
   Total transferred:  171000 bytes 
   HTML transferred:   2000 bytes 
   Requests per second:145.37 [#/sec] (mean) 
   Time per request:   3439.463 [ms] (mean) 
   Time per request:   6.879 [ms] (mean, across all concurrent 
 requests) 
   Transfer rate:

[google-appengine] Re: How to move data reliably from front to backends ?



On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote:
 Sounds interesting. but how do you handle write contention to the
 memcache datastorage structure from multiple F1's serving client side score
 submissions ?

I'm sure it could be done, I have some ideas regarding that (perhaps
vary the key structure depending on the instance/user?) but I really
don't want to pay the cost of multiple F1s, B1s, etc to test my
theory. I might mock up something on my local dev server if I have
time over the weekend, but I don't know how memcache works on the
local development eclipse plugin.

On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote:
 Also, I thought memcache had a size limit ?  I store a lot more than just
 username + score (including a full stream of all actions the user takes in
 the UI to prevent cheating).

How much do you store? My general rule of thumb is that I depend on
memcache to store 1 GB of data before it starts force-expiring objects
(this is for enterprise-level, paid apps). I'm trying to Google around
for some documentation regarding the memcache limit, but it seems that
there is very little documentation regarding memcache. Frankly, I
think shooting for a 100 MB self-imposed-limit should be fine. This is
something you really should ask Takashi.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to move data reliably from front to backends ?

Sorry, I should have been more explicit.

I thought memcache had a size limit on a single object (1MB).  Now imagine 
I have 2000 people submitting data for a game.  i don't think I will be 
fitting all that into 1MB.  Which means I need to store multiple objects 
and fan out/fan in results into memory from memcache (assuming I solve the 
write contention problem... WITHOUT making clients timeout waiting for a 
write lock!).


On Friday, August 3, 2012 2:29:20 PM UTC-4, hyperflame wrote:



 On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote: 
  Sounds interesting. but how do you handle write contention to the 
  memcache datastorage structure from multiple F1's serving client side 
 score 
  submissions ? 

 I'm sure it could be done, I have some ideas regarding that (perhaps 
 vary the key structure depending on the instance/user?) but I really 
 don't want to pay the cost of multiple F1s, B1s, etc to test my 
 theory. I might mock up something on my local dev server if I have 
 time over the weekend, but I don't know how memcache works on the 
 local development eclipse plugin. 

 On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote: 
  Also, I thought memcache had a size limit ?  I store a lot more than 
 just 
  username + score (including a full stream of all actions the user takes 
 in 
  the UI to prevent cheating). 

 How much do you store? My general rule of thumb is that I depend on 
 memcache to store 1 GB of data before it starts force-expiring objects 
 (this is for enterprise-level, paid apps). I'm trying to Google around 
 for some documentation regarding the memcache limit, but it seems that 
 there is very little documentation regarding memcache. Frankly, I 
 think shooting for a 100 MB self-imposed-limit should be fine. This is 
 something you really should ask Takashi. 


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/kkZtq7HtixsJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Problem in authorizing my App Engine app to access Big Query Cloud Storage

2012-08-03 Thread Stephen Lewis

It seems that you can't do this via the API Console, but you can achieve 
the same effect by settings appropriate ACLs to grant permissions to your 
service account using gsutil:

https://developers.google.com/storage/docs/gsutil_reference_guide#setorgetacls

We've done this with an API project owned by a Google Apps account, and it 
works fine.

Stephen

On Friday, 3 August 2012 13:38:04 UTC+1, Vivek Madurai wrote:

 Hi ,
 I am a Google Apps account user of mydomain.com. I have my application 
 running at myappspot.  I created a Big Query project in Google apis 
 console, as I wanted to analyse data from myappspot using Big Query. 

 As per the 'Authorize App Engine app' step in this tutorial 
 https://developers.google.com/bigquery/articles/datastoretobigquery#auth, 
 I tried adding myappspotid.@appspot.gserviceaccount.com under the Team 
 tab. It doesn't allow me to do that and shows an error saying *User 
 myappspotid@appspot.gserviceaccount.com may not be added to project My 
 Project Only members from domain xxx.com may be added* 

 When I tried creating another Google apis project, using gmail (instead of 
 my mydomain.com Google Apps id), I was able to succeed. Can somebody 
 help me with whats going wrong with this step, when using a Google Apps 
 account?


 Thanks
 Vivek


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Lav8VXnG2c8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to move data reliably from front to backends ?

Just moved the scoring over to CloudSQL . and got Featured on Google 
Play Store 30 min ago.

Let's PRAY that Cloud SQL saves our ass or else I am screwed.


On Friday, August 3, 2012 2:38:32 PM UTC-4, Richard wrote:

 Sorry, I should have been more explicit.

 I thought memcache had a size limit on a single object (1MB).  Now imagine 
 I have 2000 people submitting data for a game.  i don't think I will be 
 fitting all that into 1MB.  Which means I need to store multiple objects 
 and fan out/fan in results into memory from memcache (assuming I solve the 
 write contention problem... WITHOUT making clients timeout waiting for a 
 write lock!).


 On Friday, August 3, 2012 2:29:20 PM UTC-4, hyperflame wrote:



 On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote: 
  Sounds interesting. but how do you handle write contention to the 
  memcache datastorage structure from multiple F1's serving client side 
 score 
  submissions ? 

 I'm sure it could be done, I have some ideas regarding that (perhaps 
 vary the key structure depending on the instance/user?) but I really 
 don't want to pay the cost of multiple F1s, B1s, etc to test my 
 theory. I might mock up something on my local dev server if I have 
 time over the weekend, but I don't know how memcache works on the 
 local development eclipse plugin. 

 On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote: 
  Also, I thought memcache had a size limit ?  I store a lot more than 
 just 
  username + score (including a full stream of all actions the user takes 
 in 
  the UI to prevent cheating). 

 How much do you store? My general rule of thumb is that I depend on 
 memcache to store 1 GB of data before it starts force-expiring objects 
 (this is for enterprise-level, paid apps). I'm trying to Google around 
 for some documentation regarding the memcache limit, but it seems that 
 there is very little documentation regarding memcache. Frankly, I 
 think shooting for a 100 MB self-imposed-limit should be fine. This is 
 something you really should ask Takashi. 



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/s0PWD5RHOssJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to move data reliably from front to backends ?


I'm assuming you need storage space to log past user actions so you
can prevent cheating, correct? If so, couldn't you just log, say, the
past 5 (or some relatively small number) of actions and check those
for cheating?

It's difficult to talk hypothetically about these issues without a
diagram or flowchart of what is actually happening in your
application.

On Aug 3, 1:38 pm, Richard steven...@gmail.com wrote:
 Sorry, I should have been more explicit.

 I thought memcache had a size limit on a single object (1MB).  Now imagine
 I have 2000 people submitting data for a game.  i don't think I will be
 fitting all that into 1MB.  Which means I need to store multiple objects
 and fan out/fan in results into memory from memcache (assuming I solve the
 write contention problem... WITHOUT making clients timeout waiting for a
 write lock!).



 On Friday, August 3, 2012 2:29:20 PM UTC-4, hyperflame wrote:

  On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote:
   Sounds interesting. but how do you handle write contention to the
   memcache datastorage structure from multiple F1's serving client side
  score
   submissions ?

  I'm sure it could be done, I have some ideas regarding that (perhaps
  vary the key structure depending on the instance/user?) but I really
  don't want to pay the cost of multiple F1s, B1s, etc to test my
  theory. I might mock up something on my local dev server if I have
  time over the weekend, but I don't know how memcache works on the
  local development eclipse plugin.

  On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote:
   Also, I thought memcache had a size limit ?  I store a lot more than
  just
   username + score (including a full stream of all actions the user takes
  in
   the UI to prevent cheating).

  How much do you store? My general rule of thumb is that I depend on
  memcache to store 1 GB of data before it starts force-expiring objects
  (this is for enterprise-level, paid apps). I'm trying to Google around
  for some documentation regarding the memcache limit, but it seems that
  there is very little documentation regarding memcache. Frankly, I
  think shooting for a 100 MB self-imposed-limit should be fine. This is
  something you really should ask Takashi.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to move data reliably from front to backends ?

Congrats!

Let's see some graphs afterwards, I'd be interested in seeing how
Cloud SQL holds up.

On Aug 3, 2:34 pm, Richard steven...@gmail.com wrote:
 Just moved the scoring over to CloudSQL . and got Featured on Google
 Play Store 30 min ago.

 Let's PRAY that Cloud SQL saves our ass or else I am screwed.



 On Friday, August 3, 2012 2:38:32 PM UTC-4, Richard wrote:

  Sorry, I should have been more explicit.

  I thought memcache had a size limit on a single object (1MB).  Now imagine
  I have 2000 people submitting data for a game.  i don't think I will be
  fitting all that into 1MB.  Which means I need to store multiple objects
  and fan out/fan in results into memory from memcache (assuming I solve the
  write contention problem... WITHOUT making clients timeout waiting for a
  write lock!).

  On Friday, August 3, 2012 2:29:20 PM UTC-4, hyperflame wrote:

  On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote:
   Sounds interesting. but how do you handle write contention to the
   memcache datastorage structure from multiple F1's serving client side
  score
   submissions ?

  I'm sure it could be done, I have some ideas regarding that (perhaps
  vary the key structure depending on the instance/user?) but I really
  don't want to pay the cost of multiple F1s, B1s, etc to test my
  theory. I might mock up something on my local dev server if I have
  time over the weekend, but I don't know how memcache works on the
  local development eclipse plugin.

  On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote:
   Also, I thought memcache had a size limit ?  I store a lot more than
  just
   username + score (including a full stream of all actions the user takes
  in
   the UI to prevent cheating).

  How much do you store? My general rule of thumb is that I depend on
  memcache to store 1 GB of data before it starts force-expiring objects
  (this is for enterprise-level, paid apps). I'm trying to Google around
  for some documentation regarding the memcache limit, but it seems that
  there is very little documentation regarding memcache. Frankly, I
  think shooting for a 100 MB self-imposed-limit should be fine. This is
  something you really should ask Takashi.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to move data reliably from front to backends ?

Well, Cloud SQL is NOT the answer it is topping out around 500 users.  
The extra's don't make it into the DB within the 10 second window.  Then 
they get shown in the next window.

I can 'fix' this by silently deleting the extra's and doing a client update 
that will insert the user into the results leaderboard if they are not 
there.  HOWEVER, the user's stats will then possibly be wrong (best 
game/etc).



On Friday, August 3, 2012 3:43:43 PM UTC-4, hyperflame wrote:

 Congrats! 

 Let's see some graphs afterwards, I'd be interested in seeing how 
 Cloud SQL holds up. 

 On Aug 3, 2:34 pm, Richard steven...@gmail.com wrote: 
  Just moved the scoring over to CloudSQL . and got Featured on Google 
  Play Store 30 min ago. 
  
  Let's PRAY that Cloud SQL saves our ass or else I am screwed. 
  
  
  
  On Friday, August 3, 2012 2:38:32 PM UTC-4, Richard wrote: 
  
   Sorry, I should have been more explicit. 
  
   I thought memcache had a size limit on a single object (1MB).  Now 
 imagine 
   I have 2000 people submitting data for a game.  i don't think I will 
 be 
   fitting all that into 1MB.  Which means I need to store multiple 
 objects 
   and fan out/fan in results into memory from memcache (assuming I solve 
 the 
   write contention problem... WITHOUT making clients timeout waiting for 
 a 
   write lock!). 
  
   On Friday, August 3, 2012 2:29:20 PM UTC-4, hyperflame wrote: 
  
   On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote: 
Sounds interesting. but how do you handle write contention to 
 the 
memcache datastorage structure from multiple F1's serving client 
 side 
   score 
submissions ? 
  
   I'm sure it could be done, I have some ideas regarding that (perhaps 
   vary the key structure depending on the instance/user?) but I really 
   don't want to pay the cost of multiple F1s, B1s, etc to test my 
   theory. I might mock up something on my local dev server if I have 
   time over the weekend, but I don't know how memcache works on the 
   local development eclipse plugin. 
  
   On Aug 3, 1:13 pm, Richard steven...@gmail.com wrote: 
Also, I thought memcache had a size limit ?  I store a lot more 
 than 
   just 
username + score (including a full stream of all actions the user 
 takes 
   in 
the UI to prevent cheating). 
  
   How much do you store? My general rule of thumb is that I depend on 
   memcache to store 1 GB of data before it starts force-expiring 
 objects 
   (this is for enterprise-level, paid apps). I'm trying to Google 
 around 
   for some documentation regarding the memcache limit, but it seems 
 that 
   there is very little documentation regarding memcache. Frankly, I 
   think shooting for a 100 MB self-imposed-limit should be fine. This 
 is 
   something you really should ask Takashi. 


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/vRAVNTOlQ0MJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to move data reliably from front to backends ?

Ditched Cloud SQL.  Went back to the old system of savign lightweight DB 
objects.  I have no idea what to do tonight when DB queries stop working 
again (like they do every night for the last week!)
 

On Friday, August 3, 2012 3:55:01 PM UTC-4, Richard wrote:

 Well, Cloud SQL is NOT the answer it is topping out around 500 users.  
 The extra's don't make it into the DB within the 10 second window.  Then 
 they get shown in the next window.

 I can 'fix' this by silently deleting the extra's and doing a client 
 update that will insert the user into the results leaderboard if they are 
 not there.  HOWEVER, the user's stats will then possibly be wrong (best 
 game/etc).

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/-iswVVyWTPwJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] IOError: [Errno 13] file not accessible after install setuptools

2012-08-03 Thread coto

I installed setuptools on my Macbook Pro Mountain Lion, and Babel with 
Jinja2 in order to run Babel locally.

$ sh setuptools-0.6c11-py2.7.egg --prefix=~ 
--install-dir=/Library/Python/2.7/site-packages/
$ easy_install babel jinja2

After install those packages, I have this warning per any request:

WARNING  2012-08-03 20:56:32,586 py_zipimport.py:139] Can't open zipfile 
/Library/Python/2.7/site-packages/setuptools-0.6c11-py2.7.egg: IOError: 
[Errno 13] file not accessible: 
'/Library/Python/2.7/site-packages/setuptools-0.6c11-py2.7.egg'


But after 2 minutes with any request I have this error and I have to 
restart the application

INFO 2012-08-03 21:00:50,713 py_zipimport.py:148] 
zipimporter('/Library/Python/2.7/site-packages/setuptools-0.6c11-py2.7.egg', 
'')
ERROR2012-08-03 21:00:51,061 wsgi.py:189] 
Traceback (most recent call last):
  File 
/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py,
 
line 187, in Handle
handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File 
/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/runtime/wsgi.py,
 
line 225, in _LoadHandler
handler = __import__(path[0])
  File 
/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py,
 
line 676, in Decorate
return func(self, *args, **kwargs)
  File 
/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py,
 
line 1858, in load_module
...
  File 
/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/tools/dev_appserver_import_hook.py,
 
line 592, in __init__
raise IOError(errno.EACCES, 'file not accessible', filename)
IOError: [Errno 13] file not accessible: 
'/Library/Python/2.7/site-packages/setuptools-0.6c11-py2.7.egg'
INFO 2012-08-03 21:01:11,506 dev_appserver.py:2952] GET / HTTP/1.0 
500 -

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/c6iLtrN0AcMJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to move data reliably from front to backends ?

Currently moved to the following:
 
Lightweight Score() db object has a timestamp with the following:
  timestamp = db.DateTimeProperty(auto_now=True)
 
B1 backend reaps all Score()'s with a timestamp  1 min old.
 
20 secs after the reap, the B1 deletes all Score()'s with timestamp at 
least 10 min old.
 
My theory is:
 - When the DB is playing up and creating a Score() is VERY slow and so 
they don't show up in queries... having an 'old' Score() already there will 
make the put() strongly consistent and force an instant update.
 - Queries based on a timestamp diff will then return the Score
 - It may take 1 round for a player to get into the leaderboard, since the 
first put() will take some time for things to settle.
 
Coments/thoughts ?
 
Any
On Friday, August 3, 2012 4:34:02 PM UTC-4, Richard wrote:

 Ditched Cloud SQL.  Went back to the old system of savign lightweight DB 
 objects.  I have no idea what to do tonight when DB queries stop working 
 again (like they do every night for the last week!)
  

 On Friday, August 3, 2012 3:55:01 PM UTC-4, Richard wrote:

 Well, Cloud SQL is NOT the answer it is topping out around 500 
 users.  The extra's don't make it into the DB within the 10 second window.  
 Then they get shown in the next window.

 I can 'fix' this by silently deleting the extra's and doing a client 
 update that will insert the user into the results leaderboard if they are 
 not there.  HOWEVER, the user's stats will then possibly be wrong (best 
 game/etc).



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/ArKfUqdFOcQJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: How to move data reliably from front to backends ?

Number of users in a game:
 

522
577
575
602
623
653
259   - WTF ?
684

 
 
So, queries are still slow/lazy/bad/don't work properly. G.
 
The fail list:
 - use a synchronised cron job to reap Score() . had to build my own 
NTP query engine because cron is unreliable under load
 - save object in a put() and query for it later  fails to get all 
objects
 - push objects into a PULL queue and get them using a backend  4-10 
second delay before things can be seen in the PULL queue
 - Cloud SQL ... client submission tops out around 500 entries then the 
backend cannot connect to reap the results!
 - keep Score objects in the DB and just update them . they dont get 
found in a query
  - in memory backend  requires MANY back end instances and does not 
scale automatically
 
Anyone have any other possibilities to try ?
 
-R
 

On Friday, August 3, 2012 5:08:31 PM UTC-4, Richard wrote:

 Currently moved to the following:
  
 Lightweight Score() db object has a timestamp with the following:
   timestamp = db.DateTimeProperty(auto_now=True)
  
 B1 backend reaps all Score()'s with a timestamp  1 min old.
  
 20 secs after the reap, the B1 deletes all Score()'s with timestamp at 
 least 10 min old.
  
 My theory is:
  - When the DB is playing up and creating a Score() is VERY slow and so 
 they don't show up in queries... having an 'old' Score() already there will 
 make the put() strongly consistent and force an instant update.
  - Queries based on a timestamp diff will then return the Score
  - It may take 1 round for a player to get into the leaderboard, since the 
 first put() will take some time for things to settle.
  
 Coments/thoughts ?
  
 Any
 On Friday, August 3, 2012 4:34:02 PM UTC-4, Richard wrote:

 Ditched Cloud SQL.  Went back to the old system of savign lightweight DB 
 objects.  I have no idea what to do tonight when DB queries stop working 
 again (like they do every night for the last week!)
  

 On Friday, August 3, 2012 3:55:01 PM UTC-4, Richard wrote:

 Well, Cloud SQL is NOT the answer it is topping out around 500 
 users.  The extra's don't make it into the DB within the 10 second window.  
 Then they get shown in the next window.

 I can 'fix' this by silently deleting the extra's and doing a client 
 update that will insert the user into the results leaderboard if they are 
 not there.  HOWEVER, the user's stats will then possibly be wrong (best 
 game/etc).



-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Jmu7SkNGZxQJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Custom SSL certificate with more than two intermediate CAs

2012-08-03 Thread johnP

Are you using a VIP certificate?  If so, you need to change the cname, as 
Cayden said.  The ghs cname does not support VIP -  SNI only.  







-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/1I4hK8uqZyoJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

On Sat, Aug 4, 2012 at 6:33 AM, Richard steven...@gmail.com wrote:

 Number of users in a game:


 522
 577
 575
 602
 623
 653
 259   - WTF ?
 684



 So, queries are still slow/lazy/bad/don't work properly. G.

 The fail list:
  - use a synchronised cron job to reap Score() . had to build my own
 NTP query engine because cron is unreliable under load
  - save object in a put() and query for it later  fails to get all
 objects
  - push objects into a PULL queue and get them using a backend  4-10
 second delay before things can be seen in the PULL queue
  - Cloud SQL ... client submission tops out around 500 entries then
 the backend cannot connect to reap the results!
  - keep Score objects in the DB and just update them . they dont get
 found in a query
   - in memory backend  requires MANY back end instances and does not
 scale automatically


Just wanted to make sure...
Have you seen my post about auto-scaling in-memory backends?



 Anyone have any other possibilities to try ?

 -R


 On Friday, August 3, 2012 5:08:31 PM UTC-4, Richard wrote:

 Currently moved to the following:

 Lightweight Score() db object has a timestamp with the following:
   timestamp = db.DateTimeProperty(auto_now=**True)

 B1 backend reaps all Score()'s with a timestamp  1 min old.

 20 secs after the reap, the B1 deletes all Score()'s with timestamp at
 least 10 min old.

 My theory is:
  - When the DB is playing up and creating a Score() is VERY slow and so
 they don't show up in queries... having an 'old' Score() already there will
 make the put() strongly consistent and force an instant update.
  - Queries based on a timestamp diff will then return the Score
  - It may take 1 round for a player to get into the leaderboard, since
 the first put() will take some time for things to settle.

 Coments/thoughts ?

 Any
 On Friday, August 3, 2012 4:34:02 PM UTC-4, Richard wrote:

 Ditched Cloud SQL.  Went back to the old system of savign lightweight DB
 objects.  I have no idea what to do tonight when DB queries stop working
 again (like they do every night for the last week!)


 On Friday, August 3, 2012 3:55:01 PM UTC-4, Richard wrote:

 Well, Cloud SQL is NOT the answer it is topping out around 500
 users.  The extra's don't make it into the DB within the 10 second window.
 Then they get shown in the next window.

 I can 'fix' this by silently deleting the extra's and doing a client
 update that will insert the user into the results leaderboard if they are
 not there.  HOWEVER, the user's stats will then possibly be wrong (best
 game/etc).

  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/Jmu7SkNGZxQJ.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.




-- 
Takashi Matsuo

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-03 Thread Francois Masurel

Will we have an explanation from Google or CloudFlare about what went wrong?

Everything seems to be back to normal at the moment but for how long?  Is 
GAE CDN incompatible?

Thanx for your answers.

Francois

On Wednesday, August 1, 2012 9:06:05 PM UTC+2, Takashi Matsuo (Google) 
wrote:

 Hi Jeff, 

 I've escalated this issue. However, I don't think we can completely 
 turn this off. 
 What was the main reason that you're using CloudFlare? 

 -- Takashi 

 On Thu, Aug 2, 2012 at 3:13 AM, Jeff Schnitzer j...@infohazard.org 
 wrote: 
  Ok, this is fucked up.  Visit http://www.voo.st/, and get this: 
  
  https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg 
  
  Something is triggering a false positive from a totally undocumented 
  Google security system.  This is really, REALLY not ok.  We are losing 
  sales and looking like total idiots to our customers: 
  
  - 
  Our systems have detected unusual traffic from your computer network. 
  Please try your request again later. Why did this happen? 
  
  This page appears when Google automatically detects requests coming 
  from your computer network which appear to be in violation of the 
  Terms of Service. The block will expire shortly after those requests 
  stop. 
  
  This traffic may have been sent by malicious software, a browser 
  plug-in, or a script that sends automated requests. If you share your 
  network connection, ask your administrator for help — a different 
  computer using the same IP address may be responsible. Learn more 
  
  Sometimes you may see this page if you are using advanced terms that 
  robots are known to use, or sending requests very quickly. 
  
  IP address: 208.90.212.26 
  Time: 2012-08-01T18:02:00Z 
  URL: http://www.voo.st/ 
  - 
  
  We use CloudFlare as a reverse proxy.  Wild guess is that some sort of 
  automated security system is cutting in and detecting CF's proxy as an 
  attack. 
  
  PLEASE TURN THIS OFF NOW. 
  
  Jeff 
  
  -- 
  You received this message because you are subscribed to the Google 
 Groups Google App Engine group. 
  To post to this group, send email to google-appengine@googlegroups.com. 
  To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com. 
  For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en. 
  



 -- 
 Takashi Matsuo 


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/hAyrgn0xZQwJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] What are the pros and cons of using Google App engine for my startup?

2012-08-03 Thread Brandon Thomson


I'm building a startup, and I'm considering GAE as the platform, however
I've been having a hard time finding information on why a startup might
consider GAE instead of the many cloud providers out there. Let me explain
what I'm working on.


GAE is really hard to beat for most startups. I think the only reason
I'd avoid it is if I had special needs which it doesn't support yet,
like SSD storage, low-latency communication to virtual machines, or
geographic distribution of application servers.

And if I felt like I needed one of those I'd try hard to reconsider
because GAE is such a pleasure to use and saves a lot of effort and
time.

Brandon

--
Brandon Thomson
b...@brandonthomson.com

--
You received this message because you are subscribed to the Google Groups Google 
App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: What are the pros and cons of using Google App engine for my startup?

It would be a good fit for most of those subjects, but keep in mind
that GAE cannot access email through IMAP or POP3 because it doesn't
support sockets, only standard url fetch. If you want to include email
in your service, you'll have to base a proxy somewhere else, such as
Rackspace, Heroku/AWS, etc.

On Aug 3, 4:37 am, Levi Campbell levicc00...@gmail.com wrote:
 I'm building a startup, and I'm considering GAE as the platform, however
 I've been having a hard time finding information on why a startup might
 consider GAE instead of the many cloud providers out there. Let me explain
 what I'm working on.

 I'm a big fan of David Allen's Getting Things Done: The Art of Stress-Free
 Productivityhttp://www.amazon.com/Getting-Things-Done-Stress-Free-Productivity/dp...,
 and after trying several tools and online services that (claim to)
 implement the GTD methodology, I couldn't find anything that I loved, so I
 decided to build my own and make it available as a SaaS offering. This app
 will allow users to pull in their info_crap from email, facebook, twitter
 (and yes, I do have plans to add support for more social networks.), and
 RSS feeds and organize it by relationship to the sender (i.e. family,
 work colleague, vendor, and the like), project (i.e. planning a family
 vacation.), and context (Either the when or where something should
 happen.).\

 Would GAE be a good fit for the application I'm developing? Why?

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-03 Thread Damon Billian

Hi Francois,

A Google employee already posted about the issue.

Jeff, et al.--

We have verified that a configuration change on our side led to
certain requests being denied / redirected.  The rollback of this
change started earlier this morning and should be completed shortly.
We are actively looking into measures that we can take to ensure that
issues like this are caught prior to rolling out to production.

If your application continues to be impacted please contact me directly.

-- Chris

Product Manager, Google App Engine



On Fri, Aug 3, 2012 at 3:31 PM, Francois Masurel f.masu...@gmail.com wrote:
 Will we have an explanation from Google or CloudFlare about what went wrong?

 Everything seems to be back to normal at the moment but for how long?  Is
 GAE CDN incompatible?

 Thanx for your answers.

 Francois

 On Wednesday, August 1, 2012 9:06:05 PM UTC+2, Takashi Matsuo (Google)
 wrote:

 Hi Jeff,

 I've escalated this issue. However, I don't think we can completely
 turn this off.
 What was the main reason that you're using CloudFlare?

 -- Takashi

 On Thu, Aug 2, 2012 at 3:13 AM, Jeff Schnitzer j...@infohazard.org
 wrote:
  Ok, this is fucked up.  Visit http://www.voo.st/, and get this:
 
  https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg
 
  Something is triggering a false positive from a totally undocumented
  Google security system.  This is really, REALLY not ok.  We are losing
  sales and looking like total idiots to our customers:
 
  -
  Our systems have detected unusual traffic from your computer network.
  Please try your request again later. Why did this happen?
 
  This page appears when Google automatically detects requests coming
  from your computer network which appear to be in violation of the
  Terms of Service. The block will expire shortly after those requests
  stop.
 
  This traffic may have been sent by malicious software, a browser
  plug-in, or a script that sends automated requests. If you share your
  network connection, ask your administrator for help — a different
  computer using the same IP address may be responsible. Learn more
 
  Sometimes you may see this page if you are using advanced terms that
  robots are known to use, or sending requests very quickly.
 
  IP address: 208.90.212.26
  Time: 2012-08-01T18:02:00Z
  URL: http://www.voo.st/
  -
 
  We use CloudFlare as a reverse proxy.  Wild guess is that some sort of
  automated security system is cutting in and detecting CF's proxy as an
  attack.
 
  PLEASE TURN THIS OFF NOW.
 
  Jeff
 
  --
  You received this message because you are subscribed to the Google
  Groups Google App Engine group.
  To post to this group, send email to google-appengine@googlegroups.com.
  To unsubscribe from this group, send email to
  google-appengine+unsubscr...@googlegroups.com.
  For more options, visit this group at
  http://groups.google.com/group/google-appengine?hl=en.
 



 --
 Takashi Matsuo

 --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/hAyrgn0xZQwJ.

 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Is Google Apps for your Domain going away and what's the impact for GAE users?

2012-08-03 Thread Emanuele Ziglioli

I'm puzzled: http://support.google.com/a/bin/answer.py?hl=enanswer=2669622

1. What's apps for your team? Google always used apps for your domain, 
where did that name come from
2. GAE has forced us to create a Google Apps for your Domain account in 
order to map the app to a custom domain and for using SSL
3. Are being forced to upgrade our free Google Apps for your Domain to a 
pay-for Apps for Business? Enough of paying, enough of multiple bills, 
can't you just have one Google bill that covers everything?
4. Three week notice? you must be joking!

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/9Bz56OyDXPIJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

Hi Takashi,

Yes, I read your post with a theoretical model, but unfortunately, I don't 
really know how to tell it to scale up/down ?

-R

On Friday, August 3, 2012 6:07:10 PM UTC-4, Takashi Matsuo (Google) wrote:


 Just wanted to make sure...
 Have you seen my post about auto-scaling in-memory backends?




-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/9Z77HBrZDEYJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] Is Google Apps for your Domain going away and what's the impact for GAE users?

2012-08-03 Thread Drake

Apps for your Team is a thing no one used, and is different from apps for
domains.

This likely won't affect you unless you got bamboozled a long while ago by
one of the people who was a Apps Affiliate.

And you'd remember because the thing would have ruined your life so long ago
that you'd never have moved to GAE.

 

From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of Emanuele Ziglioli
Sent: Friday, August 03, 2012 5:01 PM
To: google-appengine@googlegroups.com
Subject: [google-appengine] Is Google Apps for your Domain going away and
what's the impact for GAE users?

 

I'm puzzled: http://support.google.com/a/bin/answer.py?hl=en
http://support.google.com/a/bin/answer.py?hl=enanswer=2669622
answer=2669622

 

1. What's apps for your team? Google always used apps for your domain,
where did that name come from

2. GAE has forced us to create a Google Apps for your Domain account in
order to map the app to a custom domain and for using SSL

3. Are being forced to upgrade our free Google Apps for your Domain to a
pay-for Apps for Business? Enough of paying, enough of multiple bills,
can't you just have one Google bill that covers everything?

4. Three week notice? you must be joking!

-- 
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/9Bz56OyDXPIJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

Dynamic backend instances automatically scale up/down.
On Aug 4, 2012 9:21 AM, Richard steven...@gmail.com wrote:

 Hi Takashi,

 Yes, I read your post with a theoretical model, but unfortunately, I don't
 really know how to tell it to scale up/down ?

 -R

 On Friday, August 3, 2012 6:07:10 PM UTC-4, Takashi Matsuo (Google) wrote:


 Just wanted to make sure...
 Have you seen my post about auto-scaling in-memory backends?


  --
 You received this message because you are subscribed to the Google Groups
 Google App Engine group.
 To view this discussion on the web visit
 https://groups.google.com/d/msg/google-appengine/-/9Z77HBrZDEYJ.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at
 http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: How to move data reliably from front to backends ?