Re: [google-appengine] Need help: Pathological instancing behavior

2012-08-01 Thread Kristopher Giesing
So, this behavior is completely reproducible. 1) Shut down all instances. 2) Set min instances to 1 (leave all other sliders at default values). 3) Save settings and wait for instance to spin up. 4) Issue one GET request from browser to a specific API endpoint. This is an API call so it is not

Re: [google-appengine] Need help: Pathological instancing behavior

2012-08-01 Thread Kristopher Giesing
The above post notwithstanding, the original problem I was having - which caused me to try to set min instances to 1 to begin with - seems to have gone away. I know this is probably going to fall into the PEBCAK bucket, but I really don't think I did anything in particular to cause the

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Jeff Schnitzer
It's hard to imagine this working without some sort of background thread in the frontend instances. Otherwise how do you get the frontend to commit its data to whatever datasource you want? You can't really hold open every connection for 2s... at max concurrency of 10, you'd need 100 instances.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Richard Watson
On Wed, Aug 1, 2012 at 10:50 AM, Jeff Schnitzer j...@infohazard.org wrote: It's hard to imagine this working without some sort of background thread in the frontend instances. Otherwise how do you get the frontend to commit its data to whatever datasource you want? I suspect I misunderstand

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Jeff Schnitzer
On Wed, Aug 1, 2012 at 2:36 AM, Richard Watson richard.wat...@gmail.com wrote: You could also scale the amount in the list depending on how many requests per second you're getting. If 1000, batch like mad. If 20, submit every request. The risk with this is that you get 1000's, then

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread alex
I don't know. Do you? If not, I think it is at least worth a consideration. -- alex On Aug 1, 2012 1:33 AM, Jeff Schnitzer j...@infohazard.org wrote: On Tue, Jul 31, 2012 at 3:48 PM, alex a...@cloudware.it wrote: At this point I think the GAE toolbox is empty. How about Cloud SQL? Can

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Richard Watson
On Wed, Aug 1, 2012 at 12:13 PM, Jeff Schnitzer j...@infohazard.org wrote: Anything that involves batching in a frontend risks orphaning data in the frontend... there's just no efective way to ensure that batching happens and that the queue is purged when done. It's definitely not

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Richard
The thing is GAE does a wonderful job. It has been fine for a year or so. Now suddenly, we are getting bitten by 'eventual consistency'. And not at peak load either! This is hitting us at the lowest load time and at the same time each day. So, maybe we were just lucky before

Re: [google-appengine] Re: Observed: put()'s completion time is dependent on time of day

2012-08-01 Thread Richard
Hi Takashi, This is exactly my problem. Along with the fact that it seems to be time-of-day dependent. Luckily/unluckily, this just happens to coincide with the time when I have the least number of users playing. -R On Wednesday, August 1, 2012 1:16:54 AM UTC-4, Takashi Matsuo (Google)

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Takashi Matsuo
Hi Richard, I actually played your game and probably encountered the exact issue. BTW, it's a cool addictive game. I agree that eventually consistent window should ideally be more steady than now. I have passed your issue to the engineering team. I'll get back to you when I have some updates.

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Takashi Matsuo
I forgot to mention about one possible workaround which could mitigate your pain. In the frontend instances, maybe you can get those small entities with a newly created keys just after putting them in order to apply the change locally. This article describes this behavior well:

[google-appengine] size of allocated memcache

2012-08-01 Thread Aleksei Rovenski
Hi all, I have somewhat wierd memcache stats. As I understood from reading the docs memcache size grows with the traffic of your app. However in practice I don't see that, quite the opposite actually. I have 2 apps, one is for development (almost zero traffic) and the other one is live (5-30

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Richard
Ok, so I read that web page, and I understand that the put() has completed, but not the commit(). However, I don't really see the potential solution you are suggesting :( Can you (or someone else) please explain in a bit more detail ? On Wednesday, August 1, 2012 10:55:35 AM UTC-4, Takashi

Re: [google-appengine] Re: Observed: put()'s completion time is dependent on time of day

2012-08-01 Thread Hernan Liendo
We've also the same problem. We even had to implement a versioning controlling algorithm to prevent ugly dirty reads. Our appid is 'racetownzc' On Wednesday, August 1, 2012 10:57:50 AM UTC-3, Richard wrote: Hi Takashi, This is exactly my problem. Along with the fact that it seems to be

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Jeff Schnitzer
I presume the essential line is this: In the (now standard) High Replication Datastore, the transaction typically is completely applied within a few hundred milliseconds after the commit returns. However, even if it is not completely applied, subsequent reads, writes, and ancestor queries will

[google-appengine] Unable to see/join the hang-out Asynchronous Tasks with App Engine: 'Last to Cross the Finish Line'

2012-08-01 Thread Ian Marshall
Was I the only person unable to see/join the hang-out Asynchronous Tasks with App Engine: 'Last to Cross the Finish Line' at https://developers.google.com/events/ahNzfmdvb2dsZS1kZXZlbG9wZXJzcg4LEgVFdmVudBjW7asDDA ? It was due to start at 0900 hrs today (US Pacific Time) and to last 1 hour.

[google-appengine] Re: Unable to see/join the hang-out Asynchronous Tasks with App Engine: 'Last to Cross the Finish Line'

2012-08-01 Thread Ian Marshall
Afternote: the URL which I pasted into my post does not work. Please disregard it. On Aug 1, 6:13 pm, Ian Marshall ianmarshall...@gmail.com wrote: Was I the only person unable to see/join the hang-out Asynchronous Tasks with App Engine: 'Last to Cross the Finish Line' at    

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Richard
Yep... got that. However, a query() will still return stale data between the put() and the (internal) commit(). Which is where I believe I am sitting... On Wednesday, August 1, 2012 12:39:41 PM UTC-4, Jeff Schnitzer wrote: I presume the essential line is this: In the (now standard) High

[google-appengine] Custom SSL certificate with more than two intermediate CAs

2012-08-01 Thread Nacho Coloma
Hi, I have just configured a certificate for our own custom domain (VIP) and it is working fine, but Android browsers are rejecting to connect. Investigating, it seems that I should include the full chain of intermediate CAs to the uploaded PEM file, but that's not possible since AppEngine

[google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Jeff Schnitzer
Ok, this is fucked up. Visit http://www.voo.st/, and get this: https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg Something is triggering a false positive from a totally undocumented Google security system. This is really, REALLY not ok. We are losing sales and looking like total

[google-appengine] HEAD request not supported

2012-08-01 Thread Kate
I am getting the error bellow which I understand refers to an unsupported HEAD request. However the only form I have is a google search form. So I do not understand why I am getting this error. What should I look for? 128.227.150.12 - - [01/Aug/2012:11:17:36 -0700] HEAD / HTTP/1.1 405 124 -

Re: [google-appengine] How to move data reliably from front to backends ?

2012-08-01 Thread Joshua Smith
I haven't been following this thread to closely. Can you summarize the problem as you understand it at this point? On Aug 1, 2012, at 2:01 PM, Richard steven...@gmail.com wrote: Yep... got that. However, a query() will still return stale data between the put() and the (internal) commit().

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Richard
Ok, based on Takashi's suggestion, I now do the following: new_user_score = Score() new_user_score.member = thing new_user_score.put() # next part is new based on Takashi's suggestion k = new_user_score.key() db.get(k) WOW... I mean... HOLY CRIPES! Time to process each request went from

Re: [google-appengine] How to move data reliably from front to backends ?

2012-08-01 Thread Richard
Summary so far: I have a massive multiplayer Android game. It has tight timings. Sometimes (at the same time of day.. when load is lowest), GAE will do a put() and 5-10 seconds later when I do a query all to create leaderboards for that game round, I get stale results. for some of the

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Michael Hermus
If you collect the keys of newly created 'small entities' and send them in batch to the back end for reaping, you could use a batch get() which would force all the entities to roll forward, and eliminates the eventual consistency issue. Unfortunately, I believe that no matter the solution

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Michael Hermus
Our posts crossed paths in the ether. I don't think you should do 'get(key)' per request on the front end; rather, you should collect the keys and send them to the back end for use in a single batch get() call. It still might not completely suit your needs, but it would be much closer. On

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Richard
The backend has 5 seconds to do reap all the results, create a leaderboard and make it available to the front ends. This part runs in under 2 seconds typically atm. I don't believe there is any way to move the keys to the back end (PULL queues and urlget has been shown to not work), execute a

Re: [google-appengine] How to move data reliably from front to backends ?

2012-08-01 Thread Joshua Smith
I don't know if this will help (depends on some specifics of your data model), but I've found that using a query to get a list of entities, followed by a get to actually get the data, is a good workaround for many eventual consistency issues. Here's snippet of code that provides the

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Takashi Matsuo
On Thu, Aug 2, 2012 at 3:24 AM, Richard steven...@gmail.com wrote: Ok, based on Takashi's suggestion, I now do the following: new_user_score = Score() new_user_score.member = thing new_user_score.put() # next part is new based on Takashi's suggestion k = new_user_score.key() db.get(k)

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Takashi Matsuo
this: https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg Something is triggering a false positive from a totally undocumented Google security system. This is really, REALLY not ok. We are losing sales and looking like total idiots to our customers: - Our systems have detected

[google-appengine] Re: APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Kaan Soral
There were many people experiencing the same problem before, after you spoke very highly of CloudFlare I asked them If they experience any problems with GAE, they said no, but I didn't believe These are all offtopic, another offtopic question: Why don't you switch to GAE SSL? Is it the

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Richard
Hi Takashi Joshua, How about using my user's id as the key ? eg: new_score = Score(key_name='ahFkZXZ-c3Zlbi13b3JkaGVyb3ILCxIEVXNlchjFAww') I assume this would not count as monotonically increasing ? @ Joshua: At the moment, I create all new Score objects for each player of that game round

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Jeff Schnitzer
/, and get this: https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg Something is triggering a false positive from a totally undocumented Google security system. This is really, REALLY not ok. We are losing sales and looking like total idiots to our customers: - Our systems have

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Damon Billian
://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg Something is triggering a false positive from a totally undocumented Google security system. This is really, REALLY not ok. We are losing sales and looking like total idiots to our customers: - Our systems have detected unusual traffic from

[google-appengine] Re: APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Jeff Schnitzer
On Wed, Aug 1, 2012 at 12:10 PM, Kaan Soral kaanso...@gmail.com wrote: There were many people experiencing the same problem before, after you spoke very highly of CloudFlare I asked them If they experience any problems with GAE, they said no, but I didn't believe I'm pissed because this

Re: [google-appengine] How to move data reliably from front to backends ?

2012-08-01 Thread Joshua Smith
I think that approach sounds pretty good. The trick of pasting the new user's data in client-side is similar to a technique I have used to ensure newly added records appear in the list. On Aug 1, 2012, at 3:11 PM, Richard steven...@gmail.com wrote: Comments/thoughts ? -- You received this

[google-appengine] Why I am I getting these HEAD / HTTP/1.1 errors?

2012-08-01 Thread Kate
I can't work out why I am getting these errors. In 13 hours I have 26,205 of them. I have no forms in my code. I do have urlfetch If as I have been told the error means that the head method is noot implemented, how do I set up my app to support the Head methiod? I cannot find anywhere I

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Takashi Matsuo
On Thu, Aug 2, 2012 at 4:11 AM, Richard steven...@gmail.com wrote: Hi Takashi Joshua, How about using my user's id as the key ? eg: new_score = Score(key_name='ahFkZXZ-c3Zlbi13b3JkaGVyb3ILCxIEVXNlchjFAww') I assume this would not count as monotonically increasing ? I don't think it's ok.

Re: [google-appengine] Why I am I getting these HEAD / HTTP/1.1 errors?

2012-08-01 Thread Joshua Smith
Looks to me like somebody is doing head requests to your app using curl. curl is a command line program that downloads web pages, similar to wget. This particular user is coming from linux, possibly from here: Portugal, Braga, Universidade do Minho (although reverse IP search results aren't

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Richard
Hi Takashi, Yeah, the first few characters would probably be very similar. I am not sure if that would be a problem ? A single user has only one score in the leaderboard for the game. So, overwriting the score the next time is not a problem, provided we know when to NOT include a user in the

[google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Mauricio Aristizabal
As others have said, why not Cloud SQL? Mysql can handle tens of thousands of inserts per second into an average-width table: http://www.mysqlperformanceblog.com/2012/05/16/benchmarking-single-row-insert-performance-on-amazon-ec2/ I don't know what kind of performance you can expect from

Re: [google-appengine] Why I am I getting these HEAD / HTTP/1.1 errors?

2012-08-01 Thread Kate
Thanks for that, but these requests are coming from many different ip addresses and I can see no pattern to them. I am just wondering why there are so many and if it is affecting my quota. Also I do not have a main.py. I have an index.py. I'm wondering now how it works at all! On Wednesday,

[google-appengine] Re: APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Jeff Schnitzer
On Wed, Aug 1, 2012 at 12:14 PM, Jeff Schnitzer j...@infohazard.org wrote: I'm pissed because this behavior is undocumented. Google doesn't say don't access your site through a shared reverse proxy because we may shut you down with some sort of automated threat detection. This isn't just a

[google-appengine] Thousands of head requests to my app using curl

2012-08-01 Thread Kate
See example below. I am getting thousands of these per day and want to stop them. They are throwing HEAD / HTTP/1/1 errors. They come from different IP addresses. I'm stumped! 2012-08-01 14:25:52.668 / 405 51ms 0kb curl/7.18.2 (i386-redhat-linux-gnu) libcurl/7.18.2 NSS/3.12.2.0

[google-appengine] How can I block curl requests

2012-08-01 Thread Kate
I am getting tens of thousands of curl requests - many thousands time browser requests and I want to block them. I'm using python. They coming from many different IPs most in Europe. If I can't stop them I will have to close my site or go to a new provider. Thank in advance, Kate --

[google-appengine] Re: APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Damon Billian
190.93.240.0/20 IPv6 2400:cb00::/32 2606:4700::/32 2803:f800::/32 On Wednesday, August 1, 2012 11:13:42 AM UTC-7, Jeff Schnitzer wrote: Ok, this is fucked up. Visit http://www.voo.st/, and get this: https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg Something is triggering

[google-appengine] Re: APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Francois Masurel
Getting the same really annoying error for a very very low traffic web site 5 requests per minute : http://www.filhot.com How can this be? I'm using CloudFlare too. Google, please respond. Francois On Wednesday, August 1, 2012 10:08:42 PM UTC+2, Jeff Schnitzer wrote: On Wed, Aug 1, 2012

Re: [google-appengine] How can I block curl requests

2012-08-01 Thread Rerngvit Yanggratoke
I think you can blacklist those IPs. Have a look in https://developers.google.com/appengine/docs/java/config/dos#About_dos_xmlor https://developers.google.com/appengine/docs/python/config/dos. On Wed, Aug 1, 2012 at 10:43 PM, Kate mss.ka...@gmail.com wrote: I am getting tens of thousands of

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Francois Masurel
On Thu, Aug 2, 2012 at 3:13 AM, Jeff Schnitzer j...@infohazard.org wrote: Ok, this is fucked up. Visit http://www.voo.st/, and get this: https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg Something is triggering a false positive from a totally undocumented Google

[google-appengine] My site bombarded with thousands of URL requests from different IPs - how to stop them

2012-08-01 Thread Kate
I've posted about this problem before but I don't think I have explained it adequately. I am getting tens of thousands of CURL requests throwing HEAD method errors. This is thousands of times more than normal browser requests. I can't block them as they come from different IPs and you are only

Re: [google-appengine] How can I block curl requests

2012-08-01 Thread Kate
Yes I can but there's a limit to the number - 100 I think, and there are tens of thousands of them. I guess I'll just have to move from GAE as there doesn't seem any way of doing this and I can't be paying for these requests. On Wednesday, August 1, 2012 4:51:04 PM UTC-4, rerngvit yanggratoke

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Jeff Schnitzer
This definitely sounds like a good strategy... especially with memory tables and no indexes. Back when cloud sql was in beta, there was a restriction of 5 queries per second, so I wrote it off for doing any kind of heavy lifting. Presumably that restriction has been lifted. Jeff On Wed, Aug 1,

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Jeff Schnitzer
Would this really help? There's still going to be a splitting issue with the index tablet, and that one you will have a hard time distributing the keyspace for. Jeff On Wed, Aug 1, 2012 at 12:52 PM, Richard steven...@gmail.com wrote: Hi Takashi, Yeah, the first few characters would probably

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Rick Mangi
we can completely turn this off. What was the main reason that you're using CloudFlare? -- Takashi On Thu, Aug 2, 2012 at 3:13 AM, Jeff Schnitzer j...@infohazard.org wrote: Ok, this is fucked up. Visit http://www.voo.st/, and get this: https://img.skitch.com/20120801

[google-appengine] Re: Instances/Java go crazy

2012-08-01 Thread Mos
And again I have to pay for GAE issues: On Jul 30 the Frontend Instance Hours goes beyond the free limit. First time this week in my application history. Remember: The application was unusable on this day because GAE starts instances like crazy (on low traffic). Frontent Instance Hours were

Re: [google-appengine] My site bombarded with thousands of URL requests from different IPs - how to stop them

2012-08-01 Thread Joshua Smith
It really sounds like your site is under attack (specifically, a DDOS attack). Switching hosting providers won't help (they'll just attack the new site). Blocking based on the UA string (to reject curl requests only) is not a standard feature of any hosting service. And besides, spoofing the UA

Re: [google-appengine] How can I block curl requests

2012-08-01 Thread Kyle Finley
Hi Kate, Maybe you could look into cloudflare.com it offers protection agains DDOShttp://en.wikipedia.org/wiki/Denial-of-service_attack attacks, which is what these request appear to be. - Kyle -- You received this message because you are subscribed to the Google Groups Google App Engine

[google-appengine] can Compute Engine VM talk to AppEngine Datastore?

2012-08-01 Thread Trung Pham
I'm interested in running my frontend servers in ComputeEngine cloud but have my apps use DataStore. The reason why I want to do this so I can have long running websocket sessions open. And being able to use NodeJS runtime. Thanks. -- You received this message because you are subscribed to

[google-appengine] Re: Error accesing application on GAE

2012-08-01 Thread Fábio Peruchi
Hi! I'm having this error below on my first access to my GAE application after I migrated my application from *Master/Slave Datastore* to *High Replication Datastore*. Can you help me with this? 1. 2012-07-30 06:49:36.444 Nested in org.springframework.beans.factory.BeanCreationException:

[google-appengine] Can I retrieve pop3 mails using Gmail from server which is behind firewall and only can be accessed through VPN?

2012-08-01 Thread 1moretime
i have pop3 account in office, which can be accessed outside office by VPN dialup. i want to use Gmail as my email client. is it possible to setup a 'GAE VPN mail server' so that Gmail can access and retrieve my work mail? -- You received this message because you are subscribed to the Google

[google-appengine] Re: simple application does not work!!!

2012-08-01 Thread blknit
Yes It was not worked well.But you can use another method to make it work well see my appspot at *http://studywebapp.appspot.com/*http://studywebapp.appspot.com/ you will get the answer. 在 2012年6月16日星期六UTC+8上午11时47分22秒,bhcuong2008写道: Hi, here is my simple app, but does not work test.py:

[google-appengine] Re: Early access to Page Speed Service for Google App Engine

2012-08-01 Thread Trung Pham
switch to golang? On Monday, July 9, 2012 11:00:19 PM UTC-7, Anders wrote: You really need to fix the slow cold starts of GAE apps. It can take 10 seconds to load a webpage that usually loads in a fraction of a second. See issue:

[google-appengine] Application stuck Activating Billing

2012-08-01 Thread Paul Carvill
My application — ID: lbi-pulse — is currently stuck in Activating Billing status, and has been for 24 hours. Is there any way of getting support or resolving this issue? thanks, paul -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view

[google-appengine] How to show favicon.ico runtime is go?

2012-08-01 Thread Cloud Tao
Dear all: in my localhost, it can display favicon.ico in the browser, when i upload gae server, it can't show. directory tree: app.yaml static webapp follow is my fiile content, yaml: - url: /favicon\.ico static_files: static/images/favicon.ico upload: static/images/favicon.ico file.go:

[google-appengine] Re: HTTP Digest - nonce calculation?

2012-08-01 Thread kpriya
hii, i want to ask that how to calculate nonce?actually i am writing code for counter -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit

[google-appengine] Problem with storing a dictionary as a list in live GAE

2012-08-01 Thread Sean Bedford
Hi all, I'm in the process of updating one of my GAE app data sets to include a dictionary-esque object within the data store. To achieve this, I have written some helper functions that convert key/value pairs to single entities within a list that are split by a keyword. This seems to work

Re: [google-appengine] deploying the App Engine: I just can seem to do it!

2012-08-01 Thread Hrushikesh Patil
Hi, Although I am registered getting the same error I how do i get the whitelisted appspot app for my account? Thanks Hrushi On Thursday, July 12, 2012 3:19:18 AM UTC+5:30, Dan HoHilevoet wrote: Hi, Google Cloud Endpoints is currently a trusted tester feature. In order to be able to

Re: [google-appengine] Subclass my model classes

2012-08-01 Thread Neal Ehardt
For the benefit of the archive, the solution is the PolyModel class. This was fixed in 2009. https://developers.google.com/appengine/articles/polymodel -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web

[google-appengine] How to solve the aggregate query with condition in Google App Engine

2012-08-01 Thread Neo
Suppose, In my website, I ask users to input some string. A user can input string multiple times. Whenever any user inputs a string, I log it in the database along with the time. Many strings can be same, even though inputted by different users. In the home page, I need to give the interface

[google-appengine] Unique Integer ID for a non primary key field for Entities in Google App Engine

2012-08-01 Thread Neo
I have an Entity type say URLInfo which keeps info about URLs. The primary key of this entity is URL itself ( that makes sure that I always have unique URLs in the datastore). I also want unique integer id for each url so that sharing the id becomes easier. Though, I can use GUIDs, but that

[google-appengine] Email sent but not received on other side

2012-08-01 Thread Shia Poems
I have this google apps email and I am bcc'ing some people and I send it out, I get no bounce-back messege and no error. And it goes into my sent email. But it never reaches the other side? Why? I have another 2 other google apps domain, and those work perfectly. the only difference between

[google-appengine] Re: APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Evan Mezeske
I'd just like to jump in and mention that this is happening to my app as well (www.schoolseatingcharts.com). I am in the same boat as Jeff: I use CloudFlare in front of my appengine service. My main reason is the same as Jeff's: CloudFlare offers VIP SSL at 1/5 the price of appengine. I

Re: [google-appengine] TransientError on Text Search API ?

2012-08-01 Thread Yoav Aviram
I had the same issue found out it was caused by an incorrect field default type (string instead of a NumberField). On Sunday, June 24, 2012 4:13:26 PM UTC+1, John Wheeler wrote: Steve, I'm running across this error now with the same code pretty much, except I'm trying to sort by a

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread masterblaster
On Thu, Aug 2, 2012 at 3:13 AM, Jeff Schnitzer j...@infohazard.org wrote: Ok, this is fucked up. Visit http://www.voo.st/, and get this: https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg Something is triggering a false positive from a totally undocumented Google

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Damon Billian
, this is fucked up. Visit http://www.voo.st/, and get this: https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg Something is triggering a false positive from a totally undocumented Google security system. This is really, REALLY not ok. We are losing sales and looking like total idiots

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Francois MASUREL
this off. What was the main reason that you're using CloudFlare? -- Takashi On Thu, Aug 2, 2012 at 3:13 AM, Jeff Schnitzer j...@infohazard.org wrote: Ok, this is fucked up. Visit http://www.voo.st/, and get this: https://img.skitch.com/20120801-cd1h98pqwb8e8qryct9yjcqwgk.jpg

[google-appengine] 10mb deploy size limit

2012-08-01 Thread Phil McDonnell
Does anyone know what is counted in the size limit for deploying? It would seem my war and src directories are counted. Are jars counted too? Are they compressed? I'm trying to trim the size of my project, but I'm not entirely sure why it's too big. I assume there is no way to expand this limit?

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Damon Billian
completely turn this off. What was the main reason that you're using CloudFlare? -- Takashi On Thu, Aug 2, 2012 at 3:13 AM, Jeff Schnitzer j...@infohazard.org wrote: Ok, this is fucked up. Visit http://www.voo.st/, and get this: https://img.skitch.com/20120801

RE: [google-appengine] How can I block curl requests

2012-08-01 Thread Drake
You can check the user agent of the request. Is the Use agent the same? From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Kate Sent: Wednesday, August 01, 2012 2:15 PM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] How can I

RE: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Drake
CloudFlare biting you in the ass. It happens. Bonus... Cloudflare doesn't form its proxy request headers correctly. Check the archive this isn't undocumented I've mentioned a lot when helping other people with the same error. -- You received this message because you are subscribed to the

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Jeff Schnitzer
On Wed, Aug 1, 2012 at 3:47 PM, Drake drak...@digerat.com wrote: CloudFlare biting you in the ass. What stops me from crafting requests that Google will recognize as an attack and running them through your CDN in the middle of the Survivor season finale, shutting you down? I mean, other than

Re: [google-appengine] My site bombarded with thousands of URL requests from different IPs - how to stop them

2012-08-01 Thread Kate
Thanks Joshue, How do I file a production issue? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/xPmxxyiQfPsJ. To post to this group, send email to

Re: [google-appengine] How can I block curl requests

2012-08-01 Thread Jeff Schnitzer
On Wed, Aug 1, 2012 at 2:55 PM, Kyle Finley kylefin...@gmail.com wrote: Hi Kate, Maybe you could look into cloudflare.com it offers protection agains DDOS attacks, which is what these request appear to be. Oh, the irony... Jeff -- You received this message because you are subscribed to the

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Joshua Smith
Who had 3:47pm PDT in the pool for when Brandon would pipe up with his I told you so? Anyone? :) On Aug 1, 2012, at 6:47 PM, Drake drak...@digerat.com wrote: CloudFlare biting you in the ass. It happens. Bonus... Cloudflare doesn't form its proxy request headers correctly. Check the

Re: [google-appengine] My site bombarded with thousands of URL requests from different IPs - how to stop them

2012-08-01 Thread Joshua Smith
http://code.google.com/p/googleappengine/issues/list New Issue Production Issue template Follow the instructions in the template On Aug 1, 2012, at 6:56 PM, Kate mss.ka...@gmail.com wrote: Thanks Joshue, How do I file a production issue? -- You received this message because you are

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Damon Billian
Whatever, Drake (Brandon). Hi Jeff, We're in contact with our Google contact to find out what the issue is. I'll post as I have any additional details to share. On Wed, Aug 1, 2012 at 3:47 PM, Drake drak...@digerat.com wrote: CloudFlare biting you in the ass. It happens. Bonus...

RE: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Drake
Not if they form their request headers properly. CF sends IPs that are malformed, nonsensical, and non-responsive. (and often blank) My CDN solution says Request on behalf of X the right way. Oh, and CF has already tried doing that to me. More than once. -- You received this message because

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Damon Billian
Hi, We have heard back from our contact at Google and this is being worked on (don't have an eta for a full fix yet will let you know...but hoping within the next day or less). It also looks like our IPs are going to get whitelisted to prevent these issues in the future. On Wed, Aug 1, 2012

Re: [google-appengine] How can I block curl requests

2012-08-01 Thread Kyle Finley
Yes, I should mention that today might not be a good day to test CloudFlare, though. https://groups.google.com/forum/?fromgroups#!topic/google-appengine/Q6yQ4d9ov-o On Aug 1, 2012, at 5:59 PM, Jeff Schnitzer wrote: On Wed, Aug 1, 2012 at 2:55 PM, Kyle Finley kylefin...@gmail.com wrote: Hi

RE: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Drake
That's what y'all say every 9 months... https://groups.google.com/forum/?fromgroups#!topic/google-appengine/om1DF_61 Xl4 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Jeff Schnitzer
What evidence do you have to suspect that Google is treating CF as an attack because of the way they form headers? That doesn't even make sense - if CF is mangling headers, presumably they've been doing it for a while, and it wasn't being blocked before. More likely there's some sort of real

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Damon Billian
Hi Drake, We have already had the confirmation from the Google Security Engineer that is our contact that he will look into this. It is probably best not to comment on discussions that you are not involved in can't read minds. On Wed, Aug 1, 2012 at 4:07 PM, Drake drak...@digerat.com wrote:

Re: [google-appengine] How can I block curl requests

2012-08-01 Thread Kyle Finley
Even if she did the check in a middleware, she would still have to handle/reject the requests by spinning up instances, though. Right? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit

RE: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Drake
Instead of whitelisting your IPs Fix your Proxy. Google is not the only place with these issues. Don't lie about the IP the request is on behalf of, don't create IPs that don't exist or are in unassigned ranges. Don't try to piggy back session headers that aren't from the same session. Google

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Jeff Schnitzer
On Wed, Aug 1, 2012 at 4:07 PM, Drake drak...@digerat.com wrote: That's what y'all say every 9 months... https://groups.google.com/forum/?fromgroups#!topic/google-appengine/om1DF_61 Xl4 Brandon, this is why I can't trust anything you say anymore. That thread is unrelated to this issue, and

RE: [google-appengine] How can I block curl requests

2012-08-01 Thread Drake
Yes you'd still have to check user agent, but that could be very early in the processing, and returning a denied message will typically cause the offender to go away. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Kyle Finley Sent: Wednesday,

Re: [google-appengine] Re: How to move data reliably from front to backends ?

2012-08-01 Thread Mauricio Aristizabal
I think the 5 QPS restriction is still there... but it's only for connections from outside GAE, AFAIK to discourage using CloudSQL from an outside app. For GAE there are only limits on concurrent connections (1000) and requests (100), and request/response size (16MB)

Re: [google-appengine] APP DOWN due to some sort of undocumented Google security system

2012-08-01 Thread Damon Billian
you are being deliberately disingenuous. I vote for this, personally. Hi Drake. I'm pretty sure our engineering team, given their background at major internet companies and the like, are probably pretty familiar with creating properly formed headers. On Wed, Aug 1, 2012 at 4:15 PM, Jeff

[google-appengine] Re: My site bombarded with thousands of URL requests from different IPs - how to stop them

2012-08-01 Thread hyperflame
In the interim, it would be a good idea to put up a static page stating that your website is under emergency maintenance or similar, just so your customers have some information. On Aug 1, 6:01 pm, Joshua Smith joshuaesm...@charter.net wrote: http://code.google.com/p/googleappengine/issues/list

[google-appengine] Re: 10mb deploy size limit

2012-08-01 Thread Phil McDonnell
Did something recently change here? I've been trimming the fat from my app for half the afternoon and I'm still not there. This wasn't an issue for my team as of yesterday. Any ideas what could be happening? Any ideas on debugging it? Are other seeing this? Thanks, Phil On Wed, Aug 1, 2012 at

  1   2   >