[google-appengine] [Ideas] NodeJS on Appengine - Local Development Approach in 2021

[Kaan Soral]

Hi everyone,

Old timers might remember me, I've been using App Engine since inception, 
started with Python and continued with Python, originally, App Engine 
always provided a fully functioning replica of all services for local 
development, nowadays nothing is available for local testing

I wonder if anyone is developing a multi-functional NodeJS app that uses 
Tasks, Databases, Storage etc. and how you are approaching local 
development (These are the initial services that come to mind, but there 
are soo many more now)

Back when I last tested ( 
https://groups.google.com/g/google-appengine/c/EzAptrVl4SU/m/xfLqEcG9AwAJ ) 
- It seemed like the best option was to just deploy and test everything 
live, but this is obviously soo slow and so unwelcoming

I guess a logical idea is to mimic what can be mimicked, and use live 
versions of other services? Example:

*Local App*

   1. Live Firebase: local_app
   2. Live Storage: local_storage
   3. Mock Up Tasks -> Just uses setTimeout

*Live App*

   1. Live Firebase: live_app
   2. Live Storage: live_storage
   3. Cloud Tasks

Obviously this is an over simplification, as far as I see, if you 
rewrite/reroute urls etc. - It would be challenging to keep things in sync, 
the app would need to stay simple

I'm just wondering if anyone else has solved this problem or have their own 
approaches and thoughts - and maybe challenges they've overcomed (I mean, 
even the environment variable approach to settings is a challenge these 
days)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b85f0196-af84-455b-9cd5-b51d885bfdb9n%40googlegroups.com.

Re: [google-appengine] Re: [Unimportant] Old Python 2.7 Users: Anyway to shrink the local .rdbms / reduce it's size?

[Kaan Soral]

Thank you so much Alex, dropped from 215 to 10 :)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/d9995509-fffb-41ad-a685-a770902bca6bo%40googlegroups.com.

[google-appengine] Re: [Unimportant] Old Python 2.7 Users: Anyway to shrink the local .rdbms / reduce it's size?

[Kaan Soral]

Whatever system the old dev_appserver.py is using, I forgot the technology

On Monday, June 22, 2020 at 6:26:43 PM UTC+3, Jose V wrote:
>
> Hi, Could you please elaborate further on which GCP database system are 
> you using?
>
> On Saturday, June 20, 2020 at 8:36:51 PM UTC+2, Kaan Soral wrote:
>>
>> Hi Old Python 2.7 App Engine users,
>>
>> I have an unimportant question, maybe one of you figured it out
>>
>> My db.rdbms on one project is 200mb's, on another 600mb's, they both spin 
>> up instantly locally, so it doesn't matter too much, but I decided to 
>> delete unimportant entries, reduce the db size, however it seems to never 
>> reduce in size (Deleted every entry in 2 large tables, before 215.1MB's, 
>> after 215.1MB's :/)
>>
>> Any insight?
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/dd2c8fa3-784f-4cad-a46b-05b68b0c3abbo%40googlegroups.com.

[google-appengine] [Unimportant] Old Python 2.7 Users: Anyway to shrink the local .rdbms / reduce it's size?

[Kaan Soral]

Hi Old Python 2.7 App Engine users,

I have an unimportant question, maybe one of you figured it out

My db.rdbms on one project is 200mb's, on another 600mb's, they both spin 
up instantly locally, so it doesn't matter too much, but I decided to 
delete unimportant entries, reduce the db size, however it seems to never 
reduce in size (Deleted every entry in 2 large tables, before 215.1MB's, 
after 215.1MB's :/)

Any insight?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/377c22cd-67bc-428b-8648-103a94ea7721o%40googlegroups.com.

Re: [google-appengine] Diving into App Engine is soooooooooooo hard now - My analysis

[Kaan Soral]

As far as I assumed, and I'm too optimistic these days, they'll work 
indefinitely? 
https://cloud.google.com/appengine/docs/standard/long-term-support

I'm probably never going to move my old apps to 3.7, they'll die when 2.7 
dies - they use pretty much every available original in-house app engine 
service - almost impossible to completely re-engineer - It's painful but 
all things die :/

Re-writing with NodeJS / App Engine proved more mentally challenging then I 
assumed too, you get to experience every little thing we had and now lost - 
we were developing locally with instant refreshing at every file update, 
when you move to live development that requires you to do a 2 minute 
deployment to test every change, the frictional difference is huge - I 
guess it might even be a good idea to write custom local emulators for 
everything, routines that'll just return sample values etc. - or maybe 
in-line conditions to fallback to faked responses when you are locally 
developing

Like MdeA I started using Python for only App Engine, and I certainly wish 
services weren't being sunsetted, instead more could've been introduced, 
images service for example, was the most practical solution to image 
serving - when video/video sharing apps became more popular, I was 
anxiously awaiting a "videos" service to be introduced too, but looking at 
things today, we lost them all instead of gaining more :D

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ad6eaedd-aa30-4b81-ba48-64ade286c7ed%40googlegroups.com.

[google-appengine] Re: Diving into App Engine is soooooooooooo hard now - My analysis

[Kaan Soral]

A small update, after my conclusion, I went ahead with my live trial and 
finally started a new journey

~/Desktop/APPENGINE/cloudsdk184/bin/gcloud app deploy -> after every small 
change
~/Desktop/APPENGINE/cloudsdk184/bin/gcloud app logs tail -> replaces the 
local development server - instead of instantly reading your local logs, 
you now trigger something, wait 5-10 seconds, and the "logs tail" pulls the 
live log to your Terminal after a healthy delay

I started with a simple Firestore write, received 
"UnhandledPromiseRejectionWarning: Error: 9 FAILED_PRECONDITION: The Cloud 
Firestore API is not available for Cloud Datastore projects." - then 
upgraded my unused Datastore to Native Firestore and saw my first data 
entry!

Ironically, while the Firestore advertises live capabilities, you need to 
refresh https://console.cloud.google.com/firestore/data manually everytime 
you make a change too (documents auto update, but collections don't)

I'm currently happy and excited, I just wish I came to this obvious 
conclusion earlier - Don't forget to limit your spending as a live 
environment isn't forgiving

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c634ea14-38e4-44cf-b8c9-bf54f3c6fe00%40googlegroups.com.

[google-appengine] Diving into App Engine is soooooooooooo hard now - My analysis

[Kaan Soral]

I'll whine a lot, but there's a concentrated and crisp TL;DR in the end


I've been using App Engine I guess for 10 years now? It's hard for me too, 
but also, I'm preparing to open source my current project, at each step I 
question everything, I was going to open source my game so young 
developers/kids/gamers could dive into networked game development easier 
(It's an MMORPG on Google Cloud, my dream was that open sourcing it could 
enable fast experimentation with the genre), but as it is, at the current 
state of App Engine, the bottleneck is definitely all the diversified and 
separated products, separate emulators (missing ones), separate environment 
variables for everything etc. - I can't imagine a happy scenario where one 
could just develop an App Engine app locally without spending weeks trying 
to understand what's what first - Back in the day, I think the Launcher GUI 
and the simplicity of it all, got us all hooked (This is coming from a CLI 
user)


Things used to be as simple as *"python2.7 /sdk/dev_appserver.py 
--storage_path=/storage/ --blobstore_path=/blobstore/ 
--datastore_path=/storage/db.rdbms --host=0.0.0.0 --port=8080 /path 
--require_indexes"* - We could test *EVERYTHING* locally, now we can almost 
test nothing locally


It seems that the python2.7 that's currently in the process of being 
deprecated was the last of it's kind


Instead of refactoring everything for Python 3, I decided to move onto 
NodeJS instead, the work done, the documentation, the depth of it all, on 
the surface, is very exciting, until you dive in


First of all, at each step, you're faced with initially arbitrary decisions 
that's going to eliminate you, flexible or standard? - does it really 
matter, I don't think so, firestore in datastore mode or firestore? oh my 
god, it's like each option is designed to torture and drive away a 
potential user


As a long time App Engine / Datastore user, I've simplified my data design 
to a singular dimension and simple get/set by key operations wherever 
possible, so for me the limitations of the new datastore really doesn't 
matter much, I just wanted to use the new Firestore and all the new 
exciting stuff that came with it, experiment with it, see how it can 
improve my approach, I've always designed products that regularly pinged 
the server for new data, so the realtime features etc. appealed to me a lot 
- so I'd choose the unlocked Firestore, instead of the irreversibly locked 
one, at least that decision was easy, but the not so easy part was to find 
out how I'm going to emulate it locally, authenticate it locally, use it 
locally


Honestly, it's been months since I've been entertaining idea, whenever I 
can find some free time, I re-dive back into it, but still, I achieved 
nothing, yesterday I finally discovered "gcloud beta emulators firestore 
start" - but the new "environment variable" based approach etc. is making 
me want to cry, so I gave up again (the examples/docs are from "firebase", 
the authentication/flow is from "firebase", there are no simple examples to 
just use and authenticate "firestore" locally) - When I'm configuring a 
product, I either want to configure it in-code, or configure it in .yaml 
files, with the new approach, even app id's are set through environment 
variable's ...


Just before writing this, I wondered if I'm being unjust, and wondered 
whether I should just pick an arbitrary service, experiment with it using 
NodeJS, so I picked Taskqueues, or "Cloud Tasks" as they are now called, 
once again, it's a seemingly well documented and seemingly a well designed 
product, but no matter what I did, I just couldn't find how I'm going to 
test it locally, as it turns out, it can't be tested/emulated locally? 
bummer. (Related: 
https://stackoverflow.com/questions/53826183/local-development-with-cloud-tasks-cloud-datastore-with-gae-with-python3
)


*TL;DR/Conclusion:*


I believe we need a practical, all in one and turnkey local emulation 
solution again - There's a vast/overflowing/overwhelming amount of new 
options and no easy way to test them - I think the best way to learn about 
something is to use it


*Suggestion/Takeaway:*


I think local development is no longer feasible or possible, so it seems 
like a better idea to just deploy and test live, I don't know whether it's 
the intended way now, but it's seemingly the only way. In the past we 
assumingly all had a local development version of our products, I assume we 
now should start with live development versions of our products. So when 
I'm ready to try again, I'll experiment with it all live. (It's sad that 
likely this option is only possible for paying customers, I don't know 
whether the products have safe free tiers now, I assume not)


I also wonder whether I'm the only one who feels this way, are there any 
success stories out there, happy new App Engine users?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App 

[google-appengine] Re: Python 3 standard environment data storage

[Kaan Soral]

I see a lot of questions of this nature lately, good that we're slowly 
starting to migrate/consider migrating as a community

A bit hijacking the thread; Do we really need to migrate old apps? I have a 
lot of apps that I keep alive just to keep them alive, they are keepsakes 
at this point, I don't want to kill them, their income roughly offsets 
their costs, so for me, the question is keep or kill, and I'd like to keep

Is there any point where old technologies / Python 2.7 is going to become 
unsupported?

(This Python 2.7 thing is such a bummer too, I certainly wish we could have 
a Google Python 2.7 fork and keep using it indefinitely :)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/da5bc76b-c9e9-484d-b32f-4e4a0e9162ec%40googlegroups.com.

Re: [google-appengine] Controlling costs

[Kaan Soral]

No, don't report it just yet, also I don't agree with Vitaly's advice of 
always keeping an idle instance - in addition to your setting, if you set a 
"min_pending_latency" and DON'T set a "max_pending_latency" - you'll get 
the minimal costs, "max_pending_latency" is like saying "Oh my god there's 
q request spin up an idle instance fast!" while with a 
"min_pending_latency" - it might make someone wait a bit with a low %, 
people wait a bit anyway, since consequent requests are fast, they won't 
notice it anyway

Now, actually what happens, find a graph that clearly shows billed 
instances, there being idle instances doesn't mean that you are billed for 
them! they are free

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/8b39d7a8-e538-4c24-b6f6-65423d6a18b2%40googlegroups.com.

[google-appengine] Re: Upgrade standard app engine to Python 3.7?

[Kaan Soral]

App Engine without it's services is an archaic and expensive instance 
scaling service in an age of cloud innovations, totally beats it's core 
purpose in my opinion, it's not an "App Engine" any more

I wonder if it makes more sense to migrate to something like Cloud 
Functions, from the docs, they didn't build it for websites - I'll 
experiment at one point, these docs seems promising: 
https://firebase.google.com/docs/functions/http-events

I guess what I mourn most is my lost experience with App Engine, I had all 
the gimmicks sorted out, but I guess it should be fun reinventing the 
wheel, playing with new technologies

Firebase / Firebase DB / it's realtime capabilities, in theory, seemed 
super fun to me, but having experience with App Engine, what matters is how 
well you know the service and how it performs in high throughput scenarios, 
that's why I won't easily abandon App Engine, as I exactly know how it 
performs, where the limits lie, how to work around them etc.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3f5f28ee-783d-48f5-b8e6-d2179feacc67%40googlegroups.com.

[google-appengine] Re: Controlling costs

[Kaan Soral]

Try removing: max_instances: 3
You can set: max_idle_instances: 0
Then: min_pending_latency: 200ms
Or even more
This way, instances will only fire up when they have to

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/6ec59e5a-907e-4812-ab94-2f8585b9b039%40googlegroups.com.

[google-appengine] Re: The end of PaaS ?

[Kaan Soral]

At this point linking the public issue tracker in discussions like this 
always adds insult to injury, for me, as they waste time and personally I 
haven't seen a solution yet

Example: https://issuetracker.google.com/issues/112668010 (I also believe I 
dug deeper into the issue after reporting it, there was an email exchange, 
that's when the issue got reopened but never got anywhere) - The first 
reply was very frank, the error in this case probably came from the load 
balancer or another network component, he says it won't be propagated - 99% 
of requests to an app could fail on another layer, we wouldn't suspect a 
thing - doesn't make much sense to me - this is the disconnect I'm talking 
about

On Wednesday, October 23, 2019 at 4:51:56 PM UTC+3, George (Cloud Platform 
Support) wrote:
>
> Hello Kaan, 
>
> The problems you describe here are most suitable for feature requests in 
> Public 
> Issue Tracker . Developers may assess 
> merit in each case, and decide on proper implementation. 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3bccc403-0c11-44b2-a516-f890a06d5fc2%40googlegroups.com.

[google-appengine] Re: The end of PaaS ?

[Kaan Soral]

As a long time App Engine user and still being a believer, I can't help but 
feel similar to Patrice's original feelings and confusion

One related issue: 
https://groups.google.com/forum/#!topic/appengine-ndb-discuss/xOuhDFiYKbU - 
On this migration path, we were being instructed to use a 3rd party Redis 
service

When mail was deprecated, again we were instructed to use 3rd party mail 
services

As most App Engine users, I used App Engine as I wanted an all in one 
service, where I can just "import" whatever I wanted, and use them, without 
worrying about a "perfect integration" - I wanted most edge case scenarios, 
retry scenarios etc. to be handled by App Engine

so TL;DR of my points:
[1] (Major) Everything should be in-house, which includes Google Cloud
[2] (Important but Minor I guess) Even if we use other Google Cloud 
services, their integration should be native or almost native

If I needed to give one example to [2] - I'd say for the storage, the 
buckets that App Engine references could be auto created, so a new user 
wouldn't have to deal with all the permissions, setup etc. - that's usually 
the tricky part of a cloud service, they could just take an example code, 
run it, and be "Voila!"

As [3] - Another recent gripe I have is the perceived lack of grip over App 
Engine, If I needed to give one concrete example, I recently discovered 
that end users could see 500 errors, yet you would never observe these 500 
errors in App Engine logs, it took a considerable amount of effort to track 
these, prove they existed, after back and forth, turned out they were 500 
errors from the network layer, and they "just didn't propagate" - so after 
all that effort, there was no solution, they still don't propagate, I feel 
like it's a major breach of trust on all fronts, but no one cares ... Back 
in the day, the engineers cared, we used to have Google Hangout's, they 
listened, they responded, now I also feel like App Engine is just becoming 
an old instance scaling service

With all these said, I still love App Engine, and on day to day usage, no 
issues lately

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/58655f49-89d4-4253-b618-e6855ce8f308%40googlegroups.com.

[google-appengine] Re: Is Google App Engine Launcher being deprecated? Why??

[Kaan Soral]

Turns out it was #googledown - I managed to get a token after 15 minutes, 
but didn't work, all in all, resulted in a 45 minutes down time, my luck I 
guess, as it was a month-long update

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0f38500f-228e-40f5-b2ad-73dadc41ca18%40googlegroups.com.

[google-appengine] Re: Is Google App Engine Launcher being deprecated? Why??

[Kaan Soral]

After 15 minutes, it's working now, but this certainly was a scary 
pre-warning, best to find the simple/clean appcfg alternative of the new 
cryptic/massive gcloud

On Monday, August 19, 2019 at 9:41:15 PM UTC+3, Kaan Soral wrote:
>
> Right now appcfg isn't working for me, I was right in the middle of a 
> multi-platform update, now my product (game) is defunctional for an unknown 
> time
>
> I had no idea appcfg was going to stop working, I'm shocked :(
>
> On Friday, July 26, 2019 at 6:10:41 PM UTC+3, Harmit Rishi (Cloud Platform 
> Support) wrote:
>>
>> I was able to find the official notice for the deprecation of *appcfg*. 
>> Looks 
>> like it is set to be deprecated on *July 30, 2019* while the *shutdown 
>> date *is set for *July 30,2020*. You may find further details about this 
>> here <https://cloud.google.com/appengine/docs/deprecations/>. 
>>
>> As for the launcher, we recommend moving to the Cloud SDK for future 
>> developments. You may install it here <https://cloud.google.com/sdk/>.
>>
>> Hope this helps!
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c84b8770-b755-4f48-80f4-d63d85445ec3%40googlegroups.com.

[google-appengine] Re: Is Google App Engine Launcher being deprecated? Why??

[Kaan Soral]

Right now appcfg isn't working for me, I was right in the middle of a 
multi-platform update, now my product (game) is defunctional for an unknown 
time

I had no idea appcfg was going to stop working, I'm shocked :(

On Friday, July 26, 2019 at 6:10:41 PM UTC+3, Harmit Rishi (Cloud Platform 
Support) wrote:
>
> I was able to find the official notice for the deprecation of *appcfg*. Looks 
> like it is set to be deprecated on *July 30, 2019* while the *shutdown 
> date *is set for *July 30,2020*. You may find further details about this 
> here . 
>
> As for the launcher, we recommend moving to the Cloud SDK for future 
> developments. You may install it here .
>
> Hope this helps!
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/8918ca19-94fc-45e3-b40f-3a2fc0310b21%40googlegroups.com.

[google-appengine] Re: ConnectionClosedError: Connection closed unexpectedly by server at URL - Chronic Error?

[Kaan Soral]

URL Fetch

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0ce4b4e7-feb7-41f6-8463-32e11877b46b%40googlegroups.com.

[google-appengine] ConnectionClosedError: Connection closed unexpectedly by server at URL - Chronic Error?

[Kaan Soral]

ConnectionClosedError: Connection closed unexpectedly by server at URL

I do receive these errors so frequently, to my own node servers that are on 
GCE servers, they are pretty relaxed servers so no cause why they would 
just disconnect abruptly

Also to other external API services on a daily basis

I suspect it could be an issue with the URLFetch health instead?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/52a15990-4e0a-4a74-af09-6f1fa7034d52%40googlegroups.com.

[google-appengine] Re: Network issues yesterday, left players with mis-cached files

[Kaan Soral]

Again, like most errors and issues these days, this issue was unobservable 
from the Logs too

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/950af7cd-0547-4990-85c3-a8f5f88817f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Network issues yesterday, left players with mis-cached files

[Kaan Soral]

There are Blobstore and Version Deployment issues reported 
from: https://groups.google.com/forum/#!forum/google-appengine-downtime-notify

However, some of the users of my app reported mis-served then cached static 
files, it broke my game for them

Proof attached - the error was persistent and long-standing for this 
url: http://adventure.land/js/pixi/fake/pixi.min.js?v=427

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/97c5d432-b848-4bd7-b310-96fa364c26b8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Getting inconsistent 500 error - inconsistent Error code 204 on Google App Engine PHP Standard

[Kaan Soral]

I investigated issues like this one a lot, tried to get them 
fixed: https://groups.google.com/forum/#!topic/google-appengine/mkDuThzy-3g

But no luck, they just get ignored, *solution*: use another product

Like you mentioned, most of these issues don't even hit the logs, they are 
hidden from you, and that's the biggest issue

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0ba23b7c-57cc-4389-af81-43692d4f5bd4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Anyone who uses App Engine are technically being scammed - Why? No way to assess actual error rates

[Kaan Soral]

Just happened again, a mass outage spanning more than an hour, everyone 
affected on my game, *nothing got logged in App Engine logs, I only 
captured it as I retain client errors*, characters died, got jammed, 
basically the issue just tore everything - I coded everything to be 
self-healing at this point, but when the issue happens, it causes a mass 
reporting spree, rightfully so

So I will file an SLA claim at this point, but I have a hunch I'll have to 
report to report the issue 3-4 times again for just a measle SLA 
reimbursement

This issue needs to be fixed

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e39c4020-de88-4d45-b784-b90340d748b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Anyone who uses App Engine are technically being scammed - Why? No way to assess actual error rates

[Kaan Soral]

First of all, let me state that I love App Engine, I've been using App 
Engine since the very early days, and what hurts me the most if the state 
of dismay that I feel the product is in, and getting the feeling that those 
working on the product, not loving and caring the product, as the early 
engineers did

So, the issue, I originally reported 
at: https://groups.google.com/forum/#!topic/google-appengine/95VB910CdzI

*TL;DR:* Errors on an internal network layer can happen, yet these issues 
are not logged in our Console's, they are logged in the network layer, we 
can't see them, Google can see them, yet, even when they could, they don't 
propagate these issues to our App Engine logs, even though I reported and 
requested it months ago, and, in my opinion, it's a critical issue and a *major 
breach of trust*

Why the harsh language, and the accusation of `*being scammed*` - Well, 
there's no way to assess actual error rates, see the errors, maybe the 
error rates are 0.5%, way above SLA guarantees, but as App Engine using 
developers, there's no way for us to know, as errors are being hidden, 
rather than being reported - so in order to actually catch these errors, 
you need to implement your own logging for various different usage scenarios

Technically, 2 problems:
*1)* Request is made, it never hits App Engine, it never gets logged, 
client sees error, but as the developer, you never see it
*2) *Request is made, it hits App Engine, the result never hits the 
requester, the requester sees 500, while the actual operation is a 200

Depending on your architecture, scenario (1) is better, if you are doing 
critical operations, (2) could require manual intervention

I'm developing http://adventure.land - an mmorpg that uses App Engine for 
backend, while it's a game, to prevent item duplication and to ensure data 
singularity, the operations need to be precise, at this point, it's pretty 
much like a bank infrastructure, If you check our 
Discord: https://discord.gg/44yUVeU and search for "App Engine" - you can 
see how torturous this issue is - my entire architecture now assumes a 
backend that can fail at any time for long durations (which is kind of 
nice, but, scenario (2) is out of the scope of this system, it requires an 
entirely different approach, which I'm not sure how to approach, yet)

Holiday season is a very critical time, I'm way behind schedule, I want to 
release my game, yet, once again, an App Engine issue ruined my time and 
wasted the day

Please fix this issue, report the errors, and try to prevent scenario (2) 
as much as possible

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4aab5e66-a51f-46f6-b60f-d19565d9c17f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Extremely dissatisfied with App Engine lately, Network Issues, Support doesn't help and loops

[Kaan Soral]

As a follow up, I've logged around 90 502/503 errors around `date: Thu, 11 
Oct 2018 23:15:27 GMT` - Caused by multiple ~minutely outages, I personally 
experienced one of them, and checked my logs/noticed the issue, it's been 
happening a lot, but it's so much that I don't even bother to check any more

*As usual, If you look at the App Engine error logs, you can't see any of 
these errors or any signs*

At this point it's no different than a fraud, I've reported this issue more 
than 2 months ago, it should've been solved by now - I'm not saying errors 
can't or shouldn't happen, but they should be transparent, hiding errors 
and outages like this is a deception (my suggestion, add a new log level, 
something like a "Network" level error, and log these in that section, 
otherwise, these errors will flood everyone's logs)

I don't even bother making SLA requests any more, the minuscule amounts of 
compensations doesn't even make up for the time hunting these issues - and 
things don't get better, only worse

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/43da91c3-5f78-472f-80e5-23c86d7c2e98%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Any plans to introduce Spanner natively in App Engine?

[Kaan Soral]

I guess Spanner improves over Datastore in terms of it's eventual 
consistency, but there's probably no strong reason to jump ship from 
Datastore

Still I wonder if there are any such plans ahead

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b77ea212-4b0c-46ca-953b-9d9b97f0e0e5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Extremely dissatisfied with App Engine lately, Network Issues, Support doesn't help and loops

[Kaan Soral]

An update

1) John Lowry adopted my issue and debugged it with me, he was very 
transparent and helpful and genuinely interested in helping

2) I improved my system, logged both Cloudflare wrapped and raw appspot.com 
500 errors, verified that these 500 issues do happen, and they are 
invisible to the logs (it's not a cloudflare issue, easier to observe with 
appspot.com)

*Result: If you are using App Engine, you might be losing traffic, losing 
customers, and as far as you and your logs are concerned, these issues are 
invisible - sadly, both from responses to issue, and my issue tracker 
threads (one example: https://issuetracker.google.com/issues/112668010 - 
there are other major/minor issues reported by me, none of them are solved, 
usually the responses are aimed to stonewall issues ) - None of these 
issues are being solved, no one is interested in improving anything it 
seems*

This specific issue is in the network layer, so basically, the problem is 
that, it's not being propagated to App Engine logs, it's a major breach of 
trust in my opinion, I'm not complaining about errors, errors do happen, 
apps should adapt, but the problem is, if you can't observe errors, you 
can't adapt, I'm only able to log these issues by catching them (appengine 
is down when you catch them) - keep them for 40 seconds, and report them 
after 40 seconds, at which time, assumably the network is back up, so maybe 
my logging logic is failing, maybe the network isn't back up after 40 
seconds, and I'm missing some of them too, I can't know, I likely need a 
3rd party logging solution

The most saddening aspect is, I again invested a lot of my time and energy 
in debugging the issue, helping find the cause, trying my best, yet, has 
anything changed? Nope.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/69d82e34-161a-476a-9bdc-c6ac07dc729d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Extremely dissatisfied with App Engine lately, Network Issues, Support doesn't help and loops

[Kaan Soral]

For almost 10** years

On Monday, August 13, 2018 at 4:44:02 PM UTC+3, Kaan Soral wrote:
>
> Yes, but the main problem is requests not reaching the logs, or appengine
>
> By the way, the new support strategy is now loop+exhaust
>
> Whenever I open a support ticket, I'm being looped to another place, in my 
> last attempt, I've been told this issue is not covered by SLA's: 
> https://groups.google.com/forum/#!topic/google-appengine-downtime-notify/wiXNbETOYgA
>  (full 
> downtime, and not covered, bizarre) - and I need to open a ticket on the 
> cloud platform sla page
>
> Anyway, I'm exhausted
>
> This is coming from someone who has been using App Engine *for almost a 
> year**, I've been using App Engine in simpler and simpler ways over the 
> times, as anything complex, and you hit issues that were hard to debug in 
> the past, and with the current support levels, almost impossible, for my 
> next project, I'll try not using App Engine at all, I don't know whether I 
> can dump my experience/knowledge of the platform, but I'll certainly try, 
> it's very saddening :(
>
> I really wish we could regain the dynamism of the older days, where we got 
> actual helped and found solutions to the problems, rather than passing the 
> day and ignoring issues
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e040b458-09e8-44fc-8964-d3eaf01f487d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Extremely dissatisfied with App Engine lately, Network Issues, Support doesn't help and loops

[Kaan Soral]

Yes, but the main problem is requests not reaching the logs, or appengine

By the way, the new support strategy is now loop+exhaust

Whenever I open a support ticket, I'm being looped to another place, in my 
last attempt, I've been told this issue is not covered by SLA's: 
https://groups.google.com/forum/#!topic/google-appengine-downtime-notify/wiXNbETOYgA
 (full 
downtime, and not covered, bizarre) - and I need to open a ticket on the 
cloud platform sla page

Anyway, I'm exhausted

This is coming from someone who has been using App Engine for almost a 
year, I've been using App Engine in simpler and simpler ways over the 
times, as anything complex, and you hit issues that were hard to debug in 
the past, and with the current support levels, almost impossible, for my 
next project, I'll try not using App Engine at all, I don't know whether I 
can dump my experience/knowledge of the platform, but I'll certainly try, 
it's very saddening :(

I really wish we could regain the dynamism of the older days, where we got 
actual helped and found solutions to the problems, rather than passing the 
day and ignoring issues

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/a1a63d46-afa1-4f7c-9573-8e7434df11dc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Extremely dissatisfied with App Engine lately, Network Issues, Support doesn't help and loops

[Kaan Soral]

Basically, I experience network issues like what happened here: 
https://groups.google.com/forum/#!topic/google-appengine-downtime-notify/wiXNbETOYgA

On a regular basis, since if a request fails to reach App Engine, it's 
invisible to the logs etc. - It's impossible to detect or debug these 
issues either, I only learn about them from players/users

They are usually regional too, while one player experiences the issue for a 
long duration, let's say 1-2 hours, the others usually have no issues

When I open a support ticket, I'm denied any assistance as I'm on a Bronze 
level and refuse to purchase a package, and simply get directed to the 
Financial/SLA ticket url, and when I open an Financial/SLA Credit ticket, 
my claim is again denied as I need to have a Silver/Gold support package

About the issue itself, I believe it's impossible to debug, for example, 
Cloudflare marks each request with a unique ray ID, so if a network request 
fails, it's possible to get the issue debugged from it's ray ID, but since 
App Engine has no system, what you have is just reports of 500 (Internal 
Server Error)'s that don't show up on any logs

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ec00a8f8-1367-40da-8bfa-19c9f308365b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Stackdriver: Thoughts?

[Kaan Soral]

Indeed, it seems they corrected them now, I checked originally, but thought 
maybe the reported metrics could be reduced if there are free quotas I 
didn't know about

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/51fef04c-d09e-49b6-aa28-a4c30dcf6d57%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Pull Queues: They can't really pull - Horizontal scaling is a must

[Kaan Soral]

About a month ago, I tested Pull Queues for a term project, attached my 
test files and the resulting paper (keep in mind that, since I allocate a 
short time for academic stuff, the quality is low :)

I chose Pull Queues, as they let me down hard in the past, I was hoping to 
be surprised this time, but I sadly wasn't

As a TL;DR, the results are:

1) Pull Queues are built to scale with push operations (add operations 
don't fail), however, each *single* task, takes 4 to 120ms+ to create (If 
you add 100 tasks in a batch, 400ms to 12000ms or more)
2) Pull Queues crumble with pull operations, per the documents, you can do 
10 lease operations per second (each with 1000 tasks, assumably), 
theoretically leasing 10.000 tasks / second, you can't, you can't even 
lease 1 task per millisecond on average, 600 tasks/second is a more 
realistic guess, the performance is unreliable, dwindles further when you 
add tasks in parallel too, it's clear the internal black box is built to 
handle add operations, but not lease operations
3) [Side Info, Suggestion] There are Interface bugs, running push queue 
tasks appear as not running, leased pull queue tasks appear as not leased, 
it's very easy to re-run an already running push queue task, it should be 
impossible to re-run an already running task

My personal result: Don't use Pull Queues, or use them by creating multiple 
queues (called horizontal scaling I assume, not very big on terms myself), 
and dividing data into queues, assumably by the modulus of your tasks' tag 
(In these tests, I haven't even included tags, I assume it will only make 
things worst, in the past, I tried using pull queues with tags, and lost 
more than 2 months of my life)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/eaba2258-8e5f-4b29-8f2d-c3b48cb64a65%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
<>

Re: [google-appengine] Re: zigzag merge

[Kaan Soral]

It would be indeed nice, if the production system kept metrics for index 
usage, that way, it would be obvious which indexes are needed or not

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3931eabd-023b-4650-a5d6-f8473ae26ffd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: zigzag merge

[Kaan Soral]

I might be misunderstanding the question, and interpreting a complex 
question as a simple question (By "needing an index", if you mean, need an 
index to increase lookup performance, I do misunderstand the question)

But, to assess whether you need an index or not, run the SDK with the 
"require indexes"+"don't auto add indexes" (forgot the actual arguments) 
modes, then run the operations of your app in development, delete indexes 
one by one, if operations fail with an index error, re-add that index

It's an exhaustive process, but in any case, you need to re-understand what 
your app does now, so instead of trial error, just studying the app and 
eliminating the indexes manually could require the same effort

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/87da9789-74d4-42a1-8107-00a87c612bb1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Stackdriver: Thoughts?

[Kaan Soral]

I personally use App Engine as It allows (or rather, allowed) me to 
concentrate on product (in theory), rather than itsy bitsy stuff

Up to now I ignored the Stackdriver emails, and only now checked what 
actually it was telling (Ignored because I'm here for the PaaS, If I was 
going to use multiple services individually, I'd have invested my efforts 
in AWS back in the day)

Anyway, long story short, my 3 main projects seem to be digesting 230, 460, 
and 1300GB's monthly, as I understand, I'll start paying $0.5 per GB now, 
which makes $995 monthly, considering I paid ~$600/mo for everything up to 
now, bumping that up to $1600 seems extreme, I might be doing the 
calculation wrong, or I might be logging too much stuff

Anyway, my thoughts are negative (Appreciate the very very very early 
warnings, and the effort, yet inspecting this last email, investing 15 
minutes into understanding things, It's not enough to grasp the change in 
pricing, as an example, the link in the email that's supposed to bring you 
to "Billable log volume *across a billing account* available" - directs you 
to main billing page - Just a summarised, hey, here's what you paid, and 
here's what you'll pay now email would be the best)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/2b08c9e3-e344-45b3-abad-c1ffc427e220%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Set up Google Cloud SDK or App Engine SDK or both for development?

[Kaan Soral]

By serene, I mean development should be fun and simple

You should probably start 
here: 
https://cloud.google.com/appengine/docs/standard/php/tools/using-local-server

Finding dev_appserver.py is going to be a bit challenging tho, If you were 
on Linux, this would probably be the way to 
go: https://cloud.google.com/appengine/docs/standard/php/download#php_linux

On Saturday, May 19, 2018 at 1:16:29 AM UTC+3, Richard Bernstein wrote:
>
> I am glad you added this comment. I am not sure what you mean by serene 
> manner? What I think you just said (and tell me if I misunderstood) is 
> develop with your favorite sdk on your local pc, and when you are done use 
> command line tools to upload it and set it up. In my case my code is php 
> under Windows so I should find and use App Engine PHP SDK for Windows to 
> upload and configure. Did I get this right? And not bother with phpstorm 
> for app engine?
>
>
> On May 18, 2018, at 17:54, Kaan Soral <kaan...@gmail.com > 
> wrote:
>
>
> Not really on topic, but as some sane and simplifying advice - It's best 
> to move away from IDE's like Eclipse
>
> Their purpose is to simplify development, but from your experiences too, 
> it's clear that they don't deliver on their promises, and complicate things 
> instead
>
> I'd suggest you to write the code with whatever IDE you prefer, then use 
> the command line tools separately in a serene manner
>
> My personal choice is the App Engine Python SDK for Linux, even though I'm 
> on macOS (it includes pure code, no installer, just pure python code, 
> granted things are usually more complicated for Java, but the same 
> experience might apply) - You might give it a try, and if it doesn't work, 
> you can always go back to Eclipse
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to google-appengi...@googlegroups.com .
> To post to this group, send email to google-a...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/google-appengine/c23a7055-953c-49bc-92cc-077264ca2e00%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/google-appengine/c23a7055-953c-49bc-92cc-077264ca2e00%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/8024c137-e885-486d-ba0d-6595afc7aea7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Set up Google Cloud SDK or App Engine SDK or both for development?

[Kaan Soral]

Not really on topic, but as some sane and simplifying advice - It's best to 
move away from IDE's like Eclipse

Their purpose is to simplify development, but from your experiences too, 
it's clear that they don't deliver on their promises, and complicate things 
instead

I'd suggest you to write the code with whatever IDE you prefer, then use 
the command line tools separately in a serene manner

My personal choice is the App Engine Python SDK for Linux, even though I'm 
on macOS (it includes pure code, no installer, just pure python code, 
granted things are usually more complicated for Java, but the same 
experience might apply) - You might give it a try, and if it doesn't work, 
you can always go back to Eclipse

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c23a7055-953c-49bc-92cc-077264ca2e00%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Weird New Delayed Deployments

[Kaan Soral]

Just started experiencing weird deployments, wondering whether it's a thing

Previously I never got 404's during a deployment, but with this issue, got 
404's during a deployment, which is concerning, previously, requests would 
be answered by the earlier code

And the delays:

06:17 PM Checking if deployment succeeded.

06:17 PM Will check again in 1 seconds.

06:17 PM Checking if deployment succeeded.

06:17 PM Will check again in 2 seconds.

06:17 PM Checking if deployment succeeded.

06:17 PM Will check again in 4 seconds.

06:17 PM Checking if deployment succeeded.

06:17 PM Will check again in 8 seconds.

06:17 PM Checking if deployment succeeded.

06:18 PM Will check again in 16 seconds.

06:18 PM Checking if deployment succeeded.

06:18 PM Will check again in 32 seconds.

06:18 PM Checking if deployment succeeded.

06:18 PM Will check again in 60 seconds.

06:19 PM Checking if deployment succeeded.

06:19 PM Deployment successful.

06:19 PM Checking if updated app version is serving.

06:19 PM Will check again in 5 seconds.

06:19 PM Checking if updated app version is serving.

06:19 PM Will check again in 5 seconds.

06:20 PM Checking if updated app version is serving.

06:20 PM Will check again in 5 seconds.

06:20 PM Checking if updated app version is serving.

06:20 PM Will check again in 5 seconds.

06:20 PM Checking if updated app version is serving.

06:20 PM Will check again in 5 seconds.

06:20 PM Checking if updated app version is serving.

06:20 PM Will check again in 5 seconds.

06:20 PM Checking if updated app version is serving.

06:20 PM Will check again in 5 seconds.

06:20 PM Checking if updated app version is serving.

06:20 PM Will check again in 5 seconds.

06:20 PM Checking if updated app version is serving.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/10c8ff11-9f17-4acd-8ceb-fc62c967792e%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Realtime: Firebase or Manual Node.JS Socket Servers?

[Kaan Soral]

After dealing with Firebase for 2 days, my conclusion is that the grass is 
greener on the App Engine side

We have our well defined app.yaml's, perfect CLI's, full local emulation

I don't think the "Firebase" command emulates https://databaseName
.firebaseio.com locally

Basically, I'm trying to just use Firebase for the realtime App Engine to 
Client communications, instead of polling, App Engine is going to update 
the Firebase Realtime Database instead, however, if the Firebase Realtime 
Database isn't emulated locally, can't develop anything practically

Not really the place for this discussion, but a good opportunity to 
appreciate our App Engine workflow

On Monday, October 16, 2017 at 9:33:43 PM UTC+3, George (Cloud Platform 
Support) wrote:
>
> Hello Kaan, 
>
> Real-time gaming is a possible area of efficiency for Firebase, so 1) and 
> 2) You can use Firebase. 3) You can "Run Functions Locally", as described 
> on the similarly named page 
> . 
>
> For more insight, you can read an excellent example on the "Using Firebase 
> for Real-time Events on App Engine" documentation page 
> 
> . 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/24d1c11c-7c9b-4497-b20b-34b54a7097d0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Realtime: Firebase or Manual Node.JS Socket Servers?

[Kaan Soral]

To iterate a bit on my question, It seems I'm intending to use "Firebase 
Realtime Database", which is a different thing than the new Firestore, 
since Firestore is similar to Datastore, I initially had the impression 
that it was Firebase merged into App Engine, but it's not, it's the NoSQL 
version of the Realtime one

The second link I shared is for the "Cloud Functions", so my remaining 
questions are:

A) Is there an SLA / Guaranteed reliability for Realtime Database of 
Firebase? (As far as I read, it delivers on it's promise tho)
B) The local testing and local setup aspect, I couldn't figure it out just 
yet

On Sunday, October 15, 2017 at 1:46:57 PM UTC+3, Kaan Soral wrote:
>
> For realtime stuff, I normally use Node.JS servers, and persist/back-up 
> data to App Engine, however, the scaling, spinning up the servers are 
> always manual for me, it's also not easy dealing with server exceptions 
> when they happen
>
> I'm wondering whether using Firebase is feasible for basic multiplayer 
> games, for example chess, where 200-300ms of delays don't affect gameplay 
> (but 1-2 seconds of delays are still not acceptable)
>
> Reading this: 
> https://cloud.google.com/solutions/using-firebase-real-time-events-app-engine 
> - Gave me hope
>
> But after that, I dived into this: 
> https://github.com/firebase/firebase-functions/issues/4 - Which made 
> things a bit confusing, as local testing seems possible, but not trivial
>
> This further complicated things: 
> https://firebase.google.com/docs/firestore/rtdb-vs-firestore
>
> Now the questions in my mind are:
>
> 1) Can I use Firebase?
> 2) Firebase or Cloud Firestore?
> 3) Can I test and develop for these locally?
>
> Not exactly an App Engine question, but at heart, it's an App Engine 
> question, as I want things to be as seamless as possible and go along with 
> App Engine as best as possible
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7d2cd4a2-bb74-47ef-815f-80f1e63a0c0b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Realtime: Firebase or Manual Node.JS Socket Servers?

[Kaan Soral]

For realtime stuff, I normally use Node.JS servers, and persist/back-up 
data to App Engine, however, the scaling, spinning up the servers are 
always manual for me, it's also not easy dealing with server exceptions 
when they happen

I'm wondering whether using Firebase is feasible for basic multiplayer 
games, for example chess, where 200-300ms of delays don't affect gameplay 
(but 1-2 seconds of delays are still not acceptable)

Reading 
this: 
https://cloud.google.com/solutions/using-firebase-real-time-events-app-engine 
- Gave me hope

But after that, I dived into 
this: https://github.com/firebase/firebase-functions/issues/4 - Which made 
things a bit confusing, as local testing seems possible, but not trivial

This further complicated 
things: https://firebase.google.com/docs/firestore/rtdb-vs-firestore

Now the questions in my mind are:

1) Can I use Firebase?
2) Firebase or Cloud Firestore?
3) Can I test and develop for these locally?

Not exactly an App Engine question, but at heart, it's an App Engine 
question, as I want things to be as seamless as possible and go along with 
App Engine as best as possible

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/add6c1cf-7735-4dde-84d5-c2098405c32b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] On Google Platform: What's allowed, what's not allowed, (can)/(what happens if) an active app thrown off?

[Kaan Soral]

On very very early discussions, I remember someone mentioning even x-rated 
apps being allowed, at that time, I remember checking the terms of service 
and seeing nothing against them, I wasn't interested in building an x-rated 
app, yet, it was still nice to see we were on a relaxed platform

I want to build a fun new app, yet, I'm worried it might steer into 
unexplored territories, and I wonder what happens if it gets heated, *can 
Google cease the operations of the app without notice and void the data for 
example, as an extreme scenario?*

The app will be a bit similar to the "Ripple" currency, normally, 
cryptocurrencies are liked as they are decentralised, Ripple is actually 
centralised, therefore it's fast, my project can be decentralised too, but 
I want to center it to App Engine to reduce complexities and increase the 
speed and reliability

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e05968b2-5801-4ba2-8e41-bbbeace51710%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: How to create multple service in one app engine - can any one help

[Kaan Soral]

You have to create a new .yaml, let's call it *mymodule.yaml*

application: yourapplication
module: mymodule
version: 1
runtime: python27
threadsafe: true
api_version: 1
builtins:
- deferred: on
inbound_services:
- warmup

#instance_class: F2
#automatic_scaling:
#  min_idle_instances: 1
#  max_idle_instances: 1
#  min_pending_latency: 600ms
#  max_pending_latency: 2000ms #previously just the max_ one at 900ms 
[31/05/15]

instance_class: F2 #previously the above version [21/05/17]
automatic_scaling:
  min_idle_instances: 0
  max_idle_instances: 1
  min_pending_latency: 15000ms #max
  max_pending_latency: automatic

Then, to deploy, you need to use:

*python2.7 ~/appengine-deployer/appcfg.py --no_cookies 
--noauth_local_webserver --email=your_email update app.yaml mymodule.yaml;*

Copy/pasted these from my routine, the GUI way might be different, this is 
the CLI way

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ddf1b271-5fa0-47d3-981b-2556163b70fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Mail API Quotas

[Kaan Soral]

I don't know whether I missed it, but we never learned why "exactly" Mail 
was deprecated

I wonder whether it was from a conflict of interest too, for example, it 
bothered me that my emails were marked as "Promotional", while they were 
actually "Social", in Gmail

Anyway, that's why I also added the part about "investing some effort", as 
you pointed out, there are things that can be improved

Fyi, I would love a metric based quota too, similar to AWS's solution to 
the issue, could even be more aggressive

On Friday, March 31, 2017 at 5:38:16 PM UTC+3, Jeff Schnitzer wrote:
>
> How do you know that 80% of your emails sent with Google weren’t wasted? 
> GAE never provided deliverability statistics of any sort. 
>
> On Fri, Mar 31, 2017 at 7:35 AM, Kaan Soral <kaan...@gmail.com 
> > wrote:
>
>> I'm with Luca here
>>
>> Using external services introduces new points of failures and introduces 
>> new, edge complications that only arise when you get into things
>>
>> For example, on 10/18/16 I replied to this issue, afterwards, after the 
>> mourning period, I ended up using AWS, the email routine I used for GAE was 
>> too fast for AWS, I completely forgot about their throughput limitations, 
>> 80% of my emails were wasted, a bad start to the Halloween period
>>
>> On this thread, we end up repeating our views of the issue, however, to 
>> repeat my own view, I want GAE to be as independent as possible, instead of 
>> spawning, supporting other cloud services, why not provide all basic 
>> services in-house?
>>
>> For example, a Video/Live-Streaming solution from Google would be pretty 
>> good, I like Zencoder, yet integration was a challenge, it's tooo expensive 
>> for a startup
>>
>> Anyway, my point is, in-house is always better than external, survey us, 
>> ask us what we need, invest some effort into building new things, I'm sure 
>> Email will be on top of that list (I hope Video/Streaming is a runner-up, 
>> in the age that we live in, we really need affordable and agile video 
>> solutions too)
>>
>>
>> On Friday, March 31, 2017 at 2:24:04 PM UTC+3, Bookingforce CTO wrote:
>>>
>>> Hello there,
>>>  
>>>   being CTO of a company that is betting on Google Infrastructure when I 
>>> found out about the limitation to 100 recipients, at first I thought "ah, I 
>>> have to increase that quota", then, after asking (twice) to support team, I 
>>> felt like "what's wrong with GAE ?" 
>>>
>>> The only fact that more than 100 emails is referred as "high volume" is 
>>> almost disturbing... we are using GAE to build cloud applications that 
>>> scale up using multiple instances but still we can't contact our customers 
>>> via email if they are more than 100 ? If you want to prevent spam, bill 
>>> each email, we may be willing to pay as much as we pay for other services. 
>>>
>>> If you just don't want to offer a serious service about email, you 
>>> should think to discontinue the Mail API, like this it makes no sense.
>>>
>>> Cheers,
>>>   Luca 
>>>
>>> On Wednesday, October 26, 2016 at 7:49:05 PM UTC+2, Nick (Cloud Platform 
>>> Support) wrote:
>>>>
>>>> The Public Issue Tracker thread brought our attention has gotten 
>>>> tracked now, thanks to PK. We'll update it with any progress.
>>>>
>>>> Now, as much as I'm sure we all appreciate Jeff's kind offer, I don't 
>>>> think it will be necessary to pay him $1000, since Cloud Platform 
>>>> Community 
>>>> Support are here to do a similar job at no cost. As said in other threads 
>>>> on this topic, if anybody at any time has issues migrating from any 
>>>> service 
>>>> to any other, has questions about design patterns, has questions about the 
>>>> use of our APIs or how to refactor their code, etc., this forum is the 
>>>> perfect place for such higher-level discussion and we would be happy to 
>>>> discuss in depth. 
>>>>
>>>> We never want users to feel left alone, and asking a question once has 
>>>> the benefit that all other users in a similar situation can read and learn 
>>>> from the thread. This doesn't fit under the umbrella of "platform issues / 
>>>> feature requests" (Public Issue Tracker) or "a specific error / stack 
>>>> trace" (Stack Overflow), so it would be fine to post such threads in this 
>>>> forum.
>>>

Re: [google-appengine] Mail API Quotas

[Kaan Soral]

;>> Tracker “ the issue is issue 10560 
>>>> <https://code.google.com/p/googleappengine/issues/detail?id=10560> stuck 
>>>> in state “New” for 2.5 years like 100s maybe thousands other issues in the 
>>>> public tracker.
>>>>
>>>> PK
>>>> p...@gae123.com
>>>>
>>>>
>>>>
>>>>
>>>> On Oct 18, 2016, at 8:50 AM, Joshua Smith <mrjoshu...@gmail.com> wrote:
>>>>
>>>> Nick,
>>>>
>>>> I know you are stuck in the apologist position, and there’s nothing you 
>>>> can do about it, but come on…
>>>>
>>>> It is positively absurd that GAE doesn’t include any bulk email 
>>>> capability. If you guys are concerned about spammers abusing the service, 
>>>> just charge a penny per email sent or something.
>>>>
>>>> If it’s because google can’t figure out how to provide a reliable email 
>>>> service, maybe call the team over at gmail, I hear they have some 
>>>> experience in this area.
>>>>
>>>> You will never convince GAE users that having to use a third-party to 
>>>> send email is anything but a pathetic hack.
>>>>
>>>> -Joshua
>>>>
>>>> On Oct 18, 2016, at 10:55 AM, 'Nick (Cloud Platform Support)' via 
>>>> Google App Engine <google-a...@googlegroups.com> wrote:
>>>>
>>>> Hey PK and Kaan,
>>>>
>>>> I hear your concerns. I hope that I can moderate worries by saying the 
>>>> following in regard to each of the points you brought up:
>>>>
>>>> *1. Avoid paying for attachment bits through front-end*
>>>>
>>>> While the GCS integration to Mail API attachments will be discussed in 
>>>> the next point, there's another GCS solution here which is very useful. 
>>>> You 
>>>> can generate URLs to the attachment objects and merely embed these URLs in 
>>>> the email. This ultimately has the same transfer cost for the user's 
>>>> machine as downloading from a mail server storing the attachment, but is 
>>>> slightly different since you can recall the object at a later date.
>>>>
>>>> *2. Feature requests for GCS integration to Mail API*
>>>>
>>>> We always encourage users to make use of the Public Issue Tracker 
>>>> <https://code.google.com/p/googleappengine/issues/list> to make any 
>>>> feature requests that they think of. We track them and consider them, and 
>>>> this is the best way to ensure that we know about it in a formal manner 
>>>> rather than just posting in a discussion thread. We look forward to all 
>>>> feature requests because it means users are working on the platform and 
>>>>
>>>>
>>>> *3. Platform Unity*
>>>> We definitely recognize that the power of the platform comes from its 
>>>> unity and internal organization. The various services combine to create an 
>>>> ecosystem where development can rapidly integrate many capabilities to 
>>>> produce powerful apps. Because we understand that developers depend on the 
>>>> services for app performance, we offer a minimum 1 year announcement 
>>>> before 
>>>> deprecation, as you can read in the Cloud Platform Terms of Service 
>>>> <https://cloud.google.com/terms/>. 
>>>>
>>>> We also do our best during any deprecation period to provide support 
>>>> and migration advice to developers. Community Support is a resource for 
>>>> you 
>>>> developers, so please don't hesitate to ask us any questions or open 
>>>> discussions of architecture, design, etc. We can't prevent the web or our 
>>>> services from changing over time, as all things must, but we can do our 
>>>> best to provide comprehensive support for users during these changes. 
>>>>
>>>> For the moment, everything said above and elsewhere about the Mail API 
>>>> and Sendgrid remains good advice. Sendgrid is simply far better suited to 
>>>> bulk messages, while the Mail API can be used effectively for 
>>>> administrative emails such as a password reset system, etc.
>>>>
>>>> As always, let us know your thoughts or if you have any further 
>>>> questions! We'll be happy to help.
>>>>
>>>> Cheers,
>>>>
>>>> Nick
>>>> Cloud Platform Community Suppo

Re: [google-appengine] Re: Production SDK stuck at 1.9.48?

[Kaan Soral]

I guess it was "AppEngine Launcher" back in the day, on OSX, it regularly 
popped up and wanted to update itself, there was no 2-second way to get rid 
of it either, it was pretty much behaved like the old adware's

I was afraid gcloud was going to be something like that, however, having a 
CLI system replace another CLI system isn't that bad

I think the main problem is the lack of information, one email, one 
document with simple: "You did this with appcfg, now do this with gcloud" 
document would be enough, as there seem to be 1:1 replacements to appcfg 
operations simply available

On Friday, March 3, 2017 at 11:09:37 AM UTC+3, P K wrote:
>
> Hi Kaan,
>
> I am not sure what what is the GUI you refer to. I never used a GUI, I 
> have always been using appcfg.py as a CLI tool.
>
> PK 
> www.gae123.com
>
> On Mar 2, 2017, at 11:51 PM, Kaan Soral <kaan...@gmail.com > 
> wrote:
>
> Just checked gcloud myself, compared to appcfg, I really really like the 
> examples, as in the past, the GUI approach was spammy and insufficient, I 
> like the new CLI based approach, and the documentations seem adequate too 
> (Feedback: You seriously need to send emails when changes happen, new 
> things are added, I'm ashamed that I didn't discover the gcloud CLI and the 
> new documentations up to this point)
>
> For some projects, I'm locked with old versions of appcfg, yet, for 
> deployments and new projects, gcloud seems nice
>
> I think the confusion arises from this: """ Does the method you deploy 
> with, determine the version of appengine your app runs with?! """ - 
> Because, as an outsider, the assumption is that, unless you explicitly set 
> something, the apps should always run with the latest version
>
> -- 
> You received this message because you are subscribed to a topic in the 
> Google Groups "Google App Engine" group.
> To unsubscribe from this topic, visit 
> https://groups.google.com/d/topic/google-appengine/0s1EHuF4qDs/unsubscribe
> .
> To unsubscribe from this group and all its topics, send an email to 
> google-appengi...@googlegroups.com .
> To post to this group, send email to google-a...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/google-appengine/9355e066-ddde-4492-9632-35030e83bf14%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/google-appengine/9355e066-ddde-4492-9632-35030e83bf14%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/5a025925-6971-4e65-90bc-475e2b7c33bc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Production SDK stuck at 1.9.48?

[Kaan Soral]

Just checked gcloud myself, compared to appcfg, I really really like the 
examples, as in the past, the GUI approach was spammy and insufficient, I 
like the new CLI based approach, and the documentations seem adequate too 
(Feedback: You seriously need to send emails when changes happen, new 
things are added, I'm ashamed that I didn't discover the gcloud CLI and the 
new documentations up to this point)

For some projects, I'm locked with old versions of appcfg, yet, for 
deployments and new projects, gcloud seems nice

I think the confusion arises from this: """ Does the method you deploy 
with, determine the version of appengine your app runs with?! """ - 
Because, as an outsider, the assumption is that, unless you explicitly set 
something, the apps should always run with the latest version

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/9355e066-ddde-4492-9632-35030e83bf14%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Uploading large image files and video to Google Cloud Storage

[Kaan Soral]

I'm not commenting on the issue in general, as things got a bit edgy, 
however, my .02:

I use App Engine, or other cloud products, as they make things easy for me, 
for multi-part uploads, my expectancy is having wrapper libraries that make 
multi-part uploads dead-simple (JS, Mobile: iOS, Java, Cordova, possibly 
others)

In fact, while you are at it, it could even be a large/inclusive library 
for all high level things combined into one package, and possibly, one 
could hand-pick certain things and get a smaller library if size is a 
concern

One example was the old Channel JS library, while I haven't used it myself, 
design-wise it was a nice library, tackled a complex system in a simple way

>From a cloud users' perspective, multi-part uploads is indeed a bit 
complex, and icky challenges like these consume a lot of time, and in the 
end, when you look back, you realise you spent a considerable amount of 
time on these kind of integrations/challenges and on a larger scale, it 
probably has a high impact

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7f3dcd49-e213-4def-8c87-efe914c3544d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Google has fixed 30 of 707 bugs reported in the past 12 months

[Kaan Soral]

Wow, no words to describe how awesome that is <3

On Saturday, December 17, 2016 at 2:38:06 AM UTC+3, Lorne Kligerman wrote:
>
> To close the loop here, there was a decision to postpone charging for logs 
> overages until we can provide you with more control over which Google Cloud 
> Platform logs you choose to send to Stackdriver Logging.  An email was sent 
> a little while back with more details.
>
> @Kaan, you'll be happy to know we were able to bump up the image resize 
> limit from 1600 to 3200 pixels.  It didn't make the cut this year but 
> should be out in the first release of 2017.
>
> As well we'll move forward with improved management of public issues in 
> the new year.
>
> Cheers,
> Lorne.
>
>
> On Thu, Dec 1, 2016 at 2:26 PM, Dimitri Adamou  > wrote:
>
>> Thanks for posting this topic, it has been a very frustrating ride with 
>> GAE - some of these bugs have made me use creative solutions; but holy hell 
>> are they silly bugs! (I've opened issue reports for them since over 2+ 
>> years ago and they are only "acknowledged") - would love active feedback OR 
>> I would love instructions on how to contribute to the project so I can 
>> benefit my business.
>>
>> On Thursday, November 3, 2016 at 2:04:02 PM UTC+11, pdknsk wrote:
>>>
>>>
>>> https://code.google.com/p/googleappengine/issues/list?can=1=opened-after%3Atoday-365=-id
>>>
>>> ^ All 707 (public) bugs reported in the past 12 months.
>>>
>>>
>>> https://code.google.com/p/googleappengine/issues/list?can=2=opened-after%3Atoday-365=-id
>>>
>>> ^ 322 are still open.
>>>
>>>
>>> https://code.google.com/p/googleappengine/issues/list?can=1=opened-after%3Atoday-365+closed-after%3Atoday-365+-status%3Afixed=-id
>>>
>>> ^ 355 were closed for reasons other than Fixed (Invalid, Duplicate and 
>>> the like).
>>>
>>>
>>> https://code.google.com/p/googleappengine/issues/list?can=1=opened-after%3Atoday-365+status%3Afixed=-id
>>>
>>> ^ 30 were fixed.
>>>
>>> 14 are Go bugs, which are fixed by the Go team.
>>> 4 are SDK bugs, which are usually fixed by the gcloud team.
>>> 1 was closed as fixed, but isn't.
>>> 1 is a typo.
>>>
>>> That makes 10 bugs fixed by the App Engine "Engineering Team".
>>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "Google App Engine" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/google-appengine/EFLhIU9GqjU/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to 
>> google-appengi...@googlegroups.com .
>> To post to this group, send email to google-a...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/google-appengine.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/google-appengine/306914c4-cb4b-48f6-af36-5ae6751080e5%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/d30f45c1-283a-429c-923f-9fdbb564184a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Google has fixed 30 of 707 bugs reported in the past 12 months

[Kaan Soral]

I agree, and I suspected you were talking about "Prospective Search", it 
always seemed too specific, however, "Channels" were truly unique and 
useful, if it was a scalable service, I would definitely use it - and I'm 
saddened by it's passing (I use polling heavily on one project, and have a 
nodejs/websocket system in another, if "Channels" bloomed, it would enrich 
the experience significantly)

As my .02 for the original post, I somehow gave up on support long ago, but 
it's not entirely bad either, as it is, appengine is very mature, and even 
if it received 0 updates for years, it would still be useful - but - 
indeed, receiving swift and practical technical support would've been great 
- this was my recent technical 
challenge: https://groups.google.com/forum/#!topic/google-appengine/A2_fr2L0fAY 
- It's a bizarre one, when contention for one entity occurs, it's like the 
transaction is proceeding with a corrupt ndb system, and causing an 
exception that halts the execution, Instead of creating a public issue 
tracker issue, I ended up re-structuring my architecture so contention is 
less likely, it seemed like the easier option, I really didn't have the 
energy to go through the issue tracker process

On Friday, November 25, 2016 at 5:03:00 PM UTC+2, Mark Cummins wrote:
>
> @Kaan Soral. Yes, we also heavily use the images service, it's invaluable 
> and we love it. We were also using mail, although it really wasn't a big 
> problem to move to an external service. (Though incoming is indeed more of 
> an issue) By exotic, I mean things like Prospective Search and 
>  XMPP/Channels. I don't think too many people were really affected by those 
> deprecations.
>
> On Friday, 25 November 2016 13:38:16 UTC, Kaan Soral wrote:
>>
>> It is what I meant, apologies if I was confusing
>>
>> On Friday, November 25, 2016 at 3:36:14 PM UTC+2, George Bittmann wrote:
>>>
>>> Where is the mail deprecation? Post a link if it's happening please. I 
>>> thought they were only no longer approving requests to expand beyond the 
>>> 100 daily limit. Is that what you mean?
>>>
>>>
>>> On Thursday, November 24, 2016 at 8:06:33 AM UTC-5, Kaan Soral wrote:
>>>>
>>>> Hi
>>>>
>>>> Thanks a lot for the reply again
>>>>
>>>> For images, https://cloud.google.com/appengine/docs/python/images/ - 
>>>> the 1600 resize limit started to age a bit, I'm using the images service 
>>>> heavily on https://geobird.com - and considering the recent 4K, 5K 
>>>> resolutions, even bumping the limit to 2400 would improve dynamics a lot, 
>>>> considering it would at least cover 1080p (Current solution is to just 
>>>> serve images with "=s0" - considering most devices have retina resolutions 
>>>> now, most images started becoming "=s0" - and a significant bandwidth is 
>>>> wasted for clients)
>>>>
>>>> If we analyse the case for the images service, it provides dynamic 
>>>> resizing at instant availability, It's an invaluable service, I love 
>>>> appengine because it provides building blocks like these - Instead of 
>>>> depreciating such services, it would be lovely to see new ones, for 
>>>> example, a "videos" service, even though I'm using Zencoder for video 
>>>> processing, if we compare it to the ease of using the images service, it's 
>>>> a heavy burden
>>>>
>>>> @Mark Cummins, I don't know what service you are calling exotic and 
>>>> less-used, It would be great if you could provide concrete examples, 
>>>> interacting with other app engine users, you would be amazed how each 
>>>> component is utilised and valued
>>>> If we analyse the case for mail, I personally was using it to send 
>>>> mails, so, while the deprecation was an extreme nuisance, it wasn't the 
>>>> end 
>>>> of the world, yet, communicating with others, I saw that some were using 
>>>> apps as turnkey solutions, or, they were deploying singular apps for their 
>>>> clients, so having a practical mail solution was crucial to them, some 
>>>> were 
>>>> handling incoming email, while things were simple from the outgoing email 
>>>> perspective, for an app that handled both incoming/outgoing email, the 
>>>> deprecation was much more worrying
>>>>
>>>> On Wednesday, November 23, 2016 at 3:28:07 AM UTC+2, Lorne Kligerman 
>>>> wrote:
>>>>>
>>>>> Heard loud and clear on action speaking louder than words.  We

Re: [google-appengine] Google has fixed 30 of 707 bugs reported in the past 12 months

[Kaan Soral]

It is what I meant, apologies if I was confusing

On Friday, November 25, 2016 at 3:36:14 PM UTC+2, George Bittmann wrote:
>
> Where is the mail deprecation? Post a link if it's happening please. I 
> thought they were only no longer approving requests to expand beyond the 
> 100 daily limit. Is that what you mean?
>
>
> On Thursday, November 24, 2016 at 8:06:33 AM UTC-5, Kaan Soral wrote:
>>
>> Hi
>>
>> Thanks a lot for the reply again
>>
>> For images, https://cloud.google.com/appengine/docs/python/images/ - the 
>> 1600 resize limit started to age a bit, I'm using the images service 
>> heavily on https://geobird.com - and considering the recent 4K, 5K 
>> resolutions, even bumping the limit to 2400 would improve dynamics a lot, 
>> considering it would at least cover 1080p (Current solution is to just 
>> serve images with "=s0" - considering most devices have retina resolutions 
>> now, most images started becoming "=s0" - and a significant bandwidth is 
>> wasted for clients)
>>
>> If we analyse the case for the images service, it provides dynamic 
>> resizing at instant availability, It's an invaluable service, I love 
>> appengine because it provides building blocks like these - Instead of 
>> depreciating such services, it would be lovely to see new ones, for 
>> example, a "videos" service, even though I'm using Zencoder for video 
>> processing, if we compare it to the ease of using the images service, it's 
>> a heavy burden
>>
>> @Mark Cummins, I don't know what service you are calling exotic and 
>> less-used, It would be great if you could provide concrete examples, 
>> interacting with other app engine users, you would be amazed how each 
>> component is utilised and valued
>> If we analyse the case for mail, I personally was using it to send mails, 
>> so, while the deprecation was an extreme nuisance, it wasn't the end of the 
>> world, yet, communicating with others, I saw that some were using apps as 
>> turnkey solutions, or, they were deploying singular apps for their clients, 
>> so having a practical mail solution was crucial to them, some were handling 
>> incoming email, while things were simple from the outgoing email 
>> perspective, for an app that handled both incoming/outgoing email, the 
>> deprecation was much more worrying
>>
>> On Wednesday, November 23, 2016 at 3:28:07 AM UTC+2, Lorne Kligerman 
>> wrote:
>>>
>>> Heard loud and clear on action speaking louder than words.  We have a 
>>> lot brewing and are excited to show it off when ready.
>>>
>>> There have been some deprecation announcements lately.  The objective 
>>> here is to bring App Engine in line with the rest of GCP.  This is a 
>>> challenging effort because of how long App Engine pre-dates the rest of the 
>>> system.  That being said, we do have our users best interests at heart and 
>>> we believe the future state will be a good one.  Significant improvements 
>>> have been made to the App Engine Admin API, which officially went GA 
>>> <https://cloudplatform.googleblog.com/2016/08/automate-deployments-and-traffic-splitting-with-the-App-Engine-Admin-API.html>
>>>  
>>> a couple months ago.
>>>
>>> Regarding mail, we are currently gathering requirements for a GCP wide 
>>> solution.  For images, could you give more detail on exactly which limit 
>>> you're running up against and the scenario?
>>>
>>>
>>> On Tue, Nov 22, 2016 at 2:45 PM, 'Alex (Cloud Platform Support)' via 
>>> Google App Engine <google-a...@googlegroups.com> wrote:
>>>
>>>> Hi Kaan,
>>>>
>>>> On a side note, it is possible to receive regular emails from the 
>>>> Google Cloud Platform by following these steps:
>>>>
>>>>1. Access the Cloud Platform Console
>>>>
>>>>2. Click the “3 vertical dots” on the top-right section of the menu
>>>>
>>>>3. Navigate to preferences->Updates & Offers
>>>>
>>>>4. Set email to “On” for Feature announcements
>>>>
>>>> By doing this, you will receive emails about the latest feature 
>>>> announcements on the Cloud Platform.
>>>>
>>>> Cheers,
>>>>
>>>> Alex
>>>>
>>>> -- 
>>>> You received this message because you are subscribed to a topic in the 
>>>> Google Groups "Google App Engine" group.
>>>> To unsubscribe fro

Re: [google-appengine] Google has fixed 30 of 707 bugs reported in the past 12 months

[Kaan Soral]

Hi

Thanks a lot for the reply again

For images, https://cloud.google.com/appengine/docs/python/images/ - the 
1600 resize limit started to age a bit, I'm using the images service 
heavily on https://geobird.com - and considering the recent 4K, 5K 
resolutions, even bumping the limit to 2400 would improve dynamics a lot, 
considering it would at least cover 1080p (Current solution is to just 
serve images with "=s0" - considering most devices have retina resolutions 
now, most images started becoming "=s0" - and a significant bandwidth is 
wasted for clients)

If we analyse the case for the images service, it provides dynamic resizing 
at instant availability, It's an invaluable service, I love appengine 
because it provides building blocks like these - Instead of depreciating 
such services, it would be lovely to see new ones, for example, a "videos" 
service, even though I'm using Zencoder for video processing, if we compare 
it to the ease of using the images service, it's a heavy burden

@Mark Cummins, I don't know what service you are calling exotic and 
less-used, It would be great if you could provide concrete examples, 
interacting with other app engine users, you would be amazed how each 
component is utilised and valued
If we analyse the case for mail, I personally was using it to send mails, 
so, while the deprecation was an extreme nuisance, it wasn't the end of the 
world, yet, communicating with others, I saw that some were using apps as 
turnkey solutions, or, they were deploying singular apps for their clients, 
so having a practical mail solution was crucial to them, some were handling 
incoming email, while things were simple from the outgoing email 
perspective, for an app that handled both incoming/outgoing email, the 
deprecation was much more worrying

On Wednesday, November 23, 2016 at 3:28:07 AM UTC+2, Lorne Kligerman wrote:
>
> Heard loud and clear on action speaking louder than words.  We have a lot 
> brewing and are excited to show it off when ready.
>
> There have been some deprecation announcements lately.  The objective here 
> is to bring App Engine in line with the rest of GCP.  This is a challenging 
> effort because of how long App Engine pre-dates the rest of the system.  
> That being said, we do have our users best interests at heart and we 
> believe the future state will be a good one.  Significant improvements have 
> been made to the App Engine Admin API, which officially went GA 
> 
>  
> a couple months ago.
>
> Regarding mail, we are currently gathering requirements for a GCP wide 
> solution.  For images, could you give more detail on exactly which limit 
> you're running up against and the scenario?
>
>
> On Tue, Nov 22, 2016 at 2:45 PM, 'Alex (Cloud Platform Support)' via 
> Google App Engine  wrote:
>
>> Hi Kaan,
>>
>> On a side note, it is possible to receive regular emails from the Google 
>> Cloud Platform by following these steps:
>>
>>1. Access the Cloud Platform Console
>>
>>2. Click the “3 vertical dots” on the top-right section of the menu
>>
>>3. Navigate to preferences->Updates & Offers
>>
>>4. Set email to “On” for Feature announcements
>>
>> By doing this, you will receive emails about the latest feature 
>> announcements on the Cloud Platform.
>>
>> Cheers,
>>
>> Alex
>>
>> -- 
>> You received this message because you are subscribed to a topic in the 
>> Google Groups "Google App Engine" group.
>> To unsubscribe from this topic, visit 
>> https://groups.google.com/d/topic/google-appengine/EFLhIU9GqjU/unsubscribe
>> .
>> To unsubscribe from this group and all its topics, send an email to 
>> google-appengi...@googlegroups.com .
>> To post to this group, send email to google-a...@googlegroups.com 
>> .
>> Visit this group at https://groups.google.com/group/google-appengine.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/google-appengine/c37962f7-c400-44d9-9ab8-dfd3edc7ec1f%40googlegroups.com
>>  
>> 
>> .
>>
>> For more options, visit https://groups.google.com/d/optout.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3988b0b9-29c0-475b-8d08-97ba44af3d1a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Google has fixed 30 of 707 bugs reported in the past 12 months

[Kaan Soral]

Thanks for the reply, it was good to see

As a small feedback, in essence and core, we all embraced appengine as it 
was a unified solution, each and every service amplifies the experience, so 
instead of deprecating these valuable services, please add to them instead

Two very basic examples are "images" and "mail", we've lost "mail", and 
with the resolution increases in recent years, the limits of "images" 
started being actual limits

As another constructive suggestion, it would be nice to receive regular 
emails from Google Cloud, maybe we could opt-in to gae specifically, and 
receive updates of everything new, fixed and improved

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3fc12628-be55-46c2-9f98-78b8c38256b2%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: During Contention, Entities are Returned None in NDB Transactions

[Kaan Soral]

And to clarify, both entities are fetched by using the id's of entities 
that are outside the transaction, so it's not a missing dependency thing

On Thursday, November 3, 2016 at 1:19:44 PM UTC+2, Kaan Soral wrote:
>
> Hi
>
> With high retry[60] ndb transactions, I'm getting 2 contention exceptions 
> (2 consecutive "TransactionFailedError(too much contention on these 
> datastore entities..." logs), on the third retry, the get routine returns 
> None instead of the entity, and the routine fails with an exception trying 
> to access a property of None
>
> Is this expected?
>
> It's highly unexpected to me
>
> As an additional info, it's an "XG" transaction, the "None" entity is the 
> second entity that's fetched after an exception is logged for the first 
> entity
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/1f824526-e026-4e62-ace2-d6889698cafe%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] During Contention, Entities are Returned None in NDB Transactions

[Kaan Soral]

Hi

With high retry[60] ndb transactions, I'm getting 2 contention exceptions 
(2 consecutive "TransactionFailedError(too much contention on these 
datastore entities..." logs), on the third retry, the get routine returns 
None instead of the entity, and the routine fails with an exception trying 
to access a property of None

Is this expected?

It's highly unexpected to me

As an additional info, it's an "XG" transaction, the "None" entity is the 
second entity that's fetched after an exception is logged for the first 
entity

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/f4606ef5-f8c6-468a-bb15-b9de601b5124%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: Is there something like a "back_off" for ndb.transaction's?

[Kaan Soral]

My initial problem with push queues was that they were using their retries 
completely internally, so once you get them to do "infinite retries", the 
losses became 0%

The problem with pull queues was throughput, sadly, pull queues are much 
better aimed to battle contention, especially when used together with

*Ps. I assume, for my original problem, manual retrying is the simplest 
solution*

What kind of a method with push queues do you use to solve the contention 
issue tho?
I forgot my original method that used only push queues, my current method 
uses pull queues for tasks, and push queues for processing, there are 
delays, there are losses (it's always an ongoing battle), that's why with 
my new project, I just kept it strictly to simple transactions

On Saturday, October 29, 2016 at 6:19:34 PM UTC+3, PK wrote:
>
> I have found  push task queues very reliable and use them all the time. 
>
> PK
> http://www.gae123.com
>
> On Oct 29, 2016, at 7:29 AM, Kaan Soral <kaan...@gmail.com > 
> wrote:
>
> Thanks for sharing retrying, indeed it's pretty interesting
>
> My situation is pretty unique, that's why there are usually 3 consecutive 
> requests in the worst case and at most 5-6, that's why an architecture 
> change is an overkill
>
> I had a lot of experience with both push queue and pull queue based 
> solutions in the past, and I can safely say that, "better let them content"
>
> In my opinion, both pull queues and push queues are extremely unreliable, 
> so on this project, I made it so that the occasional contention is safe to 
> occur
>
> However, I hoped with retries=6 the transactions would untangle them on 
> their own, but they don't
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to google-appengi...@googlegroups.com .
> To post to this group, send email to google-a...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/google-appengine/64683751-e390-4f16-ad0c-ec4a94246256%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/google-appengine/64683751-e390-4f16-ad0c-ec4a94246256%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/45b07b9b-7561-41a0-b295-4e2af3e9a308%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Is there something like a "back_off" for ndb.transaction's?

[Kaan Soral]

Thanks for sharing retrying, indeed it's pretty interesting

My situation is pretty unique, that's why there are usually 3 consecutive 
requests in the worst case and at most 5-6, that's why an architecture 
change is an overkill

I had a lot of experience with both push queue and pull queue based 
solutions in the past, and I can safely say that, "better let them content"

In my opinion, both pull queues and push queues are extremely unreliable, 
so on this project, I made it so that the occasional contention is safe to 
occur

However, I hoped with retries=6 the transactions would untangle them on 
their own, but they don't

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/64683751-e390-4f16-ad0c-ec4a94246256%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Is there something like a "back_off" for ndb.transaction's?

[Kaan Soral]

Often, when 3-4 transaction's hit my app at the same time, even with 
retries=6, they cause a contention exception

My custom solution to the issue would be, use retries=0, but manually 
implement a retry logic, like, sleep a random amount of time, retry

So basically, when 3-4 transaction's occur consecutively, this solution 
would un-tangle them under 60 seconds (assumption)

I'm just wondering whether this is a good solution and whether there is an 
existent solution I'm missing, basically, I want to be able to handle 5 
consecutive transaction's

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/789b10e1-48bf-4cca-9e45-aa780969e630%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Mail API Quotas

[Kaan Soral]

Good thing we are locked-in for life to App Engine. Using App Engine for 
maybe 6 years, It's part of me

Seeing this announcement, and practically experiencing the roadblock, 
devastates me

The beauty of App Engine comes from it's unity, when you take a vital part 
of that unity out and kill it, it inflicts terror and fear, I fear that 
more and more services are going to be terminated one day

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/2235fbfa-f2e2-4609-a894-58c7b4397ee8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Requesting an Immediate Mail API Quota Increase for Halloween

[Kaan Soral]

Thanks for the swift reply, I truly do appreciate it, I was intending to 
send the emails in 16 hours, now I probably have enough time to set up 
Amazon SES

On Tuesday, October 18, 2016 at 12:51:58 AM UTC+3, Adam (Cloud Platform 
Support) wrote:
>
> As of March we no longer accept mail quota increase requests. For the 
> original groups post see 
> https://groups.google.com/forum/#!topic/google-appengine/p3tma8eFCM4.
>
> Email quotas documentation:
> https://cloud.google.com/appengine/docs/quotas#Mail
>
> On Monday, October 17, 2016 at 5:31:19 PM UTC-4, Kaan Soral wrote:
>>
>> The Gmail/Bulk thing seems like a nice idea to pass the day, I don't 
>> favor mailchimp, both solutions are highly demotivational tho, for a 
>> programmer to send bulk emails manually ...
>>
>> Looked into Amazon SES's authentication routine, I might go through with 
>> that
>>
>> Going to wait a day first tho, Hoping to get a fast reply, Wish I checked 
>> the quotas earlier, I remember default quotas being higher, at 100, even a 
>> busy day of signups could deplete it easily
>>
>> On Monday, October 17, 2016 at 11:34:32 PM UTC+3, Joshua Smith wrote:
>>>
>>> If you just want to send an announcement out, there are other ways to do 
>>> that. With just 700 to send, you could just do it from gmail in a few bulk 
>>> sends. Or you could import the list to mailchimp or one of the other 
>>> mailing list services.
>>>
>>> On Oct 17, 2016, at 4:07 PM, Kaan Soral <kaan...@gmail.com> wrote:
>>>
>>> I just noticed that the default "Recipients Emailed" limit is 100, It's 
>>> a shock as I just managed to complete a Halloween email for a game I've 
>>> been working on
>>>
>>> It's a small/hobby game, I just need to email 700 people, but I'm 
>>> already late, as a lot of work has been done for Halloween, without an 
>>> announcement, that work is ~wasted
>>> (At inception, I've spent a lot of time setting up a Google Apps account 
>>> (the new version), integrated it to App Engine, paying monthly bills, just 
>>> to send emails ..)
>>>
>>> I mean, come on, 100 Default Mail's is so low, for a billing enabled 
>>> app, at least make it 1000 on default :(
>>>
>>> Anyway, I've already sent a quota increase request, the app id is 
>>> "twodimensionalgame" / http://adventure.land - Putting this here hoping 
>>> that someone would see it and maybe accelerate the process, assuming it's 
>>> even possible (I really don't like making a request like this, but given 
>>> the time constraint, I'm desperate)
>>>
>>> While building something, it's hard to keep the morale up, and 
>>> roadblocks like these are really demotivating, I'm torn right now
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "Google App Engine" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to google-appengi...@googlegroups.com.
>>> To post to this group, send email to google-a...@googlegroups.com.
>>> Visit this group at https://groups.google.com/group/google-appengine.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/google-appengine/f6dc36e6-c7c5-4f57-8a59-1f53267c944a%40googlegroups.com
>>>  
>>> <https://groups.google.com/d/msgid/google-appengine/f6dc36e6-c7c5-4f57-8a59-1f53267c944a%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>> For more options, visit https://groups.google.com/d/optout.
>>>
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/d88ffcaa-d19c-42e9-bfc1-5d3b4464bc71%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Requesting an Immediate Mail API Quota Increase for Halloween

[Kaan Soral]

The Gmail/Bulk thing seems like a nice idea to pass the day, I don't favor 
mailchimp, both solutions are highly demotivational tho, for a programmer 
to send bulk emails manually ...

Looked into Amazon SES's authentication routine, I might go through with 
that

Going to wait a day first tho, Hoping to get a fast reply, Wish I checked 
the quotas earlier, I remember default quotas being higher, at 100, even a 
busy day of signups could deplete it easily

On Monday, October 17, 2016 at 11:34:32 PM UTC+3, Joshua Smith wrote:
>
> If you just want to send an announcement out, there are other ways to do 
> that. With just 700 to send, you could just do it from gmail in a few bulk 
> sends. Or you could import the list to mailchimp or one of the other 
> mailing list services.
>
> On Oct 17, 2016, at 4:07 PM, Kaan Soral <kaan...@gmail.com > 
> wrote:
>
> I just noticed that the default "Recipients Emailed" limit is 100, It's a 
> shock as I just managed to complete a Halloween email for a game I've been 
> working on
>
> It's a small/hobby game, I just need to email 700 people, but I'm already 
> late, as a lot of work has been done for Halloween, without an 
> announcement, that work is ~wasted
> (At inception, I've spent a lot of time setting up a Google Apps account 
> (the new version), integrated it to App Engine, paying monthly bills, just 
> to send emails ..)
>
> I mean, come on, 100 Default Mail's is so low, for a billing enabled app, 
> at least make it 1000 on default :(
>
> Anyway, I've already sent a quota increase request, the app id is 
> "twodimensionalgame" / http://adventure.land - Putting this here hoping 
> that someone would see it and maybe accelerate the process, assuming it's 
> even possible (I really don't like making a request like this, but given 
> the time constraint, I'm desperate)
>
> While building something, it's hard to keep the morale up, and roadblocks 
> like these are really demotivating, I'm torn right now
>
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To unsubscribe from this group and stop receiving emails from it, send an 
> email to google-appengi...@googlegroups.com .
> To post to this group, send email to google-a...@googlegroups.com 
> .
> Visit this group at https://groups.google.com/group/google-appengine.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/google-appengine/f6dc36e6-c7c5-4f57-8a59-1f53267c944a%40googlegroups.com
>  
> <https://groups.google.com/d/msgid/google-appengine/f6dc36e6-c7c5-4f57-8a59-1f53267c944a%40googlegroups.com?utm_medium=email_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/6023a90f-5212-4fc3-8974-09ce4dd90176%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Requesting an Immediate Mail API Quota Increase for Halloween

[Kaan Soral]

I just noticed that the default "Recipients Emailed" limit is 100, It's a 
shock as I just managed to complete a Halloween email for a game I've been 
working on

It's a small/hobby game, I just need to email 700 people, but I'm already 
late, as a lot of work has been done for Halloween, without an 
announcement, that work is ~wasted
(At inception, I've spent a lot of time setting up a Google Apps account 
(the new version), integrated it to App Engine, paying monthly bills, just 
to send emails ..)

I mean, come on, 100 Default Mail's is so low, for a billing enabled app, 
at least make it 1000 on default :(

Anyway, I've already sent a quota increase request, the app id is 
"twodimensionalgame" / http://adventure.land - Putting this here hoping 
that someone would see it and maybe accelerate the process, assuming it's 
even possible (I really don't like making a request like this, but given 
the time constraint, I'm desperate)

While building something, it's hard to keep the morale up, and roadblocks 
like these are really demotivating, I'm torn right now

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/f6dc36e6-c7c5-4f57-8a59-1f53267c944a%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Python: iterating over a large dataset

[Kaan Soral]

Without going into the code, you should

a) Fetch ~100
b) Update Cursor
c) Stop if you are close to the time limit

So that you can re-run the routine with the cursor later on

So basically, instead of `for person in people:` - do a while loop and 
manually fetch entities, instead of using the `in` operator for the query

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/d8bbc2d8-b999-4622-b740-df3e03ccd07f%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Deployments keep on failing, localhost:8080 issues

[Kaan Soral]

Hi

Thanks for the reply

This is the url I 
get: 
https://accounts.google.com/o/oauth2/auth?scope=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email_uri=http%3A%2F%2Flocalhost%3A8080%2F_type=code_id=XXX.apps.googleusercontent.com_type=offline

I use appcfg.py to deploy, I think it creates a temporary webserver for 
deployments, which get stuck, this is my theory

Some findings:
1) Shutting down the dev_appserver which natively uses 8080 port doesn't 
solve the issue
2) The issue usually happens on re-deployments, when it happens, it usually 
last 10 to 30 minutes

I usually deploy in heat, sometimes a new deployment causes an issue, you 
need to re-deploy fast, this issue steals a lot of precious time that would 
otherwise be productive

I was using the 1.9.36 SDK, switched to 1.9.40, I don't know whether it 
fixes the issue

In any case, I just tried `--noauth_local_webserver` - should've tried it 
earlier, so my wasted time is on me

I'm speculating there is a temporary webserver that just serves to take 
that auth code from the browser and feed it into appcfg, that webserver 
glitches a lot (at least it did on 1.9.36, I didn't get to test whether 
things are same on 1.9.40)

On Monday, September 12, 2016 at 10:30:36 PM UTC+3, Nick (Cloud Platform 
Support) wrote:
>
> Hey Kaan,
>
> I think there's an unstated assumption here which is creating a bit of 
> difficulty in getting to a consensus on the next steps. We don't expect 
> that browser auth for gcloud should be redirecting to localhost:8080. You 
> should be given a URL like the following:
>
>
> https://accounts.google.com/o/oauth2/auth?redirect_uri=urn%3Aietf%3Awg%3Aoauth%3A2.0%3Aoob=select_account_type=code_id=12345678.apps.googleusercontent.com=https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute_type=offline
>
> In order to determine what's going on, since it's not quite clear really 
> what's even going on, let alone why it might be going on, we'll need some 
> precise technical details that can be given in a Public Issue Tracker 
> <http://code.google.com/p/google-cloud-platform/issues/list> report as 
> mentioned above. The precise code / commands relevant along with HAR 
> recordings of the browser activity, or a tcp dump of the browser's activity 
> during that time, should be sufficient.
>
> Let me know when you've made a Public Issue Tracker report so I can take a 
> look at it right away, although of course we do aim to respond to threads 
> as soon as possible and monitor the forums daily.
>
> Cheers,
>
> Nick
> Cloud Platform Community Support
>
>
> On Monday, September 12, 2016 at 4:42:02 AM UTC-4, Kaan Soral wrote:
>>
>> This started being a regular issue for me, now the auth's redirect to 
>> :8090 - I'm using the port 8090 myself, the SDK uses the 8080
>>
>> The initial deployment usually works, but for re-deployments, I'm usually 
>> hit by this issue :(
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/9f359582-0677-44d5-a949-cf7521a11d21%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Deployments keep on failing, localhost:8080 issues

[Kaan Soral]

This started being a regular issue for me, now the auth's redirect to :8090 
- I'm using the port 8090 myself, the SDK uses the 8080

The initial deployment usually works, but for re-deployments, I'm usually 
hit by this issue :(

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/d909285c-f3ac-4666-b1a2-01b8f34737b5%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Deployments keep on failing, localhost:8080 issues

[Kaan Soral]

How does the deployment auth work?

Why does it fail at times and work flawlessly at other times?

You does it redirect to localhost:8080

You could at least give some hints

On Friday, September 9, 2016 at 2:54:05 AM UTC+3, Nick (Cloud Platform 
Support) wrote:
>
> Hey Kaan,
>
> This doesn't appear to contain enough information to constitute an 
> effective start for any investigation from our side. If you believe this is 
> an issue related to the platform code, you could file an issue in the 
> Public Issue Tracker, probably with a HAR recording of the browser activity 
> during the interaction, and any command line output, along with relevant 
> info such as the version of the SDK. Be sure to attach the label 
> "Restrict-View-EditIssue" to ensure that your HAR information is not 
> exposed publicly. With that information, we'd be in a position to look 
> further. 
>
> Cheers,
>
> Nick
> Cloud Platform Community Support 
>
> On Sunday, September 4, 2016 at 4:56:52 AM UTC-4, Kaan Soral wrote:
>>
>> This used to happen every once in a while, but lately it happens 99% of 
>> the time, the browser auth process just fails to complete, it halts, 
>> redirects to http://localhost:8080/ that doesn't respond
>>
>> Any ideas?
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/0354e4ed-62c3-4886-9ca5-fecbcb3745e7%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Deployments keep on failing, localhost:8080 issues

[Kaan Soral]

How does the deployment auth work?

Why does it fail at times and work flawlessly at other times?

Why does it redirect to localhost:8080?

You could at least give some hints

On Friday, September 9, 2016 at 2:54:05 AM UTC+3, Nick (Cloud Platform 
Support) wrote:
>
> Hey Kaan,
>
> This doesn't appear to contain enough information to constitute an 
> effective start for any investigation from our side. If you believe this is 
> an issue related to the platform code, you could file an issue in the 
> Public Issue Tracker, probably with a HAR recording of the browser activity 
> during the interaction, and any command line output, along with relevant 
> info such as the version of the SDK. Be sure to attach the label 
> "Restrict-View-EditIssue" to ensure that your HAR information is not 
> exposed publicly. With that information, we'd be in a position to look 
> further. 
>
> Cheers,
>
> Nick
> Cloud Platform Community Support 
>
> On Sunday, September 4, 2016 at 4:56:52 AM UTC-4, Kaan Soral wrote:
>>
>> This used to happen every once in a while, but lately it happens 99% of 
>> the time, the browser auth process just fails to complete, it halts, 
>> redirects to http://localhost:8080/ that doesn't respond
>>
>> Any ideas?
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/1ec1737c-daf5-43e2-9d17-26d9cdd10c68%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Deployments keep on failing, localhost:8080 issues

[Kaan Soral]

This used to happen every once in a while, but lately it happens 99% of the 
time, the browser auth process just fails to complete, it halts, redirects 
to http://localhost:8080/ that doesn't respond

Any ideas?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/44e8cded-7446-4c8b-b8b2-83f479ce51c1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: InvalidSenderError: Unauthorized sender

[Kaan Soral]

If anyone experiences this issue in the future:
1) Add email / google apps account as "Editor"
2) From the email's google apps account, go to the "Google Cloud Platform" 
project
3) Go to "Settings"
4) Add the email to the sender list, you can only do this from the account, 
at this time

Not trivial but at least the new docs seem to document it

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/ec4c906f-1770-4082-b275-203aca76e9b6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: InvalidSenderError: Unauthorized sender

[Kaan Soral]

Further debugging: the reason my personal email wasn't working was because 
of capitalisation, for some reason appengine added capitals to my email 
address, and without those capitals, the email isn't sent, bizarre

The issue still remains tho, I can't send an email from the Owner I've 
added, I also added "Appengine Admin" as a permission, but no luck

On Thursday, July 28, 2016 at 11:22:27 PM UTC+3, Kaan Soral wrote:
>
> Hi
>
> I've deployed a new project, however I can't send an email from both my 
> own gmail address, and a new google apps email address that I added as an 
> "Owner" (overkill)
>
> With previous projects, both of these worked without issues, is this no 
> longer the case?
>
> (As a small suggestion, I don't want to add the google apps email as an 
> "Owner", an "Email Sender" role would become the most used role if added, 
> since I'm sure most people just add users to just send emails from those 
> addresses/aliases)
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/da091d05-6e19-4d50-94eb-e63140d77366%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] InvalidSenderError: Unauthorized sender

[Kaan Soral]

Hi

I've deployed a new project, however I can't send an email from both my own 
gmail address, and a new google apps email address that I added as an 
"Owner" (overkill)

With previous projects, both of these worked without issues, is this no 
longer the case?

(As a small suggestion, I don't want to add the google apps email as an 
"Owner", an "Email Sender" role would become the most used role if added, 
since I'm sure most people just add users to just send emails from those 
addresses/aliases)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/3160a4f8-743f-4a77-9de3-ff819a292f3c%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Show and Tell: Geobird - A Location Based Platform Built with AppEngine

[Kaan Soral]

I feel your pain, yet don't agree completely, yes back in the day we had 
some hard times, but recently things have been pretty positive - at one 
occasion, reps even pledged to "move mountains" for us, if demand needs be

For me the challenges have been more throughput related, you can't just 
blast a service and expect it to handle that blast gracefully, each service 
seems to have it's own limits that are not always apparent

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4125b256-b5a9-4e44-9cd5-806791256ef0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Mail Service Deprecation?

[Kaan Soral]

The challenging part of email sending is the verification and related dns 
uglinesses, so it's not as simple as replacing the API layer

Even setting up senders with App Engine is quite challenging, you have to 
create the email, add that email as an admin etc. (back in the old days, 
not sure how it works now, the hard part is the research and verification)

My suggestion would be to take the hard path and do what sendgrid does, 
manually hunt the abusers

Here is a funny analysis of the situation:

1) Google is in the cloud provider business, Google wants people to use 
it's cloud offerings
2) Google shuts down a cloud service, Google sends precious customers to 
another company

It's admirable how contradicting (1) + (2) is

I think the contradiction arises from (1) - Why is Google providing cloud 
services? Maybe it shouldn't, if it's pulling out of the Mail, the same 
could happen for all of the services soon

I think this is why this situation is concerning

Other than this, I roughly agree with how easy it would be to move the mail 
sending elsewhere, especially for apps that send small amount of emails, 
for >1000 emails, I don't think SendGrid would easily allow sending emails, 
as far as I contacted such services before, they considered high amount of 
emails as newsletters and required very strict consents from users and 
proof of such consent

I would probably move to AWS's email service, I have been using it on SDK 
for maybe ~5+ years, same code still works for free

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/eee70341-7bc2-40ac-8b2e-24f4469da3ed%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Show and Tell: Geobird - A Location Based Platform Built with AppEngine

[Kaan Soral]

 

Hi


I want to share Geobird's story with you today, back in the early days we 
had one more of these "Show and Tell"'s that I recall (this: 
https://groups.google.com/forum/#!topic/google-appengine/JMBF9gJvUJM) - I 
enjoyed it a lot, and I hope you enjoy this one


A bit of a back story, I fancied App Engine since it's inception, at the 
time Python was an alien language, learned Python for App Engine (let's 
switch to informal, drop capitals and call it appengine from now on), 
moved/refactored my projects of the time to appengine and served more than 
a billion pages with appengine - no regrets, many quirks, such potential


So building geobird was very enjoyable, I already had an understanding of 
nosql/python/appengine and had battle-tested/fixed routines at hand. At 
first, all the challenges were technical, things were simple, not 
technically simple, but rather existentially simple. Building new systems, 
designing new things, working around nosql limitations, battling contention 
and stuff like that.


I don't regret any of the time I spent on appengine issues and quirks, yet 
I deeply regret using html5 on mobile and solving the issues that came with 
that, especially android ones, in my opinion, only now html5 is usable on 
mobile, a year ago it probably wasn't, unless you gave a considerable chunk 
of your soul to the cause - so building a better but ios-only mobile app 
would have been a nice trade-off


One technical mistake I made was to not use transactions, currently it's 
unthinkable, transactions should be used for almost everything, yet at the 
time, I was like, "Is this a banking app, no need for transactions" - as 
soon as I launched the first prototype, I paid the price of that mistake 
over and over again. But it turned out well, I had to make everything 
transactional, refactoring everything would be almost impossible, so I came 
up with a slip-on transactional system, currently the same system handles 
contention too, so overall my initial mistake had a very positive outcome 
for the project.


Currently the backend is very satisfactionally stable, while in the initial 
days fixes/modifications and re-builds were frequent, I haven't touched the 
base routines in many months. This is what I love about appengine, it's 
challenging to get it right, but once you do, you can just relax, at least 
on the technical side of things.


Some lessons:

- Use App Engine with ease of mind yet be ready to solve challenges that 
come with it, it's worth the effort

- Concentrate more on UI/UX rather than operational excellence

- Concentrate most on PR/Marketing

- Make things simple


I procrastinated a lot by building and improving. Not wise.


Enough about the process.


Here is geobird: https://www.geobird.com/


So basically you share things with the world, pan into any part of the 
world, see stuff from that location


iOS: https://itunes.apple.com/us/app/geobird-explore-the-world/id882320727

Android: https://play.google.com/store/apps/details?id=com.geobird.app

More info: https://www.geobird.com/info

(Recently changed the vision, no more high ambitions, prepared this page, 
probably needs improvements :)


Feel free to ask my anything,

I would also love to get feedback and suggestions,

Sincerely,

Kaan

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/78ce3bf1-1d03-489c-8d82-010b44dea10b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Google App Engine multiple regions

[Kaan Soral]

I think one way or another, there's going to be a delay, why do you need to 
minimise the delay so much?

Even if the instances were spread over the world, I'm guessing there would 
be a smaller data access delay in most cases

I'm way outside US, my users are closer to EU, yet my appengine location is 
US, didn't have any issues up to now

Out of curiosity, I pinged my domain and the appspot domain, the domain is 
wrapped with Cloudflare

Appspot-> 30ms
Cloudflare-> 60ms

Before this experiment I expected Cloudflare to decrease the international 
ping, it didn't

Then I wget'd the domain

Appspot-> 0.1s
Cloudflare-> 0.2s

It obviously doesn't directly answer your question, yet it's an insight

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4ffef92a-ce73-4e87-8fee-579d1d67d9fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: AppEngine with Endpoints

[Kaan Soral]

I think your answer is No, it's not a good idea to create a separate app

One app can do it all, front-end is served by the back-end too

If I understand your question correctly that is

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/906cb0b5-5a75-4283-8772-0e5cac3d817b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Google AppEngine blocked by MailChimp because of security issues

[Kaan Soral]

I think a virtual_ip feature might be nice for these kind of problems, you 
could buy IP's, and add them as an argument to .fetch calls like 
.fetch(virtual_ip="1.2.3.4") etc. - As a lot of people are experiencing 
similar issues, mainly while fetching url's

Other than this idea, let's hope they can fix your issue

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/e7ca680a-3138-451e-8247-4c9958af32bd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Implementing a personalized feed on Google App Engine

[Kaan Soral]

"personalized content" is complex regardless of GAE

I have chronological + geo feeds, for each event, an entity is written for 
each person

Then, that person fetches those entities while digesting the feed

IMO, whatever you do, you have to pre-process things with GAE, as the 
on-request capacity is pretty low, it's not logical to dynamically generate 
a feed in a timely manner 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/61645d05-61f2-4d75-80e9-40834733377d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Unwelcome change: Paginating the task queue list

[Kaan Soral]

+1

Not that I use/can-use the console anymore

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b0e48266-e425-4839-8c3c-4ba9f262f438%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Push queues: No. of running tasks < No. of tasks in queue < Max number of concurrent tasks

[Kaan Soral]

It could be instance related, if your instances are not aggressive, it 
might occur

If tasks use too much memory etc., it might be hard to run them, my .02

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/256f5bd5-1446-40c4-a0f7-6e21436de479%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Auth issue with deployment - https://developers.google.com/accounts/docs/AuthForInstalledApps

[Kaan Soral]

No problem, thanks for the info

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c38a0890-684c-4c60-aac7-b7ea2dc7bb12%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Auth issue with deployment - https://developers.google.com/accounts/docs/AuthForInstalledApps

[Kaan Soral]

Anyway, so at this point I'm 99% sure the email/password auth is disabled 
without an email as a warning

My planned solution was to download a new sdk just to handle the 
deployments, did that now, oauth2 works well from the CLI

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/684539aa-6a11-4344-9b78-e259b86886fb%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Auth issue with deployment - https://developers.google.com/accounts/docs/AuthForInstalledApps

[Kaan Soral]

I keep re-experiencing this issue, I'm using an old SDK, is the auth 
deprecated or something?

On Tuesday, April 19, 2016 at 7:23:17 PM UTC+3, Kaan Soral wrote:
>
> Error 404: --- begin server output ---
>
>  
>
> https://developers.google.com/accounts/docs/AuthForInstalledApps 
>
> --- end server output ---
>
>
>
>
> The deployment asks for the password once, after that, it asks for the 
> password again, after the deployment has started, and fails with this error
>
>
> These is another auth issue that is marked as solved an hour ago, I'm 
> guessing this issue might be related
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/2b4ce9e2-590e-4eab-8351-4b214419f2f0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Auth issue with deployment - https://developers.google.com/accounts/docs/AuthForInstalledApps

[Kaan Soral]

 Error 404: --- begin server output ---

 

https://developers.google.com/accounts/docs/AuthForInstalledApps 

--- end server output ---




The deployment asks for the password once, after that, it asks for the 
password again, after the deployment has started, and fails with this error


These is another auth issue that is marked as solved an hour ago, I'm 
guessing this issue might be related

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/a21ee3dd-7d94-4367-a9f7-64ac0a5d9960%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Sometimes .appspot.com just doesn't respond

[Kaan Soral]

Thanks a lot for the reply Wolfram, I hope it urges the App Engine/Google 
team to give this issue a proper inspection

My issue was exactly like yours, the connection stays open, browser 
lingers, no logs whatsoever from the App Engine side

Over the course of 3 years, I received various reports of "white pages" - 
it's how users interpret the issue, since these un-answered but lingering 
requests result in browser "white pages" - so I assume the issue we 
experienced and this user-issue might be linked, so it might span 1+ years, 
however they might be unrelated too
I really hate that issues like these cause a product to decay, I probably 
lost a lot of users to this "white page" issue, because only a small 
fraction of my userbase use the report function, maybe 0.01%, yet I 
received countless reports


On Saturday, April 9, 2016 at 12:39:30 PM UTC+3, Wolfram Gürlich wrote:
>
> Hi Kaan,
>
> you're not alone. We noticed the same issue, too. We have server processes 
> making like 20request/second all the time to appengine, and all of the 
> sudden we were experiencing these lingering requests. It all began earlier 
> this year and still continues. Those request don't leave any traces in the 
> logs and keep the TCP connection open very long (like at least 20 minutes). 
> They never give any reponse at all. 
>
> The funny thing is, we first encountered these request while connecting 
> from a GCE instance. This was over .appspot.com. The error was gone when 
> we switched the connection to a custom domain. But that didn't last. We 
> then saw this errors on other machines, too - not only from GCE. And they 
> also appeared on the custom domain (SNI SSL) and also over VIP SSL.
>
> So I would think it's neither 1) or 2). Instead I would think it is a 
> problem with the Google frontend server instances that get somehow stuck 
> when dispatching the requests.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/adcdab4e-4c48-45fd-8f1e-70909ab950fd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Sometimes .appspot.com just doesn't respond

[Kaan Soral]

Like I mentioned in my OP, the white page is the result of a lingering 
request with no response, this much I deducted from the browser debuggers

Yet, the issue hit me when I was deep in another task, so I couldn't/didn't 
trace the network issue beyond the simple browser check
(I regret not debugging it further when I had the issue producing)

I'm 99.9% sure it's not a code issue, the app is very old and robust, the 
requests in question are 200-300ms

In my opinion, it's either:
1) Instance issue, no instances are assigned to requests
2) Local/temporary network issue, the requests are coming from Turkey from 
the main ISP

(1) seems more likely to me, these also happened after a deployment, and 
seemed to go away after another deployment, so I might have caught a better 
cluster during my deployments

-- -- --

Actually, not reflecting on the issue, (2) seems very likely too, because 
regularly, all google.com services are unreachable from Turkey for 2-3 
minutes, I sometimes can't reach gmail, the issue is fixes itself after 2-3 
minutes

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/9d5483b8-2fac-4091-b063-52c1bedbd949%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Sometimes .appspot.com just doesn't respond

[Kaan Soral]

This was an issue that I've been hunting for years, some people report 
"white pages" on my old app system (facebook apps) - I never could 
reproduced the issue, I assumed it was a rare client-side issue

Now, while testing my app, I've noticed the same issue, .appspot.com of my 
app just wasn't responding, I was only able to get it to respond my 
refreshing several times

The app is simple, there are no errors logged, the request simply doesn't 
go through

Never noticed the same issue on another app, yet since this other app is 
mostly Cloudflare wrapped, I wouldn't notice it

Any ideas?

It's just sad knowing this issue exists in the wild, yet I have no way of 
fixing it, it's causing me to lose users slowly

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/b508fc29-54e8-4f2e-b59f-e6b049214006%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: urlfetch.fetch reporting inconsistent results?

[Kaan Soral]

As a very sincere suggestion, try not to build anything for fb at all, if 
you can (and for instagram/twitter for that matter)

It's basically a trap for developers, you waste your resources (i), there 
is no positive outcome at all, even if you manage to get a positive 
outcome, it will be terminated with various excuses and sometimes no 
excuses at all

More on topic, unexplainable fb api failures are pretty common, I just 
ignore them at this point, try not to lose time on it, if it's not mission 
critical, in some months, it may even fix itself, even if you report it to 
fb and it gets accepted as a bug, don't expect a swift solution (because 
(i))

The only valid fb api usage at this point is b2b apps, where the 
end-business is the ones that are trapped, like page owners etc., you might 
profit from selling apps to them while their waste their times with their 
fb pages, even in this case, if you are not at the point of no return, 
return and do something useful :)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/d7e5bdba-37cb-4c54-bf18-4b8669abb5ee%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: GAE scaleability limits

[Kaan Soral]

I couldn't read your entire post as the font is just preposterous

However, GAE becomes faster and cheaper scaling up, from experience

There are some minutely limits on some services that are hard to work 
around tho, so you will surely spend a lot of time adapting to GAE 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/4c05d560-2f59-4e96-bd87-c9c8b8e60afd%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: New Cloud Console

[Kaan Soral]

Since I have years of GAE experience, I just let things roll, so I didn't 
complain when the new Console rolled out, just waiting things to play out 
is more logical and sane

However, I will just say this, the new console needs:

*1) Logs in the Main Menu*
*2) Usage History in the Main Menu, including Yesterday, maybe unofficially 
until the calculations are final*
*3) Daily Usage Total in the Dashboard *(the charges exist individually, 
yet I'm not interested in calculating the total on my own, I just use the 
calculation to quickly check that the app isn't consuming too much)

I know the main comeback is to suggest a public tracker issue, yet I had 
issues in the past, some recent, nothing happened, so I will just throw 
these here

2 years ago I wrote my own Logs Viewer, as even back then the Logs Viewer 
was unreliable, so I only used the console to check the usage and queues, 
now I don't/can't use the new Console at all, if one of my apps went 
haywire and spent $200 a day, I would refute the charges :) - as I'm 
practically blind since this new Console is so impractical and cryptic

Just list everything in one place goddammit

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/bc558030-a8fe-4a6c-8edb-eb54d2c417ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: What is storage engine underneath the so called "Search API" engine

[Kaan Soral]

Thanks Nick

I had the same impression, especially since I haven't seen anyone complain 
about reaching the limits
Sounds good

Just found this by the 
way: 
https://console.developers.google.com/appengine/search?project=__app_id__=default=6
My largest index seems to be 613MB (although I have millions of stuff 
indexed), nice to see a standalone index tracker
200GB would probably go a long way when the time comes

On Monday, January 25, 2016 at 7:48:53 PM UTC+2, Nick (Cloud Platform 
Support) wrote:
>
> Hey Kaan,
>
> I'm not sure I can comment on some of the colourful analogies employed in 
> your commentary, but I can provide the advice that you can increase your 
> quota with the Search API up to 200GB of storage by submitting a request 
> <https://support.google.com/cloud/contact/cloud_search_api_index_size_increase_request_form>,
>  
> and the documentation 
> <https://cloud.google.com/appengine/docs/quotas?hl=en#search> also 
> mentions that all the quotas can be negotiated by discussing with a support 
> representative. 
>
> We certainly aim to make all our services scalable, and a general rule to 
> keep in mind in terms of realizing this abstract statement is that it's 
> often not the service itself which has a limit, but what customers who are 
> willing to communicate with us have expressly communicated was of interest 
> to them in terms of scaling potential. We have in the past, and will again 
> in the future, "moved mountains" for customers who are interested and 
> willing to make a foray into a larger-scale deployment of any service, once 
> both technical and billing details have been discussed. 
>
> This is of course still very abstract, and should not be taken as any kind 
> of concrete commitment - my advice to anybody who might read this thread in 
> future, and those in it right now is merely to get in touch with a support 
> representative and explore your options. We are generally very flexible :)
>
> Have a great day!
>
> On Saturday, January 23, 2016 at 4:59:56 AM UTC-5, Kaan Soral wrote:
>>
>> Search API seems like "a deal with the devil" to me, for some time you 
>> get all that you could dream for, but at one point, you suffer and die 
>> painfully
>>
>> I haven't reached the limits of Search API myself yet, I haven't seen 
>> anyone that did either, but once you reach the limits, you can't add any 
>> more documents, the index gets stuck - I'm guessing, at that point, one 
>> must implement another system to start deleting unimportant documents to 
>> make room for more important stuff, this applies to my use case, but it 
>> might not apply to everyone's
>>
>> It would be great if Search API did this on it's own, if you reach the 
>> limit, send in an argument at document.add's to remove the least ranked 
>> document automatically
>>
>> Anyway, now that I've been using the Search API for some time, I regret 
>> some of the thing that I did with "datastore", I have complex and well 
>> calibrated indexes and filters, filled with workarounds and trade-off's
>>
>> At one point, I might re-visit them and just re-do things with Search API 
>> using a separate index for specific use cases
>>
>> TL;DR: Search API is awesome but look out for the limitations, it's 
>> definitely not scalable
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/006eecba-90fb-429e-80d8-4c54a055bc73%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

Re: [google-appengine] Re: What is storage engine underneath the so called "Search API" engine

[Kaan Soral]

Search API seems like "a deal with the devil" to me, for some time you get 
all that you could dream for, but at one point, you suffer and die painfully

I haven't reached the limits of Search API myself yet, I haven't seen 
anyone that did either, but once you reach the limits, you can't add any 
more documents, the index gets stuck - I'm guessing, at that point, one 
must implement another system to start deleting unimportant documents to 
make room for more important stuff, this applies to my use case, but it 
might not apply to everyone's

It would be great if Search API did this on it's own, if you reach the 
limit, send in an argument at document.add's to remove the least ranked 
document automatically

Anyway, now that I've been using the Search API for some time, I regret 
some of the thing that I did with "datastore", I have complex and well 
calibrated indexes and filters, filled with workarounds and trade-off's

At one point, I might re-visit them and just re-do things with Search API 
using a separate index for specific use cases

TL;DR: Search API is awesome but look out for the limitations, it's 
definitely not scalable

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/9960c249-55b6-4311-9666-80f327fda3b4%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Taskqueue Failing with Error Code 123 Before 10 Minute deadline

[Kaan Soral]

Apart from the main issue, deferred stores stuff in db if the payload 
overloads the taskqueue limit, so this process adds a bit of an overhead, 
but I would guess it's no more than 10 seconds (on the taskqueue side, it's 
db_pull + picker_deciphering), I haven't tested the overhead myself

I personally limit myself to 8-9 minutes to be on the safe side

On Wednesday, January 20, 2016 at 9:34:43 PM UTC+2, Michael Sander wrote:
>
> Hi Christian - Good thought. I took a look at the code in C:\Program Files 
> (x86)\Google\google_appengine\google\appengine\ext\deferred and nothing 
> jumped out at me as being different or special. The only major added 
> complexity is the introduction of pickle in the deferred library. But this 
> does not seem like a pickle issue.
>
> On Wednesday, January 20, 2016 at 2:19:12 PM UTC-5, Christian F. Howes 
> wrote:
>>
>> is there any chance that deferred tasks are somehow treated differently 
>> then regularly queued tasks?  in theory i would think they are just tasks 
>> with a special handler, but could that be why you are hitting an incorrect 
>> deadline?
>>
>> On Wednesday, January 20, 2016 at 7:27:01 AM UTC-8, Nicholas (Google 
>> Cloud Support) wrote:
>>>
>>> Thank you for posting your error logs. According to 'Google Cloud Status 
>>> ', there are no know outages for Task 
>>> Queues or App Engine modules at the moment.
>>>
>>> As you've mentioned, the default deadline for a automatic scaled modules 
>>> is 10 minutes. This is documented in 'Task deadlines 
>>> '.
>>>  
>>> Please note that the deadline can be as high as 24 hours for manual or 
>>> basic scaling modules. Also note that the DeadlineExceededException is 
>>> raised when the task's execution time nears the deadline, thus allowing you 
>>> to save your work or log something before the deadline is actually 
>>> exceeded. That being said, more information is required to get a better 
>>> idea of what might really be the culprit here:
>>>
>>>1. What is the .yaml configuration for the related task-handling 
>>>modules? (feel free to censor any project-sensitive data)
>>>2. What is the time frame? (When did this start; is it ongoing; if 
>>>not, when did it end)
>>>3. Have you encountered any other timeout or deadline exceeded 
>>>errors from other modules, including those unrelated to task queues?
>>>4. How is your queue.yaml configured 
>>> for 
>>>the queues experiencing these issues?
>>>5. Do retries 
>>>
>>> 
>>>  or 
>>>increased module scalability help reduce the frequency of these errors?
>>>
>>> Though the thread 'Suddenly App Engine not Serving Anymore: Error Code 
>>> 123 
>>> '
>>>  
>>> seems to describe similar errors, the scope of these errors and what 
>>> systems are causing them is unclear in said thread so I would advise to 
>>> continue here going forward.
>>>
>>> On Tuesday, January 19, 2016 at 9:01:51 PM UTC-5, Michael Sander wrote:

 Hi All,

 Over the past few days I've been getting a lot of Error Code 123 in my 
 Taksqueue. I understand that this error is a deadline, but Taskqueues are 
 supposed to live for 10 minutes, and I get this error far before that. It 
 is difficult to debug because these requests have practically no logging 
 information in them. My app normally spits out a lot of logs, so either 
 GAE 
 is dropping my logs or this crash occurs outside of my code.


1. 2016-01-19 20:36:45.991 /_ah/queue/deferred 500 190547ms 0kb 
 AppEngine-Google; 
(+http://code.google.com/appengine) module=default version=18

0.1.0.2 - - [19/Jan/2016:17:36:45 -0800] "POST /_ah/queue/deferred 
 HTTP/1.1" 500 0 "http://docketupdate.appspot.com/_ah/queue/deferred; 
 "AppEngine-Google; (+http://code.google.com/appengine)" 
 "docketupdate.appspot.com" ms=190547 cpu_ms=0 cpm_usd=0.65 
 queue_name=default task_name=37491651647795200701 exit_code=123 
 app_engine_release=1.9.31 
 instance=00c61b117c4864f5a842a8ae7ca496d20dbaa9f8 
 

2. E2016-01-19 20:36:45.991

Process terminated because the request deadline was exceeded. (Error 
 code 123)



 Here's another:



1. 2016-01-19 19:48:58.581 /_ah/queue/deferred 500 84488ms 0kb 
 AppEngine-Google; 
(+http://code.google.com/appengine) module=default version=18

0.1.0.2 - - 

[google-appengine] Re: Exceeded Soft Private memory usage of 128MB

[Kaan Soral]

Scenario A
1) Fetch 40 entities
2) Fetch an external url, takes 1-15 seconds
3) Put 40 entities async

Scenario B
1) Fetch 40 entities
2) Internal computation, no delay
3) Put 40 entities async

So the issue is seldom at A, yet it floods the logs at B
At both scenarios, multitude of these operations are ran, 40M entities for 
B, <<1M for A
(A is an hourly operation, while B is more of a one-time thing)

Since there is no observable data/operation loss, I didn't debug the issue

On Tuesday, January 19, 2016 at 11:41:46 PM UTC+2, Nick (Cloud Platform 
Support) wrote:
>
> Hey Kaan,
>
> Interested to try to analyze the source of the issue - what do you mean by 
> "if the routine is slow" - does this refer to the cron schedule being a 
> larger interval, or the time for requests to execute? Are you able to run 
> memory diagnostics in-request to log the rate at which memory gets used-up?
>
> On Friday, January 15, 2016 at 6:08:49 PM UTC-5, Kaan Soral wrote:
>>
>> Interesting, thanks for the link
>>
>> The app I mentioned used db, my new app uses ndb tho I don't experience 
>> memory issues frequently with it
>>
>> The app that experiences memory leaks has crons that pulls and processes 
>> 40 entities at a time, entities are <1MB, so when instances are flooded 
>> with these crons, memory issues occur frequently
>> Interestingly, if the routine is slow, they occur less, but if the 
>> routine is fast, it occurs every 30-40 requests
>>
>> On Friday, January 15, 2016 at 9:42:23 PM UTC+2, Christian F. Howes wrote:
>>>
>>> FYI, if you use NDB there is a memory leak:  
>>> https://code.google.com/p/googleappengine/issues/detail?id=9610  the 
>>> google folks are having trouble pinning it down, but it's there.  i just 
>>> let my instances restart as they run out of memory as i have not been able 
>>> to find my other memory leaks (i'm sure i have more then just the NDB leak).
>>>
>>> On Friday, January 15, 2016 at 9:50:24 AM UTC-8, Kaan Soral wrote:
>>>>
>>>> I have an app that kills instances after just 30 requests sometimes, it 
>>>> functions perfectly tho, the instances die pretty gracefully, my 
>>>> assumption 
>>>> is that they don't take anything with them, rather handle all their 
>>>> requests and die after that
>>>>
>>>> I don't upgrade to F2 as it increases the costs significantly, it's 
>>>> harder to utilise F2 instances efficiently
>>>>
>>>> My .02
>>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/2f7c95d6-3066-4c65-8ae4-5ef193d45d45%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Suddenly App Engine not Serving Anymore: Error code 123

[Kaan Soral]

Yes, manually slowing down the app almost diminishes these

But I still see 40-50 of them every now and then

On Wednesday, January 20, 2016 at 3:53:17 AM UTC+2, Michael Sander wrote:
>
> I've randomly started seeing these too. Did you do anything that worked?
>
> On Thursday, June 18, 2015 at 7:50:52 AM UTC-4, Kaan Soral wrote:
>>
>> I also see some outages from my logs
>>
>> The worst part is that the SLA refunds are not automatic, I have too many 
>> things on my plate and the SLA refund/request process is too criptic
>>
>> On Wednesday, June 17, 2015 at 8:01:36 PM UTC+3, Karl-Heinz Müller wrote:
>>>
>>> I have a few app engines running, serving different domain names using 
>>> the same application code php. 
>>>
>>> Two of them suddenly stopped serving content. In the log file I see the 
>>> following error message:
>>>
>>> I 12:56:46.855 This request caused a new process to be started for your 
>>> application, and thus caused your application code to be loaded for the 
>>> first time. This request may thus take longer and use more CPU than a 
>>> typical request for your application.
>>> E 12:56:46.855 Process terminated because the request deadline was 
>>> exceeded. (Error code 123)
>>>
>>>
>>> I have another app engine running the same application without any issues 
>>> and accessing the same data storage unit. Wondering why two shut down and 
>>> the other not. 
>>>
>>>
>>> No changes have been done. The incident started today 12:00
>>>
>>>
>>> Thanks
>>>
>>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/c2c77401-4e21-4702-b25e-b5d78c131fb8%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Exceeded Soft Private memory usage of 128MB

[Kaan Soral]

I have an app that kills instances after just 30 requests sometimes, it 
functions perfectly tho, the instances die pretty gracefully, my assumption 
is that they don't take anything with them, rather handle all their 
requests and die after that

I don't upgrade to F2 as it increases the costs significantly, it's harder 
to utilise F2 instances efficiently

My .02

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/cc753dd1-86ee-4af1-9a2e-94e48f176682%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Exceeded Soft Private memory usage of 128MB

[Kaan Soral]

Interesting, thanks for the link

The app I mentioned used db, my new app uses ndb tho I don't experience 
memory issues frequently with it

The app that experiences memory leaks has crons that pulls and processes 40 
entities at a time, entities are <1MB, so when instances are flooded with 
these crons, memory issues occur frequently
Interestingly, if the routine is slow, they occur less, but if the routine 
is fast, it occurs every 30-40 requests

On Friday, January 15, 2016 at 9:42:23 PM UTC+2, Christian F. Howes wrote:
>
> FYI, if you use NDB there is a memory leak:  
> https://code.google.com/p/googleappengine/issues/detail?id=9610  the 
> google folks are having trouble pinning it down, but it's there.  i just 
> let my instances restart as they run out of memory as i have not been able 
> to find my other memory leaks (i'm sure i have more then just the NDB leak).
>
> On Friday, January 15, 2016 at 9:50:24 AM UTC-8, Kaan Soral wrote:
>>
>> I have an app that kills instances after just 30 requests sometimes, it 
>> functions perfectly tho, the instances die pretty gracefully, my assumption 
>> is that they don't take anything with them, rather handle all their 
>> requests and die after that
>>
>> I don't upgrade to F2 as it increases the costs significantly, it's 
>> harder to utilise F2 instances efficiently
>>
>> My .02
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/1e2b4cf4-ae21-48c0-8698-43925d68a380%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Having lost all my data

[Kaan Soral]

https://code.google.com/p/vosao/source/browse/trunk/pom.xml

You are listed as a "Maintenance Developer"

Shut up and maintain your shit properly

On Thursday, January 14, 2016 at 8:02:15 PM UTC+2, Emilio S wrote:
>
> Google just KILLED ME !
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/5d8143f9-7fdc-421d-a005-ff55012493ef%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Having lost all my data

[Kaan Soral]

Funnily, google search might help you recover your data, just search your 
website on "Google.com", recover anything that is "Cached"

I'm sure we all lost stuff similar to you, It's part of computer 
engineering and computer life in general, you lose stuff, you learn not to 
lose stuff, you learn how to backup

Just don't cry and blame your mistakes on others please, I personally 
visited all the /setup's on Vosao's example websites, none of them had your 
issue

Other than these, I'm sympathetic to your issue, no one wants to experience 
what you experienced
My suggestion is, jump to the 5th step, accept what happened and move on 
from there

On Monday, January 11, 2016 at 1:44:40 PM UTC+2, Emilio S wrote:
>
> Because Google has killed me !
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/322f1192-b871-4253-9942-221f2594e2a1%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Having lost all my data

[Kaan Soral]

I'm pretty sure you are one of a kind

I feel lucky that I got a glimpse of such a unique personality

It would be even more unique if your age is 20+

On Tuesday, January 12, 2016 at 5:32:47 PM UTC+2, Emilio S wrote:
>
>
>
> Google killed me ! and 3 of its Sith applaud 
>
> Be sure that another will come soon 
>
> Dark side of computing 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/7a721d04-03c4-4a13-a825-569d14423155%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[google-appengine] Re: Google Cloud Platform wants to hear from you

[Kaan Soral]

Juan, in the beginning, the easiest way is to setup GAE Launcher (the name 
might have changed, it's the Java/Python/etc. SDK) - create a project from 
there, and upload from the program

On Tuesday, January 12, 2016 at 1:10:53 AM UTC+2, Juan Juarez wrote:
>
> Hello Katie,
>
> I am new to the Google App engine. I have been trying to learn to use the 
> GAE for about a week now and i've had some trouble getting off the ground. 
>
> I am not a seasoned developer, but i was able to follow along with the 
> google apps script docs much more easily. 
>
> This might seem like a trivial problem for most here, but i have yet to 
> understand how to upload an app to the GAE. 
>
> There's a line in the hello world tutorial that says to invoke this 
> command: "appcfg.py -A YOUR_PROJECT_ID update helloworld/". How do I 
> invoke this command? Should I invoke it in the GAE Launcher? Should I 
> invoke it in windows' command prompt? Somewhere else?
>
> From a novice's standpoint this seems to be a big barrier of entry...
>
> From all of the research that I've done on databases the Datastore sounds 
> great for what i'm trying to accomplish, but I need to find better 
> documentation on how to use it.
>
> On Wednesday, April 15, 2015 at 12:37:31 PM UTC-5, Katie Ball (Google 
> Cloud Support) wrote:
>>
>> Hi,
>>
>> My name is Katie, and I am on the Google Cloud Platform technical support 
>>  team.
>>
>> This message is to Google Cloud Platform community members, especially if 
>> you are newer to GCP. I would like to know what our team can do to help you 
>> have a better and more enjoyable experience during the first days on GCP. 
>>
>> Did you need technical support?  If so, I’d like to hear all about it.
>>
>> I’d also like to know:
>>
>>- 
>>
>>What did you find most difficult about the first-time user experience?
>>- 
>>
>>Where did you get stuck?
>>
>>
>> Please reply to the group with your answers or any ideas you have on how 
>> the technical support team can help new customers get familiar with GCP. 
>>
>> And as a thank you for the great ideas, we will be giving away support 
>> coupons worth $450 (equivalent to 3 months of silver support) to 5 lucky 
>> community members who post a response. Please make sure to reply before 
>> April 22nd. 
>>
>> Thanks for your insights, and cloud on!
>>
>> Katie
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at https://groups.google.com/group/google-appengine.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/google-appengine/6565470a-01e0-455c-9f95-38de693ac215%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.